Benchmark Gemma-4-31B AWQ-4bit on vLLM for tensor-parallel configuration.
npx skills add https://github.com/air-gapped/skills --skill vllm-gemma-4-31b
Please help me install this Agent Skill. Skill: vllm-gemma-4-31b Source: https://github.com/air-gapped/skills/tree/main/.claude/skills/vllm-gemma-4-31b Command: npx skills add https://github.com/air-gapped/skills --skill vllm-gemma-4-31b
Provide an operating-point reference for deploying Gemma-4-31B AWQ-4bit on vLLM that helps platform engineers balance throughput, latency, and memory constraints across short- and long-context workloads.
Configure Gemma-4-31B AWQ-4bit on a multi-GPU vLLM deployment using the LIGHT or PUSH recipes provided in this guide.
High-intent search queries and answers about installing and using this skill.
Optimize Gemma-4-31B AWQ-4bit deployment on vLLM by applying empirically-grounded operating points that guide tensor-parallel selection and configuration for throughput planning and latency-sensitive inference across short and long context workloads.
Size KV-cache for long context vLLM workloads by referencing the provided operating points, which align max_model_len and batch considerations to prevent HBM-bandwidth saturation and balance memory constraints.
HBM-bandwidth saturation creates inference bottlenecks by limiting memory transfer speeds; this guide identifies these workload-specific trade-offs to help platform engineers select appropriate tensor-parallel configurations.
Reproduce benchmark numbers for Gemma-4-31B using the provided references to hbm-saturation and bench-numbers, enabling reproducible preflight assessments and hardware-aware benchmarking for your deployment tuning.
Choose LIGHT versus PUSH tensor-parallel recipes based on workload-specific trade-offs for latency and throughput, using this decision guide to balance operating points against memory constraints across short and long context inputs.
Chat-template alignment is required for deploying Gemma-4-31B on vLLM, ensuring correct formatting of short and long context workloads alongside proper KV-cache sizing and tensor-parallel configuration.
ricableQuantize large language models into 4-bit AWQ checkpoints for reduced GPU memory.
agentsopeDiagnose vLLM inference issues and tune deployment configurations for GPU workloads.
cabeznoDeploy an OpenAI-compatible LLM inference server using vLLM.
cryptopafiServe large language models with high-throughput inference for OpenAI-compatible endpoints.
