vllm-observability

To monitor vLLM latency and GPU saturation, you need standardized Prometheus metrics, PromQL templates, and Grafana dashboards. This Skill provides those observability components to surface latency, throughput, queue depth, and GPU diagnostics for production deployments.

Setting up observability for a vLLM Kubernetes deployment involves standardizing metrics exposed at /metrics with vLLM-specific labels. This Skill supplies the required dashboards, alerting recipes, and tracing instructions to enable rapid incident triage and capacity planning.

Yes, you can use Grafana and Prometheus to trace vLLM queue depth issues by correlating metrics with OpenTelemetry traces. This Skill defines concrete metric names and tracing guidance to correlate traces with metrics for end-to-end debugging.

This observability toolkit supports both on-prem clusters and Kubernetes environments. It standardizes metrics and dashboards for operators and SREs running vLLM, focusing on latency and queue depth diagnostics regardless of the specific infrastructure.

Raw vLLM metrics often mislead incident triage because they lack unified alerting recipes and dashboard context. You can fix this by standardizing metrics with PromQL templates and correlating traces with metrics using the references provided in this Skill.