trtllm-serve-config-guide

What problem does it solve?

This Skill removes the trial-and-error of finding a good starting configuration for trtllm-serve by grounding the output YAML in the repository’s checked-in TensorRT-LLM configs and deployment guidance.

Core Features & Use Cases

• Source-backed config generation: Produces a starting trtllm-serve --config YAML by matching the user’s model + serving constraints to checked-in configs, avoiding speculative and out-of-scope modes by default.
• Objective-preserving selection: Keeps the user’s latency/throughput intent (e.g., Min Latency, Balanced, Max Throughput) by selecting configs using database profile labels when available.
• Guardrailed adjustments: Updates only scenario-dependent fields (like batch/token/seq limits and KV-cache settings) after reading the relevant model deployment docs, and flags any inferred/interpolated fields as unverified.

Quick Start

Ask for a non-speculative, single-node PyTorch aggregate in-flight-batching serve YAML for your model and GPU with your target concurrency and input/output lengths, specifying whether you want Min Latency, Balanced, or Max Throughput.