vllm-tool-calling

To stop vLLM tool-call leaks, apply a 3-stage defense combining server parsing, model-format validation, and client-side fallback promotion to strip leaked Hermes XML, Qwen3, and bare-JSON patterns from content while mapping them to OpenAI-standard tool_calls.

vLLM tool calling fails after model upgrades because the selected vLLM --tool-call-parser no longer matches the new model's tokenizer chat-template output format, requiring parser-to-model mapping validation to ensure format alignment.

Validate vLLM streaming tool_calls by enforcing automated smoke tests that check both stream and non-stream chat completions, ensuring tool_calls are present and that tool syntax does not leak into the content field across chunk boundaries.

Yes, vLLM supports OpenAI-compatible tool_calls for both streaming and non-streaming chat completions, but requires hardening through parser-to-model mapping and client-side promotion to detect Hermes, Qwen3, and bare-JSON formats reliably.

The best way to detect Hermes and Qwen3 tool-call formats in vLLM is using client-side promotion logic that identifies these specific XML patterns alongside bare-JSON cases, then promotes them into standard tool_calls while stripping leaked content.

No, a single vLLM tool-call parser cannot safely cover multiple model formats, because each model's tokenizer chat-template outputs differ, requiring strict parser-to-model mapping and fallback promotion to handle format regressions.