gem-llm-test-inference

Run inference and tool-calling smoke tests against a local GEM-LLM vLLM endpoint.

Updated Apr 4, 2026

Search Tags:#tool calling #streaming #openai sdk #vllm #inference testing #json mode

Ready to run

npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-test-inference

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: gem-llm-test-inference
Source: https://github.com/saintgo7/claude-skills/tree/main/gem-llm-test-inference
Command: npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-test-inference

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Verifying that a locally deployed GEM-LLM vLLM endpoint can reliably produce correct chat responses and complete tool-calling flows before you rely on it in production.

Core Features & Use Cases

Inference smoke tests: Runs basic chat completions to confirm the model is responding as expected.
Tool-calling verification: Exercises function/tool calls with tool_choice=auto and checks tool call outputs.
Streaming validation: Confirms streaming behavior works and returns tokens incrementally.
OpenAI SDK compatibility checks: Runs equivalent tests via the OpenAI client pointed at the local vLLM server.
Logging for reporting: Stores inference transcripts in jsonl files for later review (TTFT, tokens/sec, finish_reason, tool_call success).

Quick Start

Run the skill and ask for an inference and tool-calling test by executing: ./install.sh gem-llm-test-inference.

Frequently Asked Questions about gem-llm-test-inference

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I test vLLM inference and tool calling on a local endpoint?▼

Test vLLM inference by executing chat completion smoke tests, verifying tool_call outputs, and validating streaming responses. Save transcripts as jsonl files to report TTFT, tokens/sec, and finish_reason metrics for local validation.

How do I validate OpenAI SDK compatibility with a local vLLM server?▼

Validate OpenAI SDK compatibility by running equivalent chat completion and tool-calling tests via the OpenAI client pointed at the local vLLM server. Confirm the /v1/chat/completions endpoint processes function calls and streaming tokens correctly.

Does vLLM support JSON mode and tool_choice for chat completions?▼

vLLM supports JSON mode and tool_choice by exercising function calls with tool_choice=auto. Verify tool call success and finish reasons by saving inference transcripts as jsonl files for later review and reporting.

What is the best way to verify streaming behavior returns tokens incrementally in vLLM?▼

Verify vLLM streaming behavior by confirming the endpoint returns tokens incrementally during chat completions. Execute streaming validation tests to ensure the local server handles real-time response generation correctly before production deployment.

How do I log inference transcripts and tool call success metrics for vLLM?▼

Log vLLM inference transcripts by saving chat completion and tool-calling outputs as jsonl files. Capture TTFT, tokens/sec, finish_reason, and tool_call success metrics for later review and local validation reporting.