skills.rest
saintgo7
saintgo7COMMUNITY
Explore all skills by this author →
GitHub

gem-llm-test-inference

Run inference and tool-calling smoke tests against a local GEM-LLM vLLM endpoint.

Updated Apr 4, 2026
Search Tags:#tool calling#streaming#openai sdk#vllm#inference testing#json mode
One-click install
npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-test-inference
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: gem-llm-test-inference
Source: https://github.com/saintgo7/claude-skills/tree/main/gem-llm-test-inference
Command: npx skills add https://github.com/saintgo7/claude-skills --skill gem-llm-test-inference

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Verifying that a locally deployed GEM-LLM vLLM endpoint can reliably produce correct chat responses and complete tool-calling flows before you rely on it in production.

Core Features & Use Cases

  • Inference smoke tests: Runs basic chat completions to confirm the model is responding as expected.
  • Tool-calling verification: Exercises function/tool calls with tool_choice=auto and checks tool call outputs.
  • Streaming validation: Confirms streaming behavior works and returns tokens incrementally.
  • OpenAI SDK compatibility checks: Runs equivalent tests via the OpenAI client pointed at the local vLLM server.
  • Logging for reporting: Stores inference transcripts in jsonl files for later review (TTFT, tokens/sec, finish_reason, tool_call success).

Quick Start

Run the skill and ask for an inference and tool-calling test by executing: ./install.sh gem-llm-test-inference.

Frequently Asked Questions about gem-llm-test-inference

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I test vLLM inference and tool calling on a local endpoint?▼

Test vLLM inference by executing chat completion smoke tests, verifying tool_call outputs, and validating streaming responses. Save transcripts as jsonl files to report TTFT, tokens/sec, and finish_reason metrics for local validation.

How do I validate OpenAI SDK compatibility with a local vLLM server?▼

Validate OpenAI SDK compatibility by running equivalent chat completion and tool-calling tests via the OpenAI client pointed at the local vLLM server. Confirm the /v1/chat/completions endpoint processes function calls and streaming tokens correctly.

Does vLLM support JSON mode and tool_choice for chat completions?▼

vLLM supports JSON mode and tool_choice by exercising function calls with tool_choice=auto. Verify tool call success and finish reasons by saving inference transcripts as jsonl files for later review and reporting.

What is the best way to verify streaming behavior returns tokens incrementally in vLLM?▼

Verify vLLM streaming behavior by confirming the endpoint returns tokens incrementally during chat completions. Execute streaming validation tests to ensure the local server handles real-time response generation correctly before production deployment.

How do I log inference transcripts and tool call success metrics for vLLM?▼

Log vLLM inference transcripts by saving chat completion and tool-calling outputs as jsonl files. Capture TTFT, tokens/sec, finish_reason, and tool_call success metrics for later review and local validation reporting.

Related Skills

Explore More Software Engineering Skills→
EndUser123EndUser123

ai-groq

Streams Llama, Mixtral, Gemma, and DeepSeek inference via Groq's OpenAI-compatible endpoint.

Community
Advanced
qcmuuqcmuu

serving-llms-vllm

Deploy vLLM inference endpoints with quantization and tensor parallelism.

Community
Advanced
saintgo7saintgo7

llm-serving-performance-tuning

Diagnose and resolve LLM serving bottlenecks across vLLM, FastAPI, and database.

Community
Advanced
excattexcatt
14

pytest-runner

Executes Python test suites with pytest and generates detailed coverage reports.

Community
Advanced