skills.rest
vllm-project
vllm-projectOFFICIAL
Explore all skills by this author →
GitHub

vllm-omni-test

Generates and runs pytest tests for vllm-omni with CI-aligned levels, markers, and Buildkite wiring.

6.5k|1.6k|Updated Sep 11, 2025
Search Tags:#regression testing#Buildkite#ci#Test Generation#E2E Testing#vllm#PyTest
One-click install
npx skills add https://github.com/vllm-project/vllm-omni --skill vllm-omni-test
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: vllm-omni-test
Source: https://github.com/vllm-project/vllm-omni/tree/main/.claude/skills/vllm-omni-test
Command: npx skills add https://github.com/vllm-project/vllm-omni --skill vllm-omni-test

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Writing tests for vllm-omni requires knowing its L1-L4 test levels, pytest marker taxonomy (core_model, advanced_model, full_model, omni/tts/diffusion), fixture conventions, and Buildkite pipeline files. This Skill encodes all of those rules so generated tests land in the right directory, carry the right markers, and wire into the correct CI pipeline on the first try.

Core Features & Use Cases

  • Test level and marker selection: Classifies work as bugfix regression, feature coverage, or perf/benchmark, then maps it to L1-L4 levels with the correct level, model-type, hardware, and cross-cutting markers.
  • Scenario-based test skeletons: Provides ready patterns for offline inference e2e, online serving e2e, documentation example tests, perf benchmarks, and invalid-parameter validation, including fixture scope rules and mocking conventions (mocker/monkeypatch only, no unittest.mock).
  • CI wiring guidance: Shows how to register tests in Buildkite files (test-ready.yml for L1/L2, test-merge.yml for L3, test-nightly.yml for L4, test-weekly.yml for invalid-param), including source_file_dependencies and agents/plugins blocks.
  • Use Case: A contributor fixing a bug in Qwen3-Omni serving asks for a regression test; the Skill classifies it, generates an L2 online-serving test with core_model + omni markers, and outputs copy-paste local and CI-like pytest commands.

Quick Start

Ask the assistant to generate a regression test for a specific vllm-omni issue or PR, stating the model, scenario, and available hardware.

Frequently Asked Questions about vllm-omni-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a regression test for a vllm-omni bug fix?▼

Start from a minimal failing scenario and prefer the narrowest stable L1 CPU test; escalate to L2/L3 only when the bug needs real weights or serving. The Skill classifies the fix as required, recommended, or not_needed for regression coverage before writing the test.

How do I choose pytest markers for vllm-omni tests?▼

Pick one level marker (core_model for L1/L2, advanced_model for L3, full_model for L4), exactly one model-type marker (omni, tts, or diffusion), plus hardware marks via @hardware_test helpers. Baseline smoke tests often carry both core_model and advanced_model on the same function.

Where do invalid parameter validation tests go in vllm-omni?▼

Invalid-parameter and HTTP 4xx tests belong in tests/dfx/reliability/invalid_param_test/, in the route-matching script, using send_*_http_request with err_code and err_message. They run in test-weekly.yml with pytest.mark.slow plus H100 or L4 marks, never in e2e or nightly pipelines.

Can I use unittest.mock in vllm-omni L1 unit tests?▼

No. L1 tests must use the pytest-mock mocker fixture or monkeypatch for env and attribute substitution. Imports of unittest.mock, @patch decorators, and with patch(...) blocks are prohibited so patches auto-revert with the test lifecycle.

How are vllm-omni tests wired into Buildkite CI?▼

L1/L2 tests go in test-ready.yml, L3 in test-merge.yml, and L4 in test-nightly.yml, each with source_file_dependencies and full agents/plugins blocks. Diffusion L4 expansion tests are split into X2I(&A&T) and X2V nightly groups by output modality.

When should I use function-scoped omni_server fixtures?▼

Use omni_server_function or omni_runner_function only when each test needs a fresh server or runner instance, such as reliability, sleep/wakeup, or crash/restart scenarios. Default to module-scoped omni_server and omni_runner to amortize model initialization across tests.

Related Skills

Explore More Software Engineering Skills→
hsliuustc0106hsliuustc0106
84

vllm-omni-review

Review vllm-omni PRs for code quality, tests, and security issues.

Community
Advanced
samjonestersamjonester
5

pr-ci-autofix

Monitor PR CI status and triage failures across GitHub and Buildkite.

Community
Advanced
go-to-kgo-to-k
97

verify-pr

Run typecheck, lint, build, tests, and CI checks before merging pull requests.

Community
Advanced
togethercomputertogethercomputer

write-sglang-test

Automate SGLang CI/UT test setup with standardized templates and fixtures.

Official
Intermediate