response-compliance

Runs the official OpenResponses compliance test suite against a Response API endpoint.

Updated Jan 27, 2026
One-click install
npx skills add https://github.com/SmallAi-API/smaihub --skill response-compliance-smallai-api
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: response-compliance
Source: https://github.com/SmallAi-API/smaihub/tree/main/.agents/skills/response-compliance
Command: npx skills add https://github.com/SmallAi-API/smaihub --skill response-compliance-smallai-api

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Verifying that a custom Response API implementation conforms to the OpenResponses specification requires running many endpoint checks manually, which is slow and error-prone. This Skill automates running the official compliance test suite against a local or remote endpoint and surfaces schema and streaming failures. ## Core Features & Use Cases - Compliance Test Execution: Clones and runs the official openresponses/openresponses CLI test runner against your endpoint with a single command. - Targeted Test Filtering: Run individual tests such as basic-response, streaming-response, tool-calling, or multi-turn via the --filter flag. - Flexible Auth Modes: Supports dev mock-user headers, API key authentication, and custom auth header/value pairs. - Use Case: After implementing the /api/v1/responses route in LobeHub, run the suite in verbose mode to diagnose why the streaming-response test fails with a missing output items error. ## Quick Start Run the OpenResponses compliance tests against my local dev server on port 3010 using the dev auth header and show verbose output for any failures.

Frequently Asked Questions about response-compliance

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run OpenResponses compliance tests against my API endpoint?

Run bun run test:response-compliance from the packages/openapi directory with APP_URL set to your server. Pass authentication via --auth-header and --api-key flags, and use --filter to run specific tests like basic-response or streaming-response.

How do I test only specific Response API compliance cases?

Use the --filter flag with comma-separated test IDs such as basic-response, streaming-response, system-prompt, tool-calling, image-input, or multi-turn. This runs only the selected tests instead of the full suite.

What authentication modes does the compliance test support?

Three modes are supported: dev mode using the lobe-auth-dev-backend-api header with --no-bearer, standard API key auth with --api-key lb-xxx, and custom auth via --auth-header plus --api-key. The API_KEY environment variable works as an alternative to the flag.

Why does the compliance test fail with 'Failed to parse JSON'?

This error means authentication failed and the server returned an HTML redirect instead of JSON. Verify your auth header and API key are correct, and confirm the dev server has ENABLE_MOCK_DEV_USER=true set when using dev mode.

What does 'Response has no output items' mean in test results?

This failure indicates the LLM execution logic is not yet implemented in your responses service, so the endpoint returns a response without output items. Check the responses.service.ts implementation to ensure generation completes.