llm-test

Provides guidelines for writing tests for LLM-related functionality.

12.1k|1.5k|Updated Jul 12, 2023

Search Tags:#model selection #Guidelines #Testing #Test Fixtures #lLM #AI provider

One-click install

npx skills add https://github.com/elie222/inbox-zero --skill llm-test

Or copy as Structured Prompt for Agent▼

Please help me install this Agent Skill.
Skill: llm-test
Source: https://github.com/elie222/inbox-zero/tree/main/.claude/skills/llm-test
Command: npx skills add https://github.com/elie222/inbox-zero --skill llm-test

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Writing tests for LLM-powered features is tricky because model behavior varies across providers and configurations. This Skill gives developers clear guidelines so LLM tests stay consistent, provider-agnostic, and focused on the right behavior.

Core Features & Use Cases

LLM Testing Guidelines: Directs the AI to read and follow the shared testing conventions defined in the project's testing documentation.
Fixture Configuration Rules: Instructs that fixture users should not be pinned to a specific aiProvider or aiModel unless the test explicitly covers model-selection behavior.
Use Case: When adding a test for an AI email-drafting feature, the Skill ensures the test fixtures leave provider and model fields null so the test remains portable across LLM backends.

Quick Start

Write a test for the LLM reply-drafting feature following the llm-test guidelines.

Frequently Asked Questions about llm-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I write tests for LLM-powered features?▼

Follow the shared testing guidelines referenced by this Skill, which define conventions for LLM test structure. Keep tests provider-agnostic so they run consistently regardless of the underlying AI model.

Should test fixtures specify an AI provider or model?▼

No, fixture users should leave aiProvider and aiModel set to null in normal LLM tests. Only pin these values when the test explicitly covers model-selection behavior.

When should a test pin a specific aiProvider or aiModel?▼

Pin a specific provider or model only when the test is explicitly about model-selection behavior. For all other LLM tests, leaving these fields null keeps fixtures portable and avoids coupling tests to one backend.

Where are the detailed LLM testing conventions defined?▼

The detailed conventions live in the project's testing documentation at .claude/skills/testing/llm.md, which this Skill instructs the AI to read and follow before writing LLM tests.