gen-eval-scenario

Create gen-eval scenario YAML files with schema validation.

4|1|Updated Jan 16, 2026
One-click install
npx skills add https://github.com/jankneumann/agentic-coding-tools --skill gen-eval-scenario
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: gen-eval-scenario
Source: https://github.com/jankneumann/agentic-coding-tools/tree/main/.agents/skills/gen-eval-scenario
Command: npx skills add https://github.com/jankneumann/agentic-coding-tools --skill gen-eval-scenario

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires pyyaml, and includes scripts (resource) components.

What problem does it solve?

Manually writing gen-eval scenario YAML files is time‑consuming, error‑prone, and hard to keep consistent across testing contexts.

Core Features & Use Cases

  • Descriptor Auto‑Detection: Finds the appropriate interface descriptor without user input.
  • Coverage Analysis: Shows existing scenario coverage and suggests under‑tested categories.
  • Interactive Prompting: Guides the user through scenario intent, category, interfaces, and behavior.
  • Validated Output: Generates a schema‑compliant YAML file and runs a Pydantic validation step.

Use case: A developer wants to add a new failure scenario for the authentication boundary; this skill walks them through selecting transports, defining steps, and producing a ready‑to‑run YAML file.

Quick Start

Run /gen-eval-scenario --category auth-boundary --type failure to create a new scenario file.

Frequently Asked Questions about gen-eval-scenario

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate gen-eval scenario YAML files for interface testing?

You can generate gen-eval scenario YAML files interactively by specifying scenario intent, category, interfaces, and behavior; the tool then auto-detects interface descriptors and outputs a schema-validated YAML file ready for evaluation.

What is the best way to ensure my testing scenario YAML files pass schema validation?

The best way to ensure testing scenario YAML files pass schema validation is to use an interactive scenario generator that runs a Pydantic validation step automatically, verifying descriptor analysis and schema compliance before outputting the final file.

How do I find under-tested categories in my existing testing scenario coverage?

To find under-tested categories in your existing testing scenario coverage, use a coverage analysis tool that examines current scenario definitions, suggests missing categories, and auto-detects appropriate interface descriptors across transports.

Can I create failure scenarios for authentication boundaries without manually writing YAML?

Yes, you can create failure scenarios for authentication boundaries without manually writing YAML by using interactive scenario generation, which walks you through selecting transports and defining steps to produce a ready-to-run, schema-compliant YAML file.

Do I need pyyaml installed to interactively craft scenario descriptors?

Yes, you need pyyaml installed to interactively craft scenario descriptors, as it is the required dependency for parsing and generating the validated YAML output files used in gen-eval testing workflows.