skills.rest
BrennonTWilliams
BrennonTWilliamsCOMMUNITY
Explore all skills by this author →
GitHub

create-eval-from-issues

Generate FSM evaluation harnesses and DSL verification tasks from issue tracking data.

7|2|Updated Jan 3, 2026
Search Tags:#automation#testing#evaluation#dsl#fsm#harness
One-click install
npx skills add https://github.com/BrennonTWilliams/little-loops --skill create-eval-from-issues
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: create-eval-from-issues
Source: https://github.com/BrennonTWilliams/little-loops/tree/main/.gemini/skills/create-eval-from-issues
Command: npx skills add https://github.com/BrennonTWilliams/little-loops --skill create-eval-from-issues

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill eliminates the manual effort required to create robust, user-perspective evaluation harnesses for software features, ensuring that your AI agents are verified against real-world acceptance criteria rather than just code diffs.

Core Features & Use Cases

  • FSM Harness Generation: Automatically synthesizes ready-to-run FSM eval harness YAMLs from issue IDs, including proof-first gate integration.
  • DSL Task Synthesis: Generates fill-in-the-blank, correction, and transformation tasks for loop and issue files to stress-test agent reasoning.
  • Use Case: When you have a set of new feature issues, use this skill to instantly generate a verification harness that exercises the features as a real user would, ensuring quality before deployment.

Quick Start

Use the create-eval-from-issues skill to generate an FSM eval harness for the specified issue IDs.

Frequently Asked Questions about create-eval-from-issues

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate FSM evaluation harnesses from issue tracking data?▼

FSM evaluation harnesses are automatically synthesized from issue IDs as ready-to-run YAML configurations, including proof-first gate injection for rigorous feature validation against real-world acceptance criteria.

What are DSL-based verification tasks for testing AI agent reasoning?▼

DSL-based verification tasks are fill-in-the-blank, correction, and transformation tasks generated for loop and issue files to stress-test and verify AI agent reasoning against acceptance criteria.

Do I need ll-issues and ll-loop CLI tools to generate eval harnesses?▼

Yes, integration with ll-issues and ll-loop CLI tools is required to resolve issue metadata and validate generated harness configurations during FSM eval harness and DSL task generation.

Can I automate verification for multiple feature issues at once?▼

Yes, the automated harness generation supports multi-issue workflows, allowing you to instantly synthesize verification configurations that exercise features as a real user would from a set of issue IDs.

What is the best way to verify AI agents against real-world acceptance criteria instead of code diffs?▼

Automating FSM eval harness generation from issue tracking data verifies AI agents against user-perspective acceptance criteria, eliminating manual effort and ensuring feature quality before deployment.

When should I use automated DSL task synthesis for software feature validation?▼

Use DSL task synthesis when you need to stress-test agent reasoning with fill-in-the-blank, correction, and transformation tasks, ensuring new features are rigorously validated before deployment.

Related Skills

Explore More Software Engineering Skills→
BrennonTWilliamsBrennonTWilliams
7

ll-capture-issue

Create project issues from natural language input or conversation context.

Community
Intermediate
virajpvirajp
1

verify

Verify deployed environments against product blueprints by checking service health and acceptance criteria.

Community
Advanced
poz110poz110
4

jira-mcp-setup

Manage Jira issues via MCP/curl with multi-source credentials.

Community
Advanced
SunS1eep1ngSunS1eep1ng

verification-loop

Automates build, type check, lint, tests, security scan, and diff review for code.

Community
Advanced