skills.rest
ats-kinoshita-iso
ats-kinoshita-isoCOMMUNITY
Explore all skills by this author →
GitHub

bootstrap-failures

Convert bug, incident, and manual test failures into JSON evaluation criteria.

Updated Apr 11, 2026
Search Tags:#monitoring#incidents#bug reports#failures#tests#eval
One-click install
npx skills add https://github.com/ats-kinoshita-iso/trine-eval --skill bootstrap-failures
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bootstrap-failures
Source: https://github.com/ats-kinoshita-iso/trine-eval/tree/main/skills/bootstrap-failures
Command: npx skills add https://github.com/ats-kinoshita-iso/trine-eval --skill bootstrap-failures

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps teams incorporate real failure instances from bug reports, incidents, and manual tests into their eval suite, enhancing test relevance and coverage.

Core Features & Use Cases

  • Ingestion of Failure Data: Import failure cases from bug trackers, QA notes, and support tickets into a structured catalog.
  • Automated Catalog Construction: Convert raw failure reports into JSON entries with relevant metadata for evaluation purposes.
  • Use Case: Start your eval development by seeding 20-50 real failure scenarios to reflect actual user issues and system flaws.

Quick Start

Gather failure reports and add each as an entry in failure-catalog.json, then run /harness-kickoff to integrate them into your eval workflow.

Frequently Asked Questions about bootstrap-failures

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I use real production incident data to improve my eval suite?▼

You can use real production incident data to improve your eval suite by importing bug reports, QA notes, and support tickets into a structured failure catalog. This Skill converts raw failure cases into JSON entries with relevant metadata for evaluation criteria.

What is the best way to seed test scenarios with actual system failures?▼

The best way to seed test scenarios with actual system failures is to capture bug, incident, and manual test failures, then convert them into evaluation criteria. Starting with 20-50 real failure scenarios ensures your tests reflect actual user issues and system flaws.

How do I convert raw bug reports into an evaluation catalog for testing?▼

To convert raw bug reports into an evaluation catalog, add each failure case as an entry in failure-catalog.json. This process automatically structures the raw failure data into JSON entries with relevant metadata for your testing workflow.

Does this approach require a specific bug tracker to integrate failure data?▼

No specific bug tracker is required to integrate failure data. The Skill is designed to ingest failure cases from various sources, including bug trackers, QA notes, and support tickets, converting them into a structured catalog for evaluation purposes.

Why should I update my failure catalog with live production issues?▼

You should update your failure catalog with live production issues to ensure your eval suite maintains high test relevance and coverage. Continuously integrating real-world failure data reflects actual user issues and adapts the testing feedback to current system flaws.

Related Skills

Explore More Software Engineering Skills→
robikscuberobikscube

product-data-import

Convert unstructured FF&E product notes, CSV files, and JSON data into structured schedule entries.

Community
Intermediate
xu-xiangxu-xiang
1.8k

eval-harness

Define, execute, and report capability and regression evals for Claude Code sessions.

Community
Intermediate
rohit-vc26rohit-vc26

superpowers-tdd

Guide engineers through a test-first workflow with red-green-refactor cycles.

Community
Intermediate
Dokkabei97Dokkabei97
2

test-generator

Generate unit and integration tests for Jest, Vitest, pytest, and Kotest.

Community
Intermediate