bmad-eval-runner

Run artifact and trigger evaluations for skills in an isolated environment.

Updated Dec 26, 2025
One-click install
npx skills add https://github.com/tiween/tiween --skill bmad-eval-runner-tiween
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: bmad-eval-runner
Source: https://github.com/tiween/tiween/tree/main/.agents/skills/bmad-eval-runner
Command: npx skills add https://github.com/tiween/tiween --skill bmad-eval-runner-tiween

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires python, claude-code, and includes scripts (resource) and references (resource) and assets (resource) components.

What problem does it solve?

This Skill provides a clean, isolated environment to run a skill's evaluations, ensuring the results reflect the skill itself and not external factors.

Core Features & Use Cases

  • Isolated Evaluation: Runs skill evaluations in a separate environment to avoid contamination from global configurations or previous runs.
  • Artifact Evals: Execute the skill against a prompt and grade the output against expectations.
  • Trigger Evals: Measure whether the skill's description triggers Claude on a given query.
  • Use Case: When you want to evaluate a skill, run its evaluations, benchmark it, validate triggers, or grade skill outputs.

Quick Start

Run the bmad-eval-runner skill with the path to the skill directory: bmad-eval-runner /path/to/skill

Frequently Asked Questions about bmad-eval-runner

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate a Claude skill in an isolated environment?

To evaluate a skill in an isolated environment, use a runner that executes artifact and trigger evaluations separately, ensuring results reflect the skill itself without contamination from global configurations or previous runs.

What is skill trigger evaluation and how does it work?

Skill trigger evaluation measures whether a skill's description triggers Claude on a given query. It runs in an isolated environment to test if the skill activates correctly for the intended inputs.

Do I need Python and Claude Code to run skill benchmarking?

Yes, you need Python and Claude Code installed to run skill benchmarking. The evaluation process executes scripts that require these dependencies to assess skill behavior and output grading.

How do I grade skill outputs against expected results?

To grade skill outputs against expectations, run an artifact evaluation. This process executes the skill against a specific prompt and compares the generated output to predefined expected results.

Why does my skill evaluation return inconsistent results between runs?

Inconsistent skill evaluation results often stem from environmental contamination. Running evaluations in an isolated environment prevents global configurations and previous execution states from affecting the skill's behavior.

What's the best way to benchmark a skill without external interference?

The best way to benchmark a skill without external interference is to run it in an isolated environment using a dedicated evaluation runner, which separates the execution context from global configurations.