skill-creator-trigger-eval-gotchas

Diagnose false failures in skill-trigger evaluation loops and description tuning workflows.

Updated Nov 18, 2025
One-click install
npx skills add https://github.com/cajias/claude-skills --skill skill-creator-trigger-eval-gotchas
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: skill-creator-trigger-eval-gotchas
Source: https://github.com/cajias/claude-skills/tree/main/plugins/cc-authoring/skills/skill-creator-trigger-eval-gotchas
Command: npx skills add https://github.com/cajias/claude-skills --skill skill-creator-trigger-eval-gotchas

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you diagnose why a skill-trigger evaluation loop reports misleading results, such as 0% recall, wrong project roots, or failures caused by probe shadowing rather than the skill itself.

Core Features & Use Cases

  • Debug trigger-eval failures: Identify why a skill appears not to trigger even when verbatim trigger phrases should match.
  • Resolve environment issues: Fix common execution problems like missing Python dependencies, cwd-sensitive root resolution, and Anthropic authentication errors.
  • Avoid parallel-worker collisions: Prevent sibling probe shadowing that can make recall look artificially zero in multi-worker runs.
  • Validate real recall: Compare harness results with ground-truth Claude invocations before changing a skill description.

Quick Start

Use this skill to inspect your trigger-eval setup, identify whether shadowing or environment issues are causing bad scores, and determine the correct fix before rerunning the evaluation.

Frequently Asked Questions about skill-creator-trigger-eval-gotchas

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
Why does my Claude Code skill trigger evaluation report 0% recall even when verbatim trigger phrases should match?

Probe shadowing in parallel worker runs can make recall look artificially zero even when verbatim trigger phrases should match. Diagnosing whether the failure is caused by sibling probe collisions or the skill itself prevents unnecessary description tuning.

How do I debug wrong project root resolution in a skill evaluation loop?

Wrong project root resolution in a skill evaluation loop happens when cwd-sensitive root discovery fails. You must verify the current working directory matches the expected project root before running probe-based recall tests, ensuring ground-truth validation against real skill triggers.

What is probe shadowing in multi-worker skill evaluation and how do I prevent it?

Probe shadowing in multi-worker skill evaluation occurs when sibling probes collide during parallel runs, causing misleading false failures. Preventing parallel-worker collisions ensures that harness results accurately reflect real recall instead of artificially zero scores from overlapping probe execution.

How do I validate real skill recall before changing a skill description?

Validating real skill recall requires comparing harness results with ground-truth Claude invocations before adjusting descriptions. This ground-truth validation confirms whether a trigger-eval failure reflects an actual skill issue or an environment problem like Anthropic authentication errors.

Does skill trigger evaluation work with missing Python dependencies in Claude Code environments?

Skill trigger evaluation does not work correctly with missing Python dependencies in Claude Code environments, as execution problems cause false failures. Resolving environment issues like missing dependencies and Anthropic authentication errors is required before accurate probe-based recall testing.