skills.rest
GCWing
GCWingCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

agent-eval-canvas

Generates incident-review style Canvas reports diagnosing single agent evaluation trajectories.

1.9k|214|Updated Feb 2, 2026
Search Tags:#root cause#trajectory analysis#incident review#agent evaluation#tool call audit#LLM debugging#canvas report
One-click install
npx skills add https://github.com/GCWing/BitFun --skill agent-eval-canvas
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: agent-eval-canvas
Source: https://github.com/GCWing/BitFun/tree/main/src/crates/assembly/core/builtin_skills/agent-eval-canvas
Command: npx skills add https://github.com/GCWing/BitFun --skill agent-eval-canvas

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Debugging a single failed agent run from raw traces is slow and error-prone. This Skill turns one agent case — its trajectory, tool calls, evidence, and verdict — into a structured diagnostic report that pinpoints the decisive step, separates root causes from propagated errors, and proposes concrete repairs.

Core Features & Use Cases

  • Trajectory Diagnosis: Renders a step-by-step timeline with OK/WARN/ERROR/ROOT_CAUSE/PROPAGATED labels and a critical-step deep dive backed by trace evidence.
  • Root Cause & Propagation Analysis: Classifies concrete failure modes (planning, tool choice, retrieval, memory contamination, etc.) and maps error propagation chains across steps.
  • Tool, Cost & Safety Audit: Audits tool selection, arguments, result use, latency, tokens, duplicate work, risky actions, and guardrail behavior.
  • Use Case: After a benchmark case fails, feed the trace and expected result to the Skill to get a Canvas report showing the verdict, the decisive failing step, why later errors are downstream, and a repair backlog grouped by Prompt, Planner, Tool, Memory/RAG, Runtime, Evaluator, and Guardrail.

Quick Start

Analyze this agent run trace and create an eval canvas report showing the verdict, critical failure step, root cause, and repair plan.

Frequently Asked Questions about agent-eval-canvas

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I analyze a failed agent run trace?▼

Provide the case evidence — trajectory steps, tool calls, expected and actual results, and verdict — and the Skill generates a Canvas report with a step timeline, critical failure step deep dive, root cause classification, and a repair plan grouped by system layer.

How to find the root cause of an agent evaluation failure?▼

The report separates root-cause errors from propagated errors by tracing the chain from the first failure to the final result. It classifies concrete failure modes like planning, tool choice, retrieval, or observation understanding rather than attributing everything to model hallucination.

Can I analyze a successful agent run, not just failures?▼

Yes. For successful cases the report highlights the critical success step — the moment the agent made the run robust, such as verifying evidence, recovering from an empty result, or correcting a plan — instead of failure analysis.

What if the agent trace or expected result is missing?▼

The Skill asks for missing evidence unless you want a best-effort report. In that case unsupported fields are marked unavailable and root-cause claims are kept conservative and labeled as hypotheses.

Does the report include cost and safety analysis?▼

Yes. It covers latency, steps, LLM and tool calls, tokens, duplicate work, and invalid steps, plus risky commands, sensitive data exposure, prompt injection, and guardrail or confirmation handling when the evidence supports it.

Related Skills

Explore More Software Engineering Skills→
opendatahub-ioopendatahub-io
36

eval-run

Execute skills against test cases, score outputs with judges, and generate run summaries.

Official
Advanced
spencerpaulyspencerpauly
672

parallel-test-fixing

Distribute failing test files across parallel subagents for Jest, Vitest, and Pytest.

Community
Advanced
paulinghampaulingham
1

internal-eval-run

Orchestrate isolated inner-pipeline eval runs with concurrency and structured JSON reporting.

Community
Advanced
patronus-aipatronus-ai
2

llm-evaluation

Compute automated metrics and aggregate scores for LLM evaluation suites.

Official
Advanced