Execute predefined challenges against an agent to evaluate performance and confidence calibration.
npx skills add https://github.com/tyevans/tackline --skill challenge-run
Please help me install this Agent Skill. Skill: challenge-run Source: https://github.com/tyevans/tackline/tree/main/skills/workflows/challenge-run Command: npx skills add https://github.com/tyevans/tackline --skill challenge-run
This Skill automates the execution and rigorous evaluation of agent challenges, providing objective performance metrics and insights into agent capabilities.
billing-agent using /challenge-gen, you can run /challenge-run billing-agent to see how effectively it handles those specific scenarios and identify areas for improvement.Use the challenge-run skill to execute challenges for the agent named 'billing-agent'.
High-intent search queries and answers about installing and using this skill.
Agent performance evaluation is automated by dispatching agents against predefined challenge sets to test output, trap detection, and confidence calibration. It measures how well self-reported confidence aligns with actual performance metrics.
Confidence calibration in agent evaluation measures the alignment between an agent's self-reported confidence and its actual performance results. It provides objective metrics to determine if an agent accurately assesses its own reliability.
Edge-case and commit-replay challenge strategies are executed by dispatching agents against predefined challenge files. The commit-replay strategy specifically leverages worktree isolation to safely replay scenarios during agent evaluation.
Yes, executing agent challenges requires predefined challenge files and agent definitions from team.yaml. These inputs are necessary for comprehensive assessment of performance, trap detection, and calibration metrics.
Automating agent challenge execution involves dispatching specified agents against predefined challenge sets to assess performance, hidden traps, and ground truth acceptance criteria. It provides objective metrics for identifying areas of improvement.
Agent challenge evaluations detect hidden traps by assessing agent output against predefined acceptance criteria and ground truth. Ensure your challenge files and team.yaml agent definitions are properly configured for comprehensive assessment.
tyevansAnalyze agent performance from git logs and learnings files to generate a struggle profile.
miles990Diagnose and optimize trading strategies through a six-phase iterative workflow.
supertyrelleDispatch targeted challenges to a named agent and evaluate output.
honysyangEvaluate Large Language Models with automated metrics and human evaluation frameworks.