Analyze agent experiment runs to identify success drivers and failure modes.
npx skills add https://github.com/danicat/skills --skill experiment-analyst
Please help me install this Agent Skill. Skill: experiment-analyst Source: https://github.com/danicat/skills/tree/main/experiment-analyst Command: npx skills add https://github.com/danicat/skills --skill experiment-analyst
This Skill helps you understand why AI agents succeed or fail in specific experiment runs by turning raw run logs into evidence-backed success determinants and behavioral patterns.
Run the analysis for experiment ID 12 by executing: python3 agents/tenkai/.gemini/skills/experiment-analyst/scripts/analyze_experiment.py 12
High-intent search queries and answers about installing and using this skill.
To diagnose why AI agent experiment runs succeed or fail, analyze run_results and run_events logs in a SQLite database to extract tool usage patterns, detect errors, and identify success determinants. This process reconstructs workflows to compare winning patterns against failure loops.
Correlating tool usage with success rates across experiment alternatives requires aggregating tool correlation data from SQLite run logs. By comparing tool usage breakdowns across successful versus failed runs, you classify shell-command categories and derive performance summaries to pinpoint behavioral patterns.
Reconstructing AI agent workflows from SQLite logs involves sequencing message and tool steps from run_events data. This allows you to trace specific behavioral deep dives, compare a winning pattern against a failure loop, and identify the exact point where the run_events diverged into a failure mode.
Analyzing agent experiment logs requires a Tenkai SQLite database containing run_results and run_events logs. You must configure the TENKAI_DB_PATH environment variable to point to this database before executing the analysis scripts to compute performance summaries and tool correlations.
Yes, you can compare multiple experiment alternatives by querying run_results and run_events logs stored in SQLite. The analysis computes performance overviews, classifies shell-command categories, and aggregates tool correlations with success to print recommendations derived from the compared patterns.
dcasotaGenerate Hugo-compatible Markdown blog posts from Photon OS commit data.
braintrustdataTrace Claude Code sessions, turns, and tool calls to Braintrust.
lazyFrogLOLRecover data from corrupted or encrypted SQLite WAL files.
algorithmbasicsMonitor production apps for console errors, performance regressions, and page failures.
