experiment-analyst

Analyze agent experiment runs to identify success drivers and failure modes.

13|3|Updated Jan 29, 2026

Search Tags:#ai agents #sqlite #tool usage #experiment analysis #failure modes #success determinants #workflow reconstruction

Ready to run

npx skills add https://github.com/danicat/skills --skill experiment-analyst

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: experiment-analyst
Source: https://github.com/danicat/skills/tree/main/experiment-analyst
Command: npx skills add https://github.com/danicat/skills --skill experiment-analyst

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps you understand why AI agents succeed or fail in specific experiment runs by turning raw run logs into evidence-backed success determinants and behavioral patterns.

Core Features & Use Cases

Evidence-based experiment deconstruction: Extracts performance overview, tool usage breakdown, and top failure signals from a Tenkai experiments SQLite database.
Success determinants via tool correlation: Identifies tools that correlate with higher success or failure rates by comparing tool usage across successful vs failed runs.
Targeted behavioral deep dives: Reconstructs run workflows (message/tool steps) to compare a winning pattern against a failure loop for specific alternatives.

Quick Start

Run the analysis for experiment ID 12 by executing: python3 agents/tenkai/.gemini/skills/experiment-analyst/scripts/analyze_experiment.py 12

Frequently Asked Questions about experiment-analyst

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I diagnose why my AI agent experiment runs succeed or fail?▼

To diagnose why AI agent experiment runs succeed or fail, analyze run_results and run_events logs in a SQLite database to extract tool usage patterns, detect errors, and identify success determinants. This process reconstructs workflows to compare winning patterns against failure loops.

What is the best way to correlate tool usage with success rates across multiple experiment alternatives?▼

Correlating tool usage with success rates across experiment alternatives requires aggregating tool correlation data from SQLite run logs. By comparing tool usage breakdowns across successful versus failed runs, you classify shell-command categories and derive performance summaries to pinpoint behavioral patterns.

How do I reconstruct AI agent workflows from SQLite logs to analyze failure modes?▼

Reconstructing AI agent workflows from SQLite logs involves sequencing message and tool steps from run_events data. This allows you to trace specific behavioral deep dives, compare a winning pattern against a failure loop, and identify the exact point where the run_events diverged into a failure mode.

Do I need a specific database setup to analyze agent experiment logs?▼

Analyzing agent experiment logs requires a Tenkai SQLite database containing run_results and run_events logs. You must configure the TENKAI_DB_PATH environment variable to point to this database before executing the analysis scripts to compute performance summaries and tool correlations.

Can I compare multiple experiment alternatives using run_events and run_results logs?▼

Yes, you can compare multiple experiment alternatives by querying run_results and run_events logs stored in SQLite. The analysis computes performance overviews, classifies shell-command categories, and aggregates tool correlations with success to print recommendations derived from the compared patterns.