failure-diagnosis

Diagnoses experiment failures and outputs ranked analysis with diagnostic steps.

Updated Mar 27, 2026

Search Tags:#diagnosis #experiments #failure-analysis #research-workflow #results-analysis #diagnostic-experiments

Ready to run

npx skills add https://github.com/EmaRimoldi/Claude-scholar-extended --skill failure-diagnosis-emarimoldi

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: failure-diagnosis
Source: https://github.com/EmaRimoldi/Claude-scholar-extended/tree/main/skills/failure-diagnosis
Command: npx skills add https://github.com/EmaRimoldi/Claude-scholar-extended --skill failure-diagnosis-emarimoldi

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Systematically identifies why research-level experiments fail to confirm hypotheses and provides a ranked set of failure modes, evidence, and targeted diagnostic steps.

Core Features & Use Cases

Expected vs. Observed Gap Analysis: structure the failure as what was predicted vs what was observed, plus the size of the gap.
Systematic Failure Mode Checklist: six modes (hypothesis, implementation, hyperparameters, data, metric, baseline) with evidence and next steps.
Evidence Assessment: for each failure mode, collect evidence for/against, likelihood, and cost to verify.
Diagnostic Experiments: propose small, targeted tests (<10% of compute) to isolate causes.
State Update: update experiment-state.json to "diagnosing" and reference the analysis.

Quick Start

Describe your observed vs. expected results and your available project files to generate a complete failure-diagnosis document.

Frequently Asked Questions about failure-diagnosis

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I diagnose why my research experiment failed to confirm its hypothesis?▼

Failure analysis for experiment failures evaluates six modes: hypothesis, implementation, hyperparameters, data, metric, and baseline. It assesses evidence for and against each mode, ranking them by likelihood and verification cost to pinpoint why observed results deviated from expectations.

What is the best way to isolate the root cause of a failed machine learning experiment?▼

Diagnostic experiments isolate root causes by running small, targeted tests that consume under 10% of available compute. These tests verify specific failure modes and produce an updated experiment-state.json file to guide your next research iteration.

When should I run a systematic failure mode analysis during my research workflow?▼

Run failure mode analysis after results analysis, during training, after baseline comparisons, or mid-iteration. It is applicable whenever observed results deviate from expected predictions and you need to inform hypothesis revision or experiment redesign.

How do I structure expected versus observed results to identify experimental errors?▼

Structure the failure by explicitly documenting what was predicted versus what was observed, then quantifying the size of the gap. This structured gap analysis serves as the foundation for the subsequent failure mode evaluation.

Can I use this diagnostic process for mid-iteration training failures without losing my experiment state?▼

Yes, the diagnostic process updates your existing experiment-state.json to a 'diagnosing' state and references the new analysis. This ensures your experiment state is preserved and accurately reflects the current diagnostic phase without losing prior data.