backtest-diagnose

Diagnose and fix failing backtests by reading artifacts and code.

6.1k|1.2k|Updated Jun 9, 2022
One-click install
npx skills add https://github.com/charliedream1/ai_quant_trade --skill backtest-diagnose-charliedream1
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: backtest-diagnose
Source: https://github.com/charliedream1/ai_quant_trade/tree/main/a_%E5%85%A8%E7%BD%91%E4%BC%98%E7%A7%80%E8%B5%84%E6%BA%90/10_%E5%A4%A7%E6%A8%A1%E5%9E%8B/07_skill%E5%8C%85/vibe_trading_skills/backtest-diagnose
Command: npx skills add https://github.com/charliedream1/ai_quant_trade --skill backtest-diagnose-charliedream1

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Diagnose and fix failing or underperforming backtests by identifying root causes and guiding targeted repairs.

Core Features & Use Cases

  • Diagnostic workflow: Read existing artifacts (artifacts/metrics.csv, equity.csv, trades.csv), inspect code (code/signal_engine.py, config.json), classify the issue using the error taxonomy, apply fixes with edit_file, and re-run the backtest to verify improvements.
  • Hard-gate verification: Enforces essential checks (artifacts/metrics.csv non-empty, artifacts/equity.csv non-empty, trade_count > 0, no NaN in equity, exit_code == 0) before approving a fix.
  • Post-fix validation: Re-runs the backtest to ensure stability and correctness, and reports new metrics for comparison.
  • Hard-Gate Checklist and Taxonomy references: Aligns with the provided taxonomy to ensure robust diagnosis.

Quick Start

Provide your backtest artifacts and code; I will diagnose the failure and guide you through targeted fixes.

Frequently Asked Questions about backtest-diagnose

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fix a backtest that fails with runtime errors or shows zero trades?

Fix failing backtests by reading artifacts like metrics.csv and equity.csv, inspecting signal_engine.py, classifying the issue via taxonomy, applying targeted code fixes, and re-running to verify improvements against hard-gate checks.

What is the best way to diagnose abnormal performance in a backtesting pipeline?

Diagnosing abnormal performance requires reading output artifacts such as trades.csv and equity.csv to identify anomalies like NaN values or zero trade counts, classifying the root cause using an error taxonomy, and applying targeted code fixes to the signal engine or configuration.

How do I verify backtest fixes after modifying signal_engine.py or config.json?

Verify backtest fixes by enforcing a hard-gate checklist that confirms metrics.csv and equity.csv are non-empty, trade_count is greater than zero, equity contains no NaN, and exit_code equals zero, followed by re-running the backtest to report new metrics for comparison.

Does backtest diagnosis work with standard artifacts like metrics.csv and equity.csv?

Yes, backtest diagnosis works by reading standard pipeline artifacts including artifacts/metrics.csv, artifacts/equity.csv, and trades.csv, alongside code files like code/signal_engine.py and config.json, to classify issues and apply fixes using edit_file.

What are the limitations when trying to fix underperforming backtests automatically?

Fixing underperforming backtests requires complete artifact files and accessible code; if metrics.csv or equity.csv are empty, or if trade_count remains zero after applying fixes, the hard-gate verification will block approval and require further manual diagnosis.