report_evaluation

To evaluate weekly reports against ground-truth data, recompute metrics offline using DuckDB and MCP tools, then output a JSON payload and Markdown summary via upsert_evaluation.py for per-report scoring and backtesting.

Yes, you can backtest ticker and model combinations offline. The process reads offline DuckDB market data and generated Markdown reports to perform run-level backtesting without requiring internet access.

Automated report scoring works by recomputing ground-truth metrics from offline DuckDB data using the generation's MCP logic. It scores runs across five dimensions and aggregates results into JSON and Markdown artifacts.

You need the report_evaluation MCP server, access to a DuckDB database, and the generated Markdown report artifacts to execute the evaluation and produce scoring results.

The limitation of using offline DuckDB data for backtesting is that no internet access is permitted during evaluation. The process strictly relies on pre-existing offline data and generated report artifacts to compute ground-truth metrics.

Yes, report_evaluation produces a machine-readable JSON artifact alongside a human-readable structured Markdown summary. Both are written to the results directory for auditing and comparison purposes.