skills.rest
CYP0630
CYP0630COMMUNITY
Explore all skills by this author →
WebsiteGitHub

report_evaluation

Evaluate a ticker/model run by verifying weekly metrics with MCP.

Updated May 3, 2026
Search Tags:#markdown#mcp#market-data#duckdb#evaluate#ground-truth
One-click install
npx skills add https://github.com/CYP0630/agentic_benchmark_single_agent --skill report-evaluation-cyp0630
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: report_evaluation
Source: https://github.com/CYP0630/agentic_benchmark_single_agent/tree/main/skills/report_evaluation
Command: npx skills add https://github.com/CYP0630/agentic_benchmark_single_agent --skill report-evaluation-cyp0630

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires duckdb, numpy, pandas, pandas_ta, fastmcp, and includes scripts (resource) components.

What problem does it solve?

Evaluates a completed report_generation run for one ticker / model combination. Uses the report_evaluation_mcp server to read generated WEEKLY Markdown reports and offline DuckDB market data, then scores the run across five dimensions and aggregates run-level backtest metrics. Writes one JSON result plus one structured Markdown summary to results/report_evaluation/.

Core Features & Use Cases

  • End-to-end evaluation: reads weekly reports, computes ground-truth metrics, and aggregates run-level results without network data.
  • Ground-truth verification: reuses the same logic as the report generator (via MCP) to ensure fair scoring.
  • Output artifacts: writes a JSON payload and a human-readable Markdown summary to results/report_evaluation/.

Quick Start

Configure inputs (TICKER, TARGET_MODEL, REPORTS_ROOT, DB_PATH, OUTPUT_ROOT) and run the evaluation workflow to generate results in the expected directory.

Frequently Asked Questions about report_evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I evaluate weekly trading report metrics against ground-truth market data offline?▼

To evaluate weekly trading report metrics offline, read generated Markdown reports and verify them against offline DuckDB market data using MCP tools. This reuses report generation logic to compute ground-truth metrics and aggregate run-level backtest performance.

Can I score a completed backtest run for a single ticker without network access?▼

Yes, you can score a single ticker and model pair without network access by reading offline DuckDB data and weekly Markdown reports. The evaluation relies entirely on local MCP tools to compute metrics and generate run-level results.

How does MCP verify trading report performance using DuckDB and pandas?▼

MCP verifies trading report performance by reading weekly Markdown reports and querying offline DuckDB market data. It applies the same logic as the report generator using pandas and pandas_ta to calculate ground-truth metrics and aggregate backtest performance.

Do I need DuckDB and pandas installed to evaluate model run metrics?▼

Yes, you need DuckDB, pandas, numpy, pandas_ta, and fastmcp installed. These dependencies enable the MCP server to read offline market data, compute ground-truth metrics, and aggregate run-level backtest results for evaluation.

What is the best way to aggregate run-level backtest results from weekly Markdown reports?▼

The best way to aggregate run-level backtest results is using an MCP-powered evaluation workflow that reads weekly Markdown reports and offline DuckDB data. It scores the run across five dimensions and writes a JSON payload plus a structured Markdown summary.

Why should I use offline DuckDB data instead of live market data for report evaluation?▼

Using offline DuckDB data ensures fair scoring by reusing the same logic as the report generator without network modifications. This approach guarantees consistent ground-truth verification and accurate run-level backtest metric aggregation.

Related Skills

Explore More Data & Analytics Skills→
xueqingpengxueqingpeng
11

report_evaluation

Recompute ground-truth metrics from offline DuckDB data for weekly report evaluations.

Community
Advanced
smallnestsmallnest
190

markitdown

Convert PDFs, Word, Excel, HTML, and images with OCR into Markdown.

Community
Advanced
HaoxuanLiTHUAIHaoxuanLiTHUAI
34

Parameter Recovery Checker

Run parameter recovery studies to assess identifiability of computational cognitive models.

Community
Advanced
SuperagentsysSuperagentsys
650

quantitative-backtest

Design and validate backtest simulations for trading strategies.

Community
Advanced