What problem does it solve?
Provides a safe, repeatable smoke test for the Harness Evolver optimization pipeline to verify that tooling, CLI flags, and the evaluation flow are configured correctly before running full experiments.
Core Features & Use Cases
- Offline validation: Checks Python syntax, argparse help output, and skill-to-tool cross-references to catch integration issues without external services.
- Online mock evaluation: When LANGSMITH_API_KEY is present, runs a mock agent through setup, evaluation, results reading, and insight tracing to validate the end-to-end pipeline.
- Use Case: Run this skill after installing or changing tools to ensure setup.py, run_eval.py, read_results.py, and trace_insights.py behave as expected and produce readable outputs.
Quick Start
Run the dev:dry-run skill to smoke-test the evolve pipeline, validating tool syntax, argparse behavior, cross-references, and optional online evaluation when LANGSMITH_API_KEY is set.