harness:setup

What problem does it solve?

This Skill automates the tedious, error-prone process of preparing an LLM agent project for automated optimization by exploring the codebase, selecting an entry point, ensuring LangSmith integration, and producing a baseline evaluation.

Core Features & Use Cases

• Project discovery: Detects entry points, frameworks, virtual environments, and common dependency manifests to determine how to run the agent.
• Interactive configuration: Prompts the user to confirm or customize entry point, optimization goals, evolution mode, and test data sourcing, with safe defaults.
• Test data and setup automation: Generates or imports evaluation datasets, spawns a testgen agent when needed, builds and runs setup.py, and writes .evolver.json with sensible settings.
• Operational safeguards: Prefers project venvs, warns about missing venvs, and enforces safe LangSmith API key handling without exposing secrets.

Quick Start

Run the harness:setup skill to detect your agent, configure LangSmith, generate or import test data, and produce a baseline evaluation.