harness:setup

To set up automated baseline evaluations, you can use a configuration tool that explores your codebase, detects entry points and virtual environments, configures LangSmith, generates test data, and writes a baseline evaluation file.

Configuring LangSmith safely involves interactive prompts that handle API keys securely without exposing secrets. This ensures your agent optimization environment is authenticated and ready for baseline scoring without compromising credentials.

Yes, the setup tool applies to Python and mixed-language LLM agent projects. It automatically discovers dependency manifests, detects virtual environments, and identifies run commands across the codebase to prepare for optimization.

You can generate test data by spawning a testgen agent during setup. The tool can automatically generate evaluation datasets or import existing ones based on interactive configuration prompts and safe defaults.

The setup process prefers project virtual environments to ensure dependencies are isolated and run commands execute correctly. It warns about missing venvs to prevent environment conflicts before building and running setup scripts.

If you do not specify an entry point, project discovery automatically detects it from your codebase. You are then interactively prompted to confirm or customize the entry point, optimization goals, and evolution mode with safe defaults.