create-inspect-task

Create custom inspect-ai evaluation tasks automatically by extracting configuration from existing experiments. The Skill guides you through dataset, model, and objective setup, then generates production-ready task files and documentation in minutes instead of hours.

Yes. The Standalone Task Builder mode walks you through creating evaluation tasks from the ground up with best practices, including solver chain design, scoring selection, and parameterization, without requiring prior experiment data.

It produces a complete runnable task file, accompanying documentation, a create-inspect-task.log for tracking, and clear usage instructions. All code is ready to execute in your evaluation pipeline immediately.

The Experiment-Guided mode automatically parses experiment_summary.yaml files to pre-populate evaluation settings, eliminating manual configuration entry and reducing setup time.

After generating your evaluation task with scoring configuration, you can immediately run inspect-ai evaluations against your models to collect performance metrics and iterate on task parameters.