Benchmark policy performance and guide champion/challenger evolution.
npx skills add https://github.com/Hmbown/plugins --skill benchmark-hmbown
Please help me install this Agent Skill. Skill: benchmark Source: https://github.com/Hmbown/plugins/tree/main/plugins/autodialectics/skills/benchmark Command: npx skills add https://github.com/Hmbown/plugins --skill benchmark-hmbown
Benchmark policies and drive champion/challenger evolution across iterative evaluation cycles.
Run autodialectics benchmark to execute a suite and then invoke evolve_policy to create a challenger for comparison.
High-intent search queries and answers about installing and using this skill.
To benchmark AI policy performance, execute the autodialectics benchmark command to run a suite and quantify improvements. This establishes a baseline for iterative evaluation cycles across your policy workflows.
Champion challenger evolution creates modified policy challengers to compare against the current champion policy. You then promote the challenger or rollback to the champion based on quantified performance differences during benchmarking.
Automating policy benchmarking workflows requires installing autodialectics-mcp on PATH. You need access to CLI commands including benchmark, evolve_policy, promote_policy, and rollback to execute the end-to-end workflow.
Promote or rollback a policy challenger after comparing its benchmarked performance against the champion. Promote the challenger if it demonstrates quantifiable improvements, otherwise rollback to maintain the current champion policy.
Yes, the workflow supports iterative policy evaluation cycles. You can continuously benchmark policy performance, drive evolution by creating new challengers, and make promotion or rollback decisions across multiple evaluation suites.
HmbownReplay stored Autodialectics pipeline runs with an optional policy override.
prashsubManage AI prompts with MLflow versioning and Unity Catalog runtime retrieval.
prashsubGenerate and validate Genie Space benchmark questions with ground truth SQL.
niznik-devCreate custom inspect-ai evaluation tasks with experiment-guided and standalone modes.
