compare-model-sweep

A Pareto frontier for model evaluation summarizes quality, cost, latency, and reliability trade-offs to guide routing decisions. It identifies the set of candidate models where no single model outperforms another across all metrics simultaneously.

Yes, you can compare local and gateway model setups in the same evaluation. The sweep harness normalizes routes, endpoints, and prompts to ensure an apples-to-apples comparison across different deployment environments.

Run a deterministic model sweep by providing a workload with a fixed set of candidate models and a shared evaluation harness. The skill enforces consistent seeds, prompts, and timeouts to produce a structured export of per-candidate results.

A Pareto report for model selection includes quality, cost, latency, and reliability metrics. It aggregates per-candidate results from the evaluation harness to help engineering teams decide which models to deploy in production.

Yes, the model evaluation harness supports structured export of results. It outputs normalized routes and per-candidate evaluation data alongside the final Pareto summary to provide a complete decision aid for deployment.