Calibrate zicato scoring contracts and promotion gates for generation tournaments.
npx skills add https://github.com/pedapudi/zicato --skill zicato-tune-scoring
Please help me install this Agent Skill. Skill: zicato-tune-scoring Source: https://github.com/pedapudi/zicato/tree/main/skills/zicato-tune-scoring Command: npx skills add https://github.com/pedapudi/zicato --skill zicato-tune-scoring
This Skill helps operators tune zicato's scoring contract when generation rankings or tournament promotion decisions do not match human judgment.
Use the zicato tuning skill to review the current scoring.json and recommend calibrated weights and promotion-gate settings for the next evaluation epoch.
High-intent search queries and answers about installing and using this skill.
Calibrate scoring weights by adjusting drift, severity, judge, and pass-rate values in your scoring contract to align tournament rankings with operator priorities. This corrects mismatches between automated promotion decisions and human evaluation outcomes.
A scoring contract is a configuration file defining drift-loss weighting, pass-rate evaluation, custom judges, and nonlinear transforms. It governs how generation tournaments rank and promote harness revisions according to operator priorities.
Configure promotion gates by setting promotion margins, enforcing pass-rate monotonicity, and applying namespace safeguards. These settings balance candidate improvements against potential regressions during tournament evaluation epochs.
Yes, zicato evaluation supports declarative transforms, custom scalar plugins, and drift-reducer plugins. You can configure these advanced scoring controls alongside namespace objectives and diff-complexity rules in your contract.
Yes, a valid scoring.json contract is required before starting a new evaluation epoch. You must carefully validate deterministic scoring behavior to ensure tournament rankings and promotion decisions function correctly.
Candidate generations can reduce drift while worsening rubric outcomes when scoring weights are misaligned with operator priorities. Revise scoring weights and promotion gates in your contract so tournament decisions reflect desired evaluation tradeoffs.
pedapudiAssemble and validate zicato tournament contracts with cost and noise controls.
understudylabsAutomate GEPA-driven prompt and routing optimization with holdout protection.
theslashdojoIntegrate and manage Vite plugins in vite.config.ts.
pedapudiAudit zicato evaluation boards for harness correctness using canonical artifacts.
