Verified:GitHub

zicato-tune-scoring

Calibrate zicato scoring contracts and promotion gates for generation tournaments.

4|2|Updated May 14, 2026

Search Tags:#multi-agent systems #zicato #tournament evaluation #scoring calibration #promotion gates #drift loss #pass-rate monotonicity

Ready to run

npx skills add https://github.com/pedapudi/zicato --skill zicato-tune-scoring

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: zicato-tune-scoring
Source: https://github.com/pedapudi/zicato/tree/main/skills/zicato-tune-scoring
Command: npx skills add https://github.com/pedapudi/zicato --skill zicato-tune-scoring

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps operators tune zicato's scoring contract when generation rankings or tournament promotion decisions do not match human judgment.

Core Features & Use Cases

Weight Calibration: Adjust drift, severity, kind, judge, plan-revision, runtime, and pass-rate weights to reflect evaluation priorities.
Advanced Scoring Controls: Configure declarative transforms, custom scalar and drift-reducer plugins, namespace objectives, and diff-complexity rules.
Promotion Gate Tuning: Set promotion margins, pass-rate monotonicity, namespace safeguards, and holdout behavior to balance improvement against regressions.
Use Case: When a candidate generation reduces drift but worsens an important rubric or schema outcome, use this Skill to revise scoring weights and promotion gates so tournament decisions reflect operator priorities.

Quick Start

Use the zicato tuning skill to review the current scoring.json and recommend calibrated weights and promotion-gate settings for the next evaluation epoch.

Frequently Asked Questions about zicato-tune-scoring

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I calibrate scoring weights when tournament rankings don't match human judgment?▼

Calibrate scoring weights by adjusting drift, severity, judge, and pass-rate values in your scoring contract to align tournament rankings with operator priorities. This corrects mismatches between automated promotion decisions and human evaluation outcomes.

What is a scoring contract in multi-agent generation tournaments?▼

A scoring contract is a configuration file defining drift-loss weighting, pass-rate evaluation, custom judges, and nonlinear transforms. It governs how generation tournaments rank and promote harness revisions according to operator priorities.

How do I configure promotion gates to prevent regressions in generation tournaments?▼

Configure promotion gates by setting promotion margins, enforcing pass-rate monotonicity, and applying namespace safeguards. These settings balance candidate improvements against potential regressions during tournament evaluation epochs.

Can I use custom scoring plugins and nonlinear transforms with zicato evaluation?▼

Yes, zicato evaluation supports declarative transforms, custom scalar plugins, and drift-reducer plugins. You can configure these advanced scoring controls alongside namespace objectives and diff-complexity rules in your contract.

Do I need a valid scoring.json file before starting a new evaluation epoch?▼

Yes, a valid scoring.json contract is required before starting a new evaluation epoch. You must carefully validate deterministic scoring behavior to ensure tournament rankings and promotion decisions function correctly.

Why does my candidate generation reduce drift but worsen rubric outcomes?▼

Candidate generations can reduce drift while worsening rubric outcomes when scoring weights are misaligned with operator priorities. Revise scoring weights and promotion gates in your contract so tournament decisions reflect desired evaluation tradeoffs.