skills.rest
veegee82
veegee82COMMUNITY
Explore all skills by this author →
GitHub

define-metric

Register agent-defined metrics as loss components with calibration gates.

2|Updated Apr 20, 2026
Search Tags:#calibration#define-metric#ldd#loss-framework#metric-algebra#bias-invariance
One-click install
npx skills add https://github.com/veegee82/loss-driven-development --skill define-metric-veegee82
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: define-metric
Source: https://github.com/veegee82/loss-driven-development/tree/main/skills/define-metric
Command: npx skills add https://github.com/veegee82/loss-driven-development --skill define-metric-veegee82

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This skill enables agents to introduce and manage new measurable objectives as first-class loss components, ensuring careful calibration before they influence decisions and deployments.

Core Features & Use Cases

  • Extend the loss framework with agent-defined metrics using a formal Metric Algebra.
  • Calibrate new metrics over multiple iterations and promote them to load-bearing only after they pass the calibration gate.
  • Use domain-specific rubrics to measure performance without touching LDD core.

Quick Start

Calibrate a new metric by registering it, logging observed-predicted pairs for several iterations, and promoting it once the calibration gate passes.

Frequently Asked Questions about define-metric

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a custom metric to a loss framework for agent training?▼

You can safely introduce new metrics as first-class loss components by registering them as advisory-only initially. This ensures they do not influence decisions or deployments until carefully calibrated and promoted through a validation gate.

What are the requirements to promote a calibrated metric to load-bearing status?▼

A custom metric becomes load-bearing only after passing a strict calibration gate requiring a minimum sample size of n ≥ 5 and a mean absolute error of mae ≤ 0.15 across multiple iterations.

How does metric algebra work for extending loss frameworks?▼

Metric algebra provides deterministic accessors and versioned changes to define new agent metrics as first-class loss components, allowing you to use domain-specific rubrics for performance measurement without touching the core framework.

Can I use domain-specific rubrics to measure agent performance without modifying core components?▼

Yes, you can measure performance using domain-specific rubrics without touching LDD core logic. New metrics are registered as advisory-only components, keeping the core framework stable while you calibrate and validate the new objectives.

Why does my newly defined agent metric fail to influence decision-gating?▼

Your newly defined metric cannot influence decision-gating because it is initially registered as advisory-only. It must pass intermediate calibration gates and meet strict promotion criteria before it can load-bear and influence compositional loss or budgeting.

Related Skills

Explore More Software Engineering Skills→
ResearAIResearAI
3.3k

baseline

Standardize a reference object for quest evaluation with plan and checklist requirements.

Community
Advanced
curiositechcuriositech
10

dag-confidence-scorer

Score DAG outputs with calibrated multi-factor confidence metrics.

Official
Advanced
xg-gh-25xg-gh-25
36

workspace-git

Automates git status, diff, commit, log, stash, search and branch management for SwarmWS workspaces.

Community
Intermediate
faisalanjumfaisalanjum
3

test-agent-field-2026

Test direct invocation of agent-specific MCP tools to verify permissions.

Community
Basic