kata-agent-feedback-design

Designing an agent feedback loop requires specifying HITL gating for irreversible actions, critic-based quality review, and objective metrics to measure and remediate production behavior. This generates a structured feedback blueprint for continuous operational learning.

Tier-1 production agents require SLOs with explicit SLI definitions, error-budget policies, and alert runbooks. You must define tier-1 or tier-2 SLO YAML configurations ensuring every critical alert triggers an actionable operational response for metric-driven degradation states.

Operational metrics for agent monitoring require at least three objective metrics with canonical names, runtime measurement rules, thresholds, and windows. You must provide runtime operational definitions and remedial actions to execute when metric deviations occur across agent turns.

Human-in-the-loop gating is needed for irreversible actions executed by production agents. You define HITL confirmation modalities alongside optional LLM critic reviews to gate quality before actions proceed, ensuring safe remediation for tier-1 or tier-2 operational deployments.

Generating runbooks for critical agent alerts is a core requirement of production readiness. You must ensure every critical alert defined in your SLO YAML has a corresponding runbook, specifying actionable operational responses for per-alert remediation across agent turns.