calibrate-claim-confidence

Rewrites confident absolutes in outgoing messages when agent epistemic certainty is low.

2.5k|415|Updated Mar 28, 2026
One-click install
npx skills add https://github.com/Bitterbot-AI/bitterbot-desktop --skill calibrate-claim-confidence
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: calibrate-claim-confidence
Source: https://github.com/Bitterbot-AI/bitterbot-desktop/tree/main/skills/calibrate-claim-confidence
Command: npx skills add https://github.com/Bitterbot-AI/bitterbot-desktop --skill calibrate-claim-confidence

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

AI agents often state uncertain claims with absolute confidence, using words like "definitely", "always", or "100%" even when their underlying knowledge is weak. This Skill prevents overconfident messaging by tying language calibration directly to the agent's measured epistemic state.

Core Features & Use Cases

  • State-Bound Hedging: Reads the GCCRF epistemic state (empowerment and certainty delta) to decide whether outgoing language needs softening.
  • Automatic Message Rewriting: Intercepts send_message-shaped tool calls and replaces confident absolutes with hedged phrasing like "likely" or "typically" when empowerment drops below 0.3 and certainty is falling.
  • Confidence Preservation: Leaves absolute language intact when the agent has corroborated context and rising certainty.
  • Use Case: When the agent answers a user question on a topic where its context is thin, the outgoing reply is automatically rewritten from "this will definitely fix it" to "this will likely fix it", keeping claims honest.

Quick Start

Enable the built-in calibrate-claim-confidence interceptor so outgoing agent messages are automatically hedged whenever GCCRF empowerment is low and certainty is falling.

Frequently Asked Questions about calibrate-claim-confidence

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I stop an AI agent from making overconfident claims?

Use an interceptor that reads the agent's epistemic state before sending messages. This Skill checks GCCRF empowerment and certainty delta, then rewrites confident absolutes like "definitely" into hedged language like "likely" when confidence is unsupported.

What is GCCRF empowerment in agent messaging?

GCCRF empowerment is an epistemic metric quantifying how well the agent's current knowledge supports its statements. High empowerment means corroborated context; low empowerment means the agent is reasoning on uncertain ground and should hedge its language.

Does message hedging apply when the agent is confident?

No. When empowerment is high and certainty is rising, absolute language is left intact. The interceptor only modifies messages when empowerment falls below 0.3 and the certainty delta is negative.

How often does the confidence calibration interceptor fire?

The built-in interceptor fires up to 6 times per session. It activates only on send_message-shaped tool calls that meet the low-empowerment, falling-certainty condition.

Can I use this hedging approach outside Bitterbot?

The interceptor is implemented as a Bitterbot built-in at src/agents/skills/builtin-interceptors/calibrate-claim-confidence.ts and depends on the GCCRF epistemic state, so it requires the Bitterbot runtime to function.