golden-truth

Verifies and edits eval golden answers through multi-source triangulation and provenance tracking.

8|6|Updated Jul 31, 2026
One-click install
npx skills add https://github.com/stellar-experimental/stellar-raven --skill golden-truth-stellar-experimental
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: golden-truth
Source: https://github.com/stellar-experimental/stellar-raven/tree/main/.agents/skills/golden-truth
Command: npx skills add https://github.com/stellar-experimental/stellar-raven --skill golden-truth-stellar-experimental

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Changing an evaluation golden Q&A corpus from weak or single-source evidence silently corrupts every future eval round, A/B comparison, and re-judge. This Skill enforces a verification workflow so golden answers, key facts, avoid clauses, and grader notes are only changed after cross-class corroboration, with disputed or unverifiable facts encoded honestly instead of pinned. ## Core Features & Use Cases - Truth-domain classification: Classify each claim as real-world, corpus-grounded, or freshness-sensitive to pick the correct verification standard before editing. - Multi-source corroboration matrix: Require agreement across independent source classes (official docs, source code, live APIs, web research, docs index, empirical execution) with per-claim verdicts recorded in truth.corroboration[]. - Verdict-based encoding and lifecycle: Encode confirmed, disputed, contradicted, or unverifiable facts differently, manage case lifecycle states (proposed, active, quarantined, retired), and run lint, compile, and register checks before closing a change. - Use Case: An eval case claims a Stellar RPC numeric limit that a live probe contradicts. Use this Skill to triangulate the limit across docs and source code, update the golden with an asOf date and evidence trail, sweep sibling cases for consistency, and pass the CI gospel-change lint. ## Quick Start Use the golden-truth skill to verify and update the disputed numeric limit in eval case eval/qa/corpus/battery/<category>/<id>.json with corroborated evidence.

Frequently Asked Questions about golden-truth

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I safely update a golden answer in an eval corpus?

Classify the claim's truth domain, then corroborate it across at least two independent source classes such as official docs and source code. Record the verdict and evidence in the case's truth block, run the sibling-consistency sweep, and pass the gospel-change lint before committing.

What source classes count as independent corroboration?

The six classes are official primary docs, source code, live service APIs, general-web research, docs search index, and empirical execution. Two hits from the same class count as one, and an aggregator never corroborates its own claims.

How should disputed or unverifiable facts be encoded in goldens?

Disputed facts are never pinned: set truth.status to disputed, record corroboration rows for both sides, and add a grader caution in golden.notes. Unverifiable facts must be removed from the golden or downgraded to source-relative notes.

Why does the CI lint reject my golden corpus change?

The gospel-change lint requires any judge-facing edit to update truth.verified in the same diff with non-empty evidence and rootCause fields. Root causes must point to real defects such as improvements findings or freshness drift, not judge score rationales.

When should an eval case be quarantined or retired?

Quarantine a case when a credible truth or validity conflict is confirmed by an independent reviewer for a score-independent cause. Retire only for duplication, obsolete scope, unanswerable wording, or lost product relevance, recording evidence and reviewer in the tombstone.