docs-grounding-verifier

Verifies claim-level grounding of documentation pages against source code with file:line evidence.

3.7k|344|Updated Sep 18, 2025
One-click install
npx skills add https://github.com/microsoft/apm --skill docs-grounding-verifier
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: docs-grounding-verifier
Source: https://github.com/microsoft/apm/tree/main/.apm/skills/docs-grounding-verifier
Command: npx skills add https://github.com/microsoft/apm --skill docs-grounding-verifier

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and assets (resource) components.

What problem does it solve?

Documentation drifts from the code it describes, and page-level audits average over paragraph-level inaccuracies. This Skill fact-checks specific documentation pages claim by claim against the actual source code, returning per-claim verdicts (GROUNDED, PARTIAL, CONTRADICTED, UNSUPPORTED) with file:line evidence citations.

Core Features & Use Cases

  • Claim extraction: Extracts up to 15 atomic factual claims per page as structured JSON with grep-able keywords and expected source areas.
  • Deterministic evidence retrieval: Uses ripgrep/grep over the source tree (no LLM) so the judge can only rule on real evidence, preventing hallucinated grounding.
  • Adversarial judging: A skeptical judge prompt rules each claim-evidence tuple and cites specific file:line proof, with fix suggestions for contradicted claims.
  • Use Case: A maintainer asks "verify the registries.md page against the source code" and receives a per-claim report showing which sentences are grounded, which drifted, and exactly where the code disproves them.

Quick Start

Ask the agent to verify a specific documentation page against the source code, for example: fact-check the install command reference page claim by claim.

Frequently Asked Questions about docs-grounding-verifier

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I verify documentation accuracy against source code?

Run the grounding pipeline on the target page: extract atomic factual claims, retrieve evidence via deterministic grep over the source tree, then judge each claim-evidence tuple. You receive per-claim verdicts of GROUNDED, PARTIAL, CONTRADICTED, or UNSUPPORTED with file:line citations.

What is claim-level grounding verification for docs?

Claim-level verification adapts the RAGAS faithfulness-eval pattern to docs and code: each factual sentence in a page is checked individually against retrieved source evidence. This catches paragraph-level inaccuracies that page-level audit scores average over.

When should I use docs-grounding-verifier instead of a corpus audit?

Use it when you have specific pages to fact-check, such as after editing a few pages or reviewing high-risk content. Whole-corpus sweeps belong to docs-corpus-audit, and PR-diff triage belongs to docs-sync; this skill handles 1..N pages at claim granularity.

Does the grounding verifier modify my documentation files?

No, it returns advisory verdicts only and never edits files. Contradicted or partial claims are handed to a doc-writer step as fix suggestions, and an optional alignment loop re-verifies pages after edits until the grounding rate reaches 0.9.

How does the verifier avoid hallucinated evidence?

Evidence retrieval is fully deterministic: a shell script runs ripgrep or grep over the source tree using claim keywords, with no LLM involved. The judge can only rule on evidence actually retrieved, so it cannot invent supporting code.