observability-contract-review

Reviews instrumentation claims by verifying span and metric path coverage across exception paths.

2|Updated Jul 18, 2026
One-click install
npx skills add https://github.com/Arasz/ai-badger --skill observability-contract-review-arasz
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: observability-contract-review
Source: https://github.com/Arasz/ai-badger/tree/main/features/dotnet/skills/dotnet-workload/references/observability-contract-review
Command: npx skills add https://github.com/Arasz/ai-badger --skill observability-contract-review-arasz

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve? Code reviews of observability changes often accept claims like "all calls are instrumented" based on call-site counts, missing exception paths where spans and metrics silently go unrecorded. This Skill provides a checklist that verifies path coverage rather than call-site presence when reviewing .NET tracing and metrics changes. ## Core Features & Use Cases - Filtered-catch escape detection: Identifies catch (Exception ex) when (ex is not X) filters that let guard-thrown exceptions escape without recording, and prescribes the idempotent-record fix. - Recording correctness checks: Verifies exactly-once recording, Activity status/tag ordering before span stop, and instrumentation unit-test honesty including ActivityListener parallelization flakes. - E2E parity and CI gate review: Validates tool-surface parity tests for exact set equality, cross-file call enumeration, Speed-trait CI blind spots, and metrics-unchanged diffs. - Use Case: When a PR claims "all 19 tools emit the span/metrics contract", use this Skill to walk every throw/return path, find the four tools whose access-denied errors escape unrecorded, and issue a MAJOR finding with a concrete fix. ## Quick Start Review this pull request that claims all tool calls are instrumented and verify the span and metrics contract holds on every error path.

Frequently Asked Questions about observability-contract-review

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I review a PR that claims all calls are instrumented?

Verify path coverage, not call-site presence. Grep for helper construction sites, then walk every early-exit and exception path asking which catch records it. Filtered catches like `when (ex is not X)` are the most common hole where guard-thrown exceptions escape unrecorded.

Why do filtered catch clauses break OpenTelemetry span recording?

A filter like `catch (Exception ex) when (ex is not McpException)` exists to avoid double-recording inner-catch rethrows, but guard exceptions of the filtered type thrown before the inner try escape entirely. The fix is making record methods idempotent, then widening outer catches to record everything.

How do I fix double-counted metrics when removing a catch filter?

Add a private `_recorded` flag to the helper so RecordInvocation and RecordError no-op on second call. Inner typed catches record the domain exception first, preserving the meaningful error_type, while the outer catch safely records previously-escaped paths.

Why do ActivityListener unit tests flake in parallel test runs?

ActivityListener is process-global, so tests listening on a shared source name can capture activities from concurrent test classes, breaking assertions like ShouldHaveSingleItem. Serialize those tests in a collection with DisableParallelization set to true.

How do I verify an N/N tool-surface parity test actually covers everything?

Require exact set equality with sorted names via ShouldBe, never a Contains-list. Enumerate tool names across all test files and both call styles, since literal greps for CallAsync miss direct CallToolAsync calls and variable-built names.

When is a pre-existing instrumentation gap in scope for a PR review?

When the PR's acceptance criterion or review record claims the whole surface emits the contract, pre-existing gaps become in-scope. Flag them MAJOR and require a fix or a tracked follow-up issue, since the claim as written is the acceptance criterion.