user-value-chain-glossary

Defines MCPJam eval wire enums, stage states, failure categories, and population rules.

2.2k|272|Updated May 23, 2025
One-click install
npx skills add https://github.com/MCPJam/inspector --skill user-value-chain-glossary
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: user-value-chain-glossary
Source: https://github.com/MCPJam/inspector/tree/main/skills/user-value-chain-glossary
Command: npx skills add https://github.com/MCPJam/inspector --skill user-value-chain-glossary

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

MCPJam eval results arrive as opaque wire enums like userValue, argumentMismatch, or evaluatorErrorRateAboveMaximum, and their meanings cannot be derived from the spellings. This Skill provides the authoritative definitions so you can correctly interpret a decisionSummary, a stage chain, or stage analytics without misreading counts, verdicts, or failure locations.

Core Features & Use Cases

  • Complete vocabulary reference: Defines the six stages, five stage states, twenty-nine stage reasons, seven failure categories, four verdicts, and stage-analytics exclusion classes, verbatim from the SDK contract files.
  • Population rules: Explains measurement units, zero-denominator handling, why notEstablished is not a failure, and why stage tallies must never be summed across stages or runs.
  • Comparability checks: Lists the parity blockers and partition keys that determine whether two funnels may legitimately be compared.
  • Use Case: When an eval run returns inconclusive with evaluatorErrorRateAboveMaximum, use this Skill to learn the validity phase ran and withheld a verdict because the evaluator failed too often — a decision about measurement quality, not a server defect.

Quick Start

Explain what the wire value argumentMismatch means in my MCPJam eval decision summary and whether this run's stage analytics can be compared with last week's run.

Frequently Asked Questions about user-value-chain-glossary

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What do MCPJam eval wire enums like userValue or argumentMismatch mean?

Each wire enum has a fixed definition copied from the SDK contract files. For example, `argumentMismatch` means the call arguments did not match what the case expects, and `userValue` is the final stage asking whether the user's actual request was satisfied.

How do I interpret a decisionSummary from MCPJam eval tools?

Read the verdict, counts, and measurementUnit first, since counts are caseVariant under verdict policy v2 and trial on legacy runs. The decision summary decides the outcome; per-trial diagnostics sit underneath it as evidence and must not be tallied to re-derive the verdict.

What is the difference between inconclusive and notEstablished verdicts?

`inconclusive` is a decision the validity phase reached after running and withholding a verdict because the run did not measure the server well enough. `notEstablished` means no verdict exists at all, and it must never be reported as a regression.

Can I compare stage analytics funnels between two MCPJam runs?

Only when `stageAnalyticsParityBlockers` returns empty. Partition on runGroupId, configRevision, caseSetFingerprint, analyzer and schema versions, and require materializationState final; absent identities block comparability rather than being assumed compatible.

Why does a failed stage not tell me what to fix in my MCP server?

A first failed stage is a location and a failure category is a bucket; neither is a cause. The nearest thing to a cause is the reason on that stage's own row, which says what was observed, not what to change.

Why is a zero denominator never rendered as 0 percent in stage analytics?

A zero denominator means not measured, never zero. Rendering 0/0 as 0% reads as total failure and as 100% reads as all green, and neither was actually observed.