skills.rest
markuplint
markuplintOFFICIAL
Explore all skills by this author →
Website𝕏@markuplintGitHub

bench-triage

Triages nu-validator coverage fixtures by reading specs and driving verdicts to confirmed states.

611|63|Updated Sep 26, 2017
Search Tags:#spec compliance#markuplint#coverage gap#html linting#html standard#nu-validator#benchmark triage
One-click install
npx skills add https://github.com/markuplint/markuplint --skill bench-triage
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: bench-triage
Source: https://github.com/markuplint/markuplint/tree/main/.claude/skills/bench-triage
Command: npx skills add https://github.com/markuplint/markuplint --skill bench-triage

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

When the markuplint vs. nu-validator coverage benchmark surfaces a new nu-only fixture (a case where nu-validator flags markup but markuplint does not), someone must determine whether markuplint has a real coverage gap or nu is over-detecting. This Skill walks through that classification: reading the fixture, both tools' outputs, and the authoritative spec, then either extending markuplint or recording a spec-cited exclusion.

Core Features & Use Cases

  • Verdict-driven triage: Drives each nu-only fixture to match-error, match-clean, or nu-over by comparing nu-validator messages against markuplint violations and the HTML/ARIA/URL Living Standards.
  • Spec-cited exclusions: Records nu over-detections in excluded-ids.json with verbatim spec quotes, per-ID entries or message-substring patterns, and deferred-spec markers tied to tracking Issues.
  • Determinism and fact-check gates: Pins verdicts with --concurrency 1 runs to avoid parallel flicker, and requires every file path, npm package, and spec section cited in a filed Issue to be verified first.
  • Use Case: After a bench refresh surfaces a new nu-only entry for a <source srcset> fixture, use this Skill to read the HTML LS paragraph, confirm nu is correct, extend the relevant markuplint rule, and flip the fixture to match-error.

Quick Start

Triage the newest nu-only fixture from tests/external/snapshots/diff/nu-only.json and drive its verdict to a confirmed state.

Frequently Asked Questions about bench-triage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I triage a nu-only fixture in the markuplint bench?▼

Pick a fixture from tests/external/snapshots/diff/nu-only.json, read the nu-validator messages and markuplint violations snapshots, then read the raw HTML against the authoritative spec. Based on the spec verdict, either extend a markuplint rule or record an exclusion in excluded-ids.json.

What do match-error, match-clean, nu-only, and nu-over verdicts mean?▼

Match-error means both tools detected a violation, match-clean means neither did, nu-only means only nu-validator detected it without exclusion coverage, and nu-over means only nu-validator detected it but every message is covered by excluded-ids.json entries.

When should I record a nu over-detection in excluded-ids.json?▼

Record an exclusion only when the spec permits the markup and you can cite a verbatim spec paragraph in the reason field with a specUrl. If no spec paragraph can be cited, file an Issue to track future markuplint coverage instead of excluding.

Why does a nu-validator verdict change between bench runs?▼

Nu-validator is non-deterministic under parallel load because state is shared across requests in its runtime. Re-run with yarn bench:update --target nu --concurrency 1 --filter for the fixture, then yarn bench:compare, to confirm the verdict is stable.

What specs are authoritative when classifying a coverage gap?▼

Authoritative sources are the HTML Living Standard, DOM LS, URL LS, WAI-ARIA 1.3, and ARIA in HTML. MDN is not authoritative; quote WHATWG or W3C text verbatim when sources disagree, since living standards change over time.

What is a deferred-spec exclusion in the bench?▼

A deferred-spec exclusion covers messages where nu enforces a spec outside markuplint's tracked scope of HTML LS, WAI-ARIA, and URL LS, such as CSS syntax. Each requires a tracking Issue and a reason marker like deferred-CSS so future work can locate the entries.

Related Skills

Explore More Software Engineering Skills→
VMAFxVMAFx
3

regen-snapshots

Regenerate snapshot JSON files for CPU, SYCL, and Netflix benchmarks.

Official
Advanced
aphrody-codeaphrody-code

regen-baseline

Regenerate tracked n2b baseline snapshots for fixture and rule outputs.

Community
Intermediate
mattbutlerengineeringmattbutlerengineering
1

rialto-visual-diff-review

Classify Playwright visual diff failures as intentional, regression, or stale.

Community
Intermediate
vertexcover-iovertexcover-io
4

coverage-guard

Enforce minimum unit test coverage for tarash-gateway using coverage.json.

Official
Advanced