skills.rest
dotnet
dotnetOFFICIAL
Explore all skills by this author →
Website𝕏@dotnetfdnGitHub

review-test-failures

Classifies PR CI test failures as PR-caused or unrelated and emits a merge-readiness verdict.

23.3k|2.0k|Updated May 8, 2020
Search Tags:#test failures#Azure DevOps#triage#ci#gitHub#Pull request#Merge readiness
One-click install
npx skills add https://github.com/dotnet/maui --skill review-test-failures
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: review-test-failures
Source: https://github.com/dotnet/maui/tree/main/.github/skills/review-test-failures
Command: npx skills add https://github.com/dotnet/maui --skill review-test-failures

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Reviewing failing CI checks on a .NET MAUI pull request is slow and error-prone: failures may be flaky tests, infrastructure issues, pre-existing base-branch failures, or genuine regressions introduced by the PR. This Skill automates that triage by gathering GitHub, Azure DevOps, and Helix evidence and producing a single merge-readiness verdict.

Core Features & Use Cases

  • Deterministic failure attribution: Compares each distinct failure against recent base-branch builds, known-issue registries, and ci-scan history to classify it as Likely PR-caused, Likely unrelated, Needs human investigation, or Insufficient data.
  • Hard verdict ceiling: A computed gate caps the overall verdict so a green result is impossible while checks are pending, builds are canceled, device tests are unverified, or legs regressed versus base.
  • Structured PR comment output: Posts a compact Tests Failure Analysis comment with badges, a per-failure evidence table, coverage counts, and a recommended action.
  • Use Case: A maintainer comments /review tests on a MAUI pull request with red CI; the Skill gathers context via its PowerShell script, classifies each failure against the base branch, and reports whether the PR's CI is ready to merge.

Quick Start

Ask the agent to review the test failures on a specific dotnet/maui pull request number and report whether its CI is ready to merge.

Frequently Asked Questions about review-test-failures

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if PR test failures are caused by my changes?▼

Run the review-test-failures Skill against the pull request number. It gathers GitHub check and Azure DevOps build evidence, compares each failure against recent base-branch builds, and classifies each one as Likely PR-caused, Likely unrelated, Needs human investigation, or Insufficient data.

How does the Skill decide if a PR is ready to merge?▼

It applies a deterministic verdict ceiling computed from coverage facts such as pending checks, canceled builds, unverified device tests, and legs that regressed versus base. The overall verdict can never be more favorable than this ceiling, and per-failure classifications feed the final merge-readiness verdict.

What tools are required to run the test failure review?▼

The Skill requires the GitHub CLI (gh) for gathering PR and check data. Local execution additionally requires the Copilot CLI, and an optional AZDO_TOKEN or Azure CLI login enables authenticated Azure DevOps test result queries.

Why are green device-test checks treated as unverified?▼

XHarness exits with code 0 even when Helix device tests fail, so a green maui-pr-devicetests check is not proof of a clean run. Unless the gatherer positively confirms zero failures from Helix or the test API, the verdict ceiling is capped at Needs human investigation.

Can the Skill approve or merge a pull request?▼

No. The Skill only posts an analysis comment about CI health. It does not apply labels, trigger reruns, approve PRs, request changes, or modify code; approval remains a human-only decision.

Related Skills

Explore More Software Engineering Skills→
dotnetdotnet
23.3k

pr-build-status

Retrieve Azure DevOps build IDs and Helix test logs for GitHub PRs.

Official
Intermediate
apacheapache
79

magpie-issue-fix-workflow

Draft code fixes with automated tests for confirmed issue tracker items.

Official
Intermediate
dotnetdotnet
13

pipeline-investigation

Identify and analyze Azure DevOps pipeline failures outside Helix to determine root causes.

Official
Intermediate
tankpkgtankpkg
1

@tank/go-api-patterns

Design Go HTTP API architectures with net/http, middleware, and Docker deployment.

Official
Advanced