skills.rest
microsoft
microsoftOFFICIAL
Explore all skills by this author →
Website𝕏@OpenAtMicrosoftGitHub

fix-flaky-test

Reproduce and fix flaky tests in the microsoft/aspire repository.

6.2k|952|Updated Sep 25, 2023
Search Tags:#debugging#flaky tests#CI verification#dotnet test#test reproduction#quarantined tests
One-click install
npx skills add https://github.com/microsoft/aspire --skill fix-flaky-test-microsoft
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: fix-flaky-test
Source: https://github.com/microsoft/aspire/tree/main/.agents/skills/fix-flaky-test
Command: npx skills add https://github.com/microsoft/aspire --skill fix-flaky-test-microsoft

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill helps you reproduce and permanently fix flaky or quarantined tests in the microsoft/aspire repository, reducing intermittent CI failures that waste time and erode confidence.

Core Features & Use Cases

  • Guided investigate→reproduce→fix→verify cycle: Forces a checkpointed workflow so fixes are backed by real failure evidence rather than guesses.
  • Local fast-path reproduction: Runs targeted repeated test execution first using run-test-repeatedly.sh or run-test-repeatedly.ps1 for quick feedback.
  • CI fallback reproduction: When local reproduction fails, escalates to reproduce-flaky-tests.yml with OS targeting and graduated escalation (single-test, then quarantine-project when needed).
  • Log-driven root-cause confirmation: Separates true test failures from infrastructure failures and uses logs to confirm the real cause before editing code.
  • Two-branch PR strategy: Uses an investigation draft branch for CI verification artifacts/workflow changes, then creates a clean final fix PR.

Quick Start

Ask the Skill to fix a specific quarantined test (by issue URL or fully qualified test method name) and let it run local repeated reproduction first, then fall back to the CI reproduce workflow and produce a verified fix.

Frequently Asked Questions about fix-flaky-test

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I fix flaky tests that keep failing intermittently in CI?▼

Fix flaky tests by running a checkpointed workflow that collects failure evidence, reproduces the issue deterministically using local repeated test scripts, applies a validated code change, and verifies stability across local and CI environments before submitting a clean PR.

What is the best way to reproduce quarantined tests locally before pushing a fix?▼

The best way to reproduce quarantined tests locally is running targeted repeated test execution using run-test-repeatedly.sh or run-test-repeatedly.ps1 scripts first, providing quick feedback before escalating to CI reproduction workflows.

How do I reproduce OS-specific flaky tests when local reproduction fails?▼

When local reproduction fails, escalate reproduction of OS-specific flaky tests to the reproduce-flaky-tests.yml workflow with OS targeting and graduated escalation from single-test to quarantine-project to confirm the failure.

Why does my dotnet test pass locally but fail intermittently in CI pipelines?▼

Intermittent CI failures often stem from contention-sensitive behavior or OS-specific flakiness. Confirm the real root cause by separating true test failures from infrastructure failures through log analysis before editing code.

Can I use this workflow to fix quarantined tests in the microsoft/aspire repository?▼

Yes, this workflow specifically targets test investigation scenarios including intermittent failures and quarantined tests within the microsoft/aspire repository, guiding you through reproduction, log analysis, and CI verification.

What should I do if my flaky test fix still fails during CI verification?▼

If the fix fails during CI verification, retain CI verification artifacts and workflow changes on an investigation draft branch, then analyze the logs to confirm the root cause before creating a clean final fix PR.

Related Skills

Explore More Software Engineering Skills→
microsoftmicrosoft
6.2k

test-management

Quarantine or disable xUnit tests using QuarantineTools attributes in the aspire repository.

Official
Advanced
KangJiSseokKangJiSseok

code-review-pr-body

Generate structured pull request body drafts from git diff outputs.

Community
Advanced
bismuthdigitalbismuthdigital

ci-review

Diagnoses GitHub Actions CI run failures by inspecting job logs and identifying root causes.

Official
Advanced
polsegupolsegu

test-flakiness

Detect nondeterministic failures by parsing CI logs and aggregating per-test pass/fail histories.

Community
Intermediate