What problem does it solve?
Flaky tests erode CI trust and waste pipeline time, but triaging them manually requires digging through test history, failure patterns, and blast radius across pipelines. This Skill automates that investigation using Datadog CI Visibility data and produces a structured triage brief with a concrete recommendation.
Core Features & Use Cases
- Flaky Test Investigation: Retrieves flaky state, failure rate, root-cause category, codeowners, and CI time lost for a specific test via the pup CLI.
- Failure Pattern & Blast Radius Analysis: Pulls recent failure history, error traces, and counts impacted pipelines to determine whether failures are branch-specific or widespread.
- Actionable Recommendation: Produces a triage brief recommending fix (with category-specific fix patterns), quarantine (with explicit user approval before writing state changes), or escalation to codeowners.
- Use Case: A test like
TestMyFunc intermittently fails on main. Run this Skill to get its 30-day failure rate, root-cause category (e.g., concurrency), affected pipelines, and a decision on whether to fix the synchronization issue or quarantine the test.
Quick Start
Triage the flaky test TestMyFunc in this repository and tell me whether to fix, quarantine, or escalate it.