test-failure-investigator

Investigate Azure DevOps pipeline test failures with retry filtering and Kusto telemetry.

1|Updated Mar 4, 2026

Search Tags:#debugging #root cause analysis #test failure #kusto #ado pipeline #hypothesis driven

Ready to run

npx skills add https://github.com/azaslonov/skills --skill test-failure-investigator-azaslonov

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: test-failure-investigator
Source: https://github.com/azaslonov/skills/tree/main/test-failure-investigator
Command: npx skills add https://github.com/azaslonov/skills --skill test-failure-investigator-azaslonov

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill provides a structured, hypothesis-driven approach to systematically investigate and identify the root cause of Azure DevOps pipeline test failures, moving beyond simple log reading.

Core Features & Use Cases

Systematic Root Cause Analysis: Guides users through a phased investigation process.
Retry Filtering: Automatically filters out flaky tests that pass on retry, focusing only on blocking failures.
Hypothesis Generation & Validation: Helps formulate and test potential causes for failures.
Use Case: When a critical build fails due to test errors, use this Skill to quickly pinpoint the exact cause, understand its blast radius, and propose a fix strategy.

Quick Start

Investigate test failures in this pipeline: https://dev.azure.com/msazure/One/_build/results?buildId=149441611

Frequently Asked Questions about test-failure-investigator

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I investigate Azure DevOps pipeline test failures systematically?▼

Azure DevOps pipeline test failure investigation applies a hypothesis-driven methodology that analyzes pipeline run URLs, filters flaky retries, queries Kusto telemetry, and generates ranked root cause hypotheses with validation plans.

How does retry filtering work for flaky tests in ADO pipelines?▼

Retry filtering works by automatically excluding ADO pipeline tests that pass on retry, ensuring investigations focus strictly on blocking failures that require root cause analysis and fix strategies.

Can I use Kusto queries to identify recent changes causing test failures?▼

Yes, you can use Kusto queries to identify recent changes causing test failures by analyzing telemetry data retrieved during the investigation to correlate anomalies with pipeline errors and rank potential root causes.

What is a hypothesis-driven approach to root cause analysis for build failures?▼

A hypothesis-driven root cause analysis approach formulates potential causes for build failures, ranks them by likelihood, and generates validation plans to systematically test and confirm the exact underlying issue.

Does this test failure investigation method work with any ADO pipeline URL?▼

Yes, this test failure investigation method works with any Azure DevOps pipeline run URL, analyzing specific build details to apply retry filtering and query Kusto telemetry for that run.

What are the limitations of using automated hypothesis generation for debugging?▼

Limitations of automated hypothesis generation for debugging include reliance on available Kusto telemetry and pipeline logs, meaning it may not identify root causes hidden in unlogged external dependencies or local environment issues.