flaky-test-debugger

Diagnose intermittent test failures and recommend fixes from logs and code.

66|9|Updated Aug 5, 2024
One-click install
npx skills add https://github.com/ClipboardHealth/core-utils --skill flaky-test-debugger
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: flaky-test-debugger
Source: https://github.com/ClipboardHealth/core-utils/tree/main/plugins/core/skills/flaky-test-debugger
Command: npx skills add https://github.com/ClipboardHealth/core-utils --skill flaky-test-debugger

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

Flaky tests cause unreliable CI/CD pipelines and obscure genuine defects, making it difficult to maintain stable software releases.

Core Features & Use Cases

  • Test Flake Identification: Analyze test failures, retries, and intermittent issues across unit, component, service, and end-to-end tests.
  • Investigation Planning: Provide structured diagnosis plans based on test logs, code, and artifact analysis.
  • Fix Recommendations: Suggest targeted code or test harness modifications to mitigate flakiness.
  • Use Case: When a CI pipeline randomly fails on specific integration tests, use this skill to determine root causes and suggest minimal fixes to stabilize the build process.

Quick Start

Run the analysis and diagnosis procedure on your failing test logs and code snippets to identify possible root causes and recommended improvements.

Frequently Asked Questions about flaky-test-debugger

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose intermittent test failures in my CI/CD pipeline?

To diagnose intermittent test failures, you need to analyze test logs, code snippets, and artifacts to identify common flaky patterns. This skill provides structured diagnosis plans to pinpoint root causes across different frameworks and environments.

What causes flaky tests in automated test suites?

Flaky tests in automated test suites are caused by common flaky patterns related to timing, state pollution, or environment dependencies. Analyzing test logs and artifacts helps identify these intermittent issues across unit, component, service, and end-to-end tests.

How do I fix flaky tests across different testing frameworks?

To fix flaky tests across different frameworks, you apply targeted code or test harness modifications based on a structured diagnosis. Analyzing test logs and code snippets allows you to determine root causes and recommend minimal fixes to stabilize the build process.

Can I use test logs and artifacts to investigate random integration test failures?

Yes, you can use test logs and artifacts to investigate random integration test failures. Analyzing these outputs identifies common flaky patterns and root causes, enabling you to recommend targeted modifications to stabilize your automated test suites.

What is the best way to improve test stability for unreliable CI/CD pipelines?

The best way to improve test stability for unreliable CI/CD pipelines is to run a structured diagnosis procedure on failing test logs and code. This identifies possible root causes of flaky tests and recommends targeted test harness modifications to mitigate flakiness.