test-diff-analyzer

Analyze test execution logs to detect flaky tests and consistency issues.

18|5|Updated Nov 3, 2025
One-click install
npx skills add https://github.com/HomericIntelligence/ProjectOdyssey --skill test-diff-analyzer-homericintelligence
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: test-diff-analyzer
Source: https://github.com/HomericIntelligence/ProjectOdyssey/tree/main/.claude/skills/test-diff-analyzer
Command: npx skills add https://github.com/HomericIntelligence/ProjectOdyssey --skill test-diff-analyzer-homericintelligence

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill helps identify and debug flaky tests by analyzing differences in test execution results across multiple runs, pinpointing intermittent failures.

Core Features & Use Cases

  • Flaky Test Detection: Compares test logs to find tests that pass inconsistently.
  • Consistency Analysis: Assesses the pass rate of tests across different runs.
  • Root Cause Hypothesis: Provides insights into potential causes like timing, randomness, or resource issues.
  • Use Case: When a test fails intermittently in your CI pipeline but passes locally, this Skill can help you analyze the logs to understand why and suggest a fix.

Quick Start

Analyze test differences between two log files to find flaky tests.

Frequently Asked Questions about test-diff-analyzer

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find flaky tests in my CI pipeline logs?

To debug flaky tests, compare test execution logs across multiple runs to detect intermittent failures and calculate pass rate consistency. This analysis reports failure patterns and potential root causes like timing, randomness, or resource constraints.

What causes intermittent test failures in continuous integration?

Intermittent test failures in continuous integration are caused by timing problems, randomness, or resource constraints. Analyzing test execution logs across multiple runs detects these consistency issues and reports their potential root causes.

How do I analyze test execution logs to identify consistency issues?

You analyze test execution logs for consistency issues by comparing results across multiple runs. This assesses the pass rate of tests, detects intermittent failures, and generates a report detailing consistency scores and failure patterns.

Can I detect timing problems and randomness in my test suite?

Yes, you can detect timing problems and randomness by comparing test execution logs from multiple runs. This analysis highlights intermittent failures and generates root cause hypotheses for timing and resource constraints.

What is the best way to debug a test that passes locally but fails in CI?

The best way to debug a test that passes locally but fails in CI is to compare test execution logs across multiple runs. This detects intermittent failures and provides root cause insights like resource constraints or timing problems.

Does test analysis work without any external dependencies?

Yes, test analysis works without external dependencies, requiring only test execution log files as input. It compares these logs across multiple runs to detect flaky tests, consistency scores, and failure patterns.