flaky-analyze

Analyze flaky GitHub Actions CI failures for Pinot test classes.

6.1k|1.5k|Updated May 19, 2014
One-click install
npx skills add https://github.com/apache/pinot --skill flaky-analyze
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: flaky-analyze
Source: https://github.com/apache/pinot/tree/main/.claude/skills/flaky-analyze
Command: npx skills add https://github.com/apache/pinot --skill flaky-analyze

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Pull recent GitHub Actions failures for a Pinot test class and analyze whether they share a root cause. Uses the gh CLI. Surfaces stack traces, failure patterns, and a candidate hypothesis — does not auto-fix.

Core Features & Use Cases

  • Surface CI failures across master and PR runs for Pinot test classes.
  • Extract failing class, error messages, and top stack frames from logs.
  • Cluster failures by exception type and top frame to surface root causes and hypotheses.

Quick Start

Run /flaky-analyze RangeIndexTest to inspect the last ~20 runs on master plus PRs.

Frequently Asked Questions about flaky-analyze

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I diagnose flaky Pinot CI tests and find their root cause?

Diagnose flaky Pinot CI tests by parsing GitHub Actions logs to extract failing classes, error messages, and top stack frames, then clustering failures by exception similarity to surface root cause hypotheses.

How do I analyze GitHub Actions CI failures for specific Pinot test classes?

Analyze GitHub Actions CI failures by pulling recent master and PR runs for a specific Pinot test class, extracting failing class details and top stack frames, and clustering failures to identify shared root causes.

Does flaky-analyze work with GitHub Actions logs across different JDK and test matrices?

Yes, flaky-analyze works with GitHub Actions CI runs for Pinot, focusing on test class failures across master and PRs and across different JDK and test matrices to surface root causes.

Can I automatically fix flaky tests using this log analysis approach?

No, this log analysis approach surfaces stack traces, failure patterns, and candidate hypotheses for flaky tests but does not auto-fix them. It produces a structured report for manual root cause investigation.

What is the best way to cluster CI failures by exception type and top frame?

The best way to cluster CI failures is to parse logs for exception type and top stack frames, grouping similar test failures together to generate a structured report with candidate root cause hypotheses.