skills.rest
PostHog
PostHogOFFICIAL
Explore all skills by this author →
Website𝕏@PostHogGitHub

investigating-ci-failures

Diagnoses CI test failures to identify culprit commits, authors, and fixes using warehouse queries.

713|118|Updated Aug 11, 2020
Search Tags:#ci failures#PostHog#GitHub actions#Flaky tests#HogQL#test investigation#culprit commit
One-click install
npx skills add https://github.com/PostHog/posthog-foss --skill investigating-ci-failures
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: investigating-ci-failures
Source: https://github.com/PostHog/posthog-foss/tree/main/products/engineering_analytics/skills/investigating-ci-failures
Command: npx skills add https://github.com/PostHog/posthog-foss --skill investigating-ci-failures

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

When a CI test fails, developers waste time guessing whether their PR broke something, master is broken, or the test is flaky. This Skill turns a failing test or red CI run into a concrete verdict: whose fault it is, which commit caused it, who wrote it, and whether a fix already landed.

Core Features & Use Cases

  • Failure fingerprinting and classification: Groups CI failures by fingerprint from the engineering_analytics_ci_failures view and classifies them as PR-specific, trunk breaks, merge-queue conflicts, or flaky based on branch spread and timing.
  • Culprit commit identification: Runs boundary queries against engineering_analytics_ci_job_history to find the exact commit where master turned red, including the author, PR number, and the commit that fixed it.
  • Flakiness corroboration: Cross-checks retry attempts and the flaky-tests MCP tool to distinguish deterministic failures from flakes without wasting CI re-runs.
  • Use Case: A developer asks "why did my PR's CI fail?" The Skill fingerprints the failure, finds it appearing across many unrelated branches in a tight window, and answers: not your PR — master was broken between 08:01 and 09:58 UTC by PR #68727, fixed by #68855.

Quick Start

Ask the assistant to investigate why a specific test failed in CI and whether it was caused by your PR or by a broken master commit.

Frequently Asked Questions about investigating-ci-failures

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find out who broke master in CI?▼

Fingerprint the failing test in the engineering_analytics_ci_failures view, then run a boundary query on engineering_analytics_ci_job_history filtered to master. The first failure row after a success is the culprit commit, carrying the SHA, author name, and PR number.

How to tell if a CI test failure is flaky or a real break?▼

Check the failure's shape: many branches with sporadic occurrences over days suggests flaky, while a dense burst hitting master suggests a trunk break. Corroborate with retry attempts in ci_job_history and the flaky-tests MCP tool, since a single run cannot separate the two.

Why did my PR's CI fail when my changes look unrelated?▼

PR CI runs your branch merged with master, so one bad master commit fails every concurrently running PR. If the failure fingerprint appears on many unrelated branches in a tight window, your PR is not at fault and master was broken.

Does this work for Jest or Playwright test failures?▼

Fingerprints in ci_failures are pytest-only, so Jest, Playwright, and cargo failures are not indexed there. For those, use the raw failure logs via the ci-failure-logs or run-failure-logs MCP tools instead.

Why is a CI failure missing from the test failure data?▼

Jobs that fail before tests run, such as docker setup or runner failures, write no FAILED line, so they never appear in ci_failures. These are only visible as job conclusions in ci_job_history, which also records green runs for an honest failure rate.

When should I not use this CI investigation approach?▼

Do not use it for aggregate CI health, cost analysis, or merge bottlenecks, which belong to the diagnosing-ci-and-merge-bottlenecks skill, nor for building saved dashboards. During live incidents, check warehouse freshness first since a stale jobs table names the wrong culprit commit.

Related Skills

Explore More Software Engineering Skills→
fartybobofartybobo
4

resolve-ci-failures

Investigate CI/CD failures and coordinate approved fixes for GitHub Actions pipelines.

Official
Advanced
Iron-HamIron-Ham
29

release-notes-from-git

Generate release notes from git commit history and merged pull requests.

Community
Intermediate
pytorchpytorch
102.7k

ci-metrics

Query PyTorch CI workflows, test runs, and runner infrastructure metrics via Grafana.

Official
Intermediate
tneztnez

ci-health-check

Monitor GitHub Actions workflow runs and diagnose failures.

Community
Intermediate