mz-debug-ci

Diagnose Buildkite CI failures on Materialize pull requests using gh and Buildkite tools.

6.4k|512|Updated Feb 22, 2019
One-click install
npx skills add https://github.com/MaterializeInc/materialize --skill mz-debug-ci
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: mz-debug-ci
Source: https://github.com/MaterializeInc/materialize/tree/main/.agents/skills/mz-debug-ci
Command: npx skills add https://github.com/MaterializeInc/materialize --skill mz-debug-ci

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Investigating why CI is red on a Materialize PR requires digging through Buildkite builds, job logs, annotations, and artifacts, which is slow and error-prone when done manually. This Skill provides a structured triage workflow that finds the failing build, reads pre-extracted error annotations, fetches logs only when needed, and groups failures by root cause.

Core Features & Use Cases

  • Build and job discovery: Locate failing builds from a PR number, GitHub URL, or Buildkite URL, and list genuinely failed jobs with correct job-state semantics (failed/timed_out vs. broken or waiting_failed).
  • Annotation-first triage: Read pre-extracted error annotations with known-issue links (Linear keys like CPU-170) before touching raw logs, saving significant investigation time.
  • Root-cause categorization: Apply Materialize-specific patterns for clippy errors, testdrive cascades, SLT failures, timeouts, feature benchmark regressions, cargo-fuzz crashes, and Miri failures.
  • Known vs. new failure analysis: Use bin/ci-failures history, git log against main, and open PRs to distinguish pre-existing flakes from regressions introduced by the change.
  • Use Case: A developer sees failing checks on PR #12345 and asks why CI is red; the Skill identifies the failed jobs, finds the annotation linking to a known flaky test, and reports that only one clippy error is a genuine new issue to fix.

Quick Start

Ask the assistant to investigate why CI is failing on PR 12345 or on a pasted Buildkite build URL.

Frequently Asked Questions about mz-debug-ci

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I investigate why CI is failing on a GitHub pull request?

Run gh pr checks with the PR number to list failing checks, which link to Buildkite builds and job IDs. Then fetch the build's error annotations first, since they contain pre-extracted error messages and known flaky-test issue links before you grep raw logs.

How do I find the root cause of a Buildkite build failure?

List jobs with state failed or timed_out, then read error annotations before logs. Group failures by root cause rather than job name, since many failing jobs typically share one or two causes, and check annotations for known-issue links.

What tools are required to debug Buildkite CI failures?

You need an authenticated gh CLI plus either the Buildkite MCP server (preferred, with tools like get_build_failure_summary and search_logs) or the authenticated bk CLI. If neither Buildkite path is available, the workflow stops and tells you what to configure.

Why does a Buildkite build show more error annotations than failed jobs?

Error annotations persist from failed attempts even when a retry later passed, so a build can carry annotations from retried attempts not shown in the default job listing. Only the latest attempt of a retried job counts for build triage.

How do I tell a flaky test from a real regression in CI?

Check annotations for known-issue links and query bin/ci-failures with the error text to see its history. Also run git log from the build's commit to main on suspect files, and check recently opened PRs for an existing fix.

Why do many testdrive tests fail after the first error in a shard?

After one test crashes environmentd, all subsequent tests in that shard fail with connection errors like Name or service not known. Only the first failure matters; everything after it is a cascade, so look for the first error or FAIL in the log.