analyze-nightly

Summarize GitHub Actions nightly CI failures grouped by ownership area.

74|32|Updated Sep 13, 2024

Search Tags:#github-actions #triage #ci #log-analysis #failure-analysis #nightly #test-failures

Ready to run

npx skills add https://github.com/tenstorrent/tt-xla --skill analyze-nightly

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: analyze-nightly
Source: https://github.com/tenstorrent/tt-xla/tree/main/.claude/skills/analyze-nightly
Command: npx skills add https://github.com/tenstorrent/tt-xla --skill analyze-nightly

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Analyze nightly CI runs to produce a structured failure summary.

Core Features & Use Cases

Group failures by ownership area (PJRT unit tests, vLLM integrations, multi-host tests, performance benchmarks, examples, and PyTorch/JAX tests).
Extract and summarize first-failed steps, collect relevant logs and error messages, and produce an actionable report.
Output a triage-friendly report with architecture context when available.

Quick Start

Provide a run-id to analyze and summarize the nightly CI failures.

Frequently Asked Questions about analyze-nightly

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I summarize nightly CI failures from GitHub Actions by ownership?▼

You can summarize nightly CI failures by providing a GitHub Actions run-id to analyze. The tool reads job logs via GitHub CLI, extracts error messages, and groups failures by ownership area and architecture to produce a triage-friendly report.

Can I triage test failures across PJRT unit tests and vLLM integrations?▼

Yes, you can triage test failures across PJRT unit tests, vLLM integrations, multi-host tests, performance benchmarks, examples, and PyTorch/JAX tests. Failures are grouped by ownership area to streamline remediation.

What is the best way to extract error messages from GitHub Actions logs for triage?▼

The best way to extract error messages for triage is to analyze the nightly run-id. The tool reads job logs via GitHub CLI, identifies the first-failed steps, and collects relevant error messages into a compact, ownership-tagged report.

Do I need a specific run-id to analyze a GitHub Actions nightly run?▼

Yes, you need to provide a specific run-id to analyze and summarize the nightly CI failures. This run-id allows the tool to fetch the corresponding job logs and generate an actionable failure summary.

Does nightly CI failure analysis provide architecture context for test failures?▼

Yes, nightly CI failure analysis provides architecture context when available. The generated triage report includes this context alongside the grouped failures to help with targeted remediation.

Why group CI failures by ownership area during nightly test triage?▼

Grouping CI failures by ownership area during nightly test triage helps identify responsible teams quickly. The tool produces an ownership-tagged report that organizes failures from areas like performance benchmarks and PyTorch tests for faster remediation.