ci-perf

Analyze GitHub Actions CI timing data and prepare ranked performance fixes for pull requests.

2.7k|685|Updated Nov 14, 2023
One-click install
npx skills add https://github.com/UKGovernmentBEIS/inspect_ai --skill ci-perf
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: ci-perf
Source: https://github.com/UKGovernmentBEIS/inspect_ai/tree/main/.claude/skills/ci-perf
Command: npx skills add https://github.com/UKGovernmentBEIS/inspect_ai --skill ci-perf

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

Pull request CI in this repository is slow, and it is hard to tell whether the bottleneck is runner queue contention, long test jobs, or workflow misconfiguration. This Skill collects per-job and per-step timing data from GitHub Actions, separates queue time from execution time, and turns the findings into a ranked report plus concrete, low-risk fixes.

Core Features & Use Cases

  • CI data collection: A bundled Python script snapshots the last ~200 completed PR workflow runs via the GitHub API, capturing per-job execution seconds, wait-from-run-start, per-step timings, and pytest --durations and summary lines mined from Build test-job logs.
  • Structured analysis: Computes queue vs execution splits, critical paths per workflow, slowest tests, suite-size growth trends, cancelled-run waste, and p90 step-level variance such as erratic full-history git checkouts.
  • Report and fix workflow: Rewrites design/ci-perf/report.md each run, maintains a permanent prs.md ledger, prepares safe fixes (workflow hygiene, trivial test fixes) as a single PR, and files ripe structural proposals as issues for maintainer decision.
  • Use Case: A maintainer notices PR checks taking 40 minutes. Run the skill to discover that queue contention and a 42-second docker fixture test dominate, then ship one PR adding pytest --durations, marking the slow test, and trimming unneeded fetch-depth: 0 checkouts.

Quick Start

Ask the assistant to run the CI performance report for this repository, for example: run the ci-perf analysis and tell me where our PR wall-clock time is going.

Frequently Asked Questions about ci-perf

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I measure where GitHub Actions CI time goes for pull requests?

Run the bundled collect_ci_data.py script with an authenticated gh CLI to snapshot recent PR workflow runs. It records per-job execution and wait times, per-step timings, and pytest durations, which the analysis phase splits into queue time versus execution time.

How do I find the slowest pytest tests in CI?

Add --durations=50 to the pytest invocation in the workflow, then the collector mines those duration blocks from Build test-job logs. Aggregate across runs by median per test, including setup and teardown phases where fixture-heavy offenders hide.

Why is my CI slow even though individual jobs look fast?

The dominant cause is often runner queue contention rather than execution time, especially when many jobs fan out per PR. Check wait-from-run-start per job, subtracting predecessor completion for dependent jobs, and inspect p90 step variance such as full-history checkouts.

Can this skill push CI fixes automatically?

In interactive mode it always shows the diff and PR body and waits for approval before pushing. In scheduled unattended mode it may open one PR with up to two safe fixes, but structural changes are only filed as issues for maintainer decision.

What kinds of CI changes does the skill refuse to make?

It never proposes trimming the Python version matrix and never opens PRs for structural changes like renaming required checks, altering job topology, or changing retry policy. Those are ranked in the report and filed as issues when ripe.