skills.rest
pytorch
pytorchOFFICIAL
Explore all skills by this author โ†’
WebsiteGitHub

ci-metrics

Query PyTorch CI workflows, test runs, and runner infrastructure metrics via Grafana.

103k|29.1k|Updated Aug 13, 2016
Search Tags:#grafana#prometheus#Infrastructure#clickhouse#GitHub-actions#pyTorch#ci-metrics
One-click install
npx skills add https://github.com/pytorch/pytorch --skill ci-metrics
Or copy as Structured Prompt for Agentโ–ผ
Please help me install this Agent Skill.
Skill: ci-metrics
Source: https://github.com/pytorch/pytorch/tree/main/.claude/skills/ci-metrics
Command: npx skills add https://github.com/pytorch/pytorch --skill ci-metrics

SYSTEM DOCUMENTATION & REQUIREMENTS

๐Ÿ’ก This Skill requires gh, curl, gcx, and includes scripts (resource) components.

What problem does it solve?

PyTorch CI data is scattered across GitHub Actions, HUD, ClickHouse, and Prometheus, making it hard to diagnose failing jobs, slow runners, or queue backlogs without manually stitching dashboards together.

Core Features & Use Cases

  • Workflow and Job Analytics: Query GitHub webhook data in ClickHouse to find the most failing workflow jobs, test pass/fail counts, or per-file test run history.
  • Runner and Queue Metrics: Query Prometheus to inspect runner queue depth, jobs running per cluster, and infrastructure health over time ranges.
  • Use Case: Investigate why a pull request is stuck by checking current queue depth per runner type and correlating with recent failure rates for the affected workflow.

Quick Start

Use the ci-metrics skill to list the top 10 most failing workflow jobs on pytorch/pytorch main in the last two weeks.

Frequently Asked Questions about ci-metrics

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I query PyTorch CI job failure rates from Grafana?โ–ผ

Run the gcx-wrapper.sh datasources clickhouse query command against the default.workflow_job table, filtering by repository_full_name='pytorch/pytorch', conclusion='failure', and a time interval. Group by job name to rank the most failing jobs.

What Grafana datasources are available for PyTorch CI metrics?โ–ผ

Two primary datasources are used: grafana-clickhouse-datasource for CI and test run data, and grafanacloud-pytorchci-prom for CI infrastructure metrics. List them with gcx-wrapper.sh datasources list.

Does the ci-metrics skill require write access to the pytorch repository?โ–ผ

Yes, only users with write permission to the pytorch/pytorch repo can authenticate to the PyTorch Grafana instance. The skill provides read-only access to dashboards and queries.

Why does gcx-wrapper.sh fail with an authentication error?โ–ผ

The wrapper requires the gh CLI to be authenticated. Run gh auth login --hostname github.com --git-protocol ssh --web, then retry. The wrapper fetches a token from hud.pytorch.org using gh credentials.

Can I query test run history for a specific test file?โ–ผ

Yes, query the tests.all_test_runs table in ClickHouse filtering by file path and a recent time interval. The table is large, so always include time and file filters to limit scanned rows.

Related Skills

Explore More Software Engineering Skillsโ†’
overviewlabsoverviewlabs

github-auth

Authenticate the agent to GitHub using git credentials or the gh CLI.

Community
Advanced
Dev10x-GuruDev10x-Guru
7

Dev10x:gh-pr-create

Automate GitHub pull request creation with ticket extraction and Job Story summaries.

Official
Advanced
BoraPerusicBoraPerusic

ml-engineer

Automate design and deployment of production ML systems across PyTorch and TensorFlow.

Community
Advanced
semicolon-devteamsemicolon-devteam

supabase-fallback

Query Supabase tables via PostgREST when the Spring Boot backend is offline.

Official
Advanced