buildbuddy-flaky-tests

Fetch and rank flaky BuildBuddy targets with statistics for triage.

6|Updated Jul 9, 2019

Search Tags:#automation #continuous-integration #flaky-tests #test-analysis #buildbuddy #sample-invocations #rpcs

Ready to run

npx skills add https://github.com/sluongng/dotfiles --skill buildbuddy-flaky-tests

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: buildbuddy-flaky-tests
Source: https://github.com/sluongng/dotfiles/tree/main/config/claude/plugins/buildbuddy/skills/buildbuddy-flaky-tests
Command: npx skills add https://github.com/sluongng/dotfiles --skill buildbuddy-flaky-tests

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) and references (resource) components.

What problem does it solve?

It fetches and ranks flaky test targets from BuildBuddy, enabling quick triage, reproduction, and fixes.

Core Features & Use Cases

Retrieve flaky targets for a repository, branch, and time window using GetTargetStats.
Retrieve daily aggregates with GetDailyTargetStats to spot trends and prioritize investigation.
GetTargetFlakeSamples to capture sample invocations for top labels and guide reproduction.
Generate structured results and sample links to assist triage, PRs, and remediation efforts.

Quick Start

Run fetch_flaky_tests.py to fetch, rank, and summarize flaky targets for your BuildBuddy repository.

Frequently Asked Questions about buildbuddy-flaky-tests

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I identify flaky tests in my BuildBuddy continuous integration setup?▼

To identify flaky tests in BuildBuddy, fetch and rank flaky targets for a specific repository, branch, and time window. This surfaces failing tests by retrieving target statistics and daily aggregates to spot trends for investigation.

How do I get sample invocations for BuildBuddy targets to reproduce flaky tests?▼

To get sample invocations for BuildBuddy targets, retrieve sample flake data for top labels. This captures specific invocation instances that guide you through reproducing the exact failures and preparing the necessary fixes for your pull requests.

Can I scope flaky test analysis to a specific repository branch and time window?▼

Yes, you can scope flaky test analysis to a specific repository, branch, and time window. Optional group scoping is also supported, allowing you to filter and produce structured results tailored to your continuous integration environment.

What is the best way to analyze daily flaky test trends for remediation?▼

The best way to analyze daily flaky test trends is to retrieve daily aggregates using BuildBuddy statistics. This helps you spot daily trend patterns, prioritize investigation for the most frequent failures, and generate structured results for triage.

Do I need specific BuildBuddy RPCs to fetch flaky test statistics?▼

Yes, fetching flaky test statistics relies on the BuildBuddyService RPCs GetTargetStats, GetDailyTargetStats, and GetTargetFlakeSamples. These RPCs retrieve target data, daily aggregates, and sample invocations to surface flaky tests.