skills.rest
pingcap
pingcapOFFICIAL
Explore all skills by this author →
Website𝕏@PingCAPGitHub

tidb-test-diff-triage

Diagnose unexpected TiDB test diffs by ruling out failpoint setup and bisecting merged commits.

40.5k|6.2k|Updated Sep 6, 2015
Search Tags:#regression#go test#Git bisect#tidb#testdata#test triage#failpoint
One-click install
npx skills add https://github.com/pingcap/tidb --skill tidb-test-diff-triage
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: tidb-test-diff-triage
Source: https://github.com/pingcap/tidb/tree/main/.agents/skills/tidb-test-diff-triage
Command: npx skills add https://github.com/pingcap/tidb --skill tidb-test-diff-triage

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

TiDB contributors often see unexpected test diffs in planner, executor, or testdata outputs that seem unrelated to their PR, especially after merges or rebases. This Skill provides a structured triage workflow to determine whether the diff is a failpoint setup issue, an upstream behavior change, or a genuine local regression before touching expected outputs.

Core Features & Use Cases

  • Failpoint-first triage: Checks whether the affected package relies on failpoint instrumentation and reruns tests with failpoints enabled plus -count=1 for reproducibility.
  • Merge impact isolation: Guides minimal reproduction with -run <TestName> and git bisect to find the first bad commit in a merged range.
  • Guarded testdata updates: Prevents premature syncing of expected plan/result outputs until the root cause is proven.
  • Use Case: After rebasing a PR, a planner test's expected output no longer matches. Use this Skill to check failpoint requirements, bisect the merged commits, and classify the diff before deciding whether to update testdata.

Quick Start

Triage this unexpected TiDB test diff in my planner test that appeared after rebasing, checking failpoint setup first and bisecting if needed.

Frequently Asked Questions about tidb-test-diff-triage

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I triage unexpected TiDB test diffs after a rebase?▼

First rule out failpoint setup by checking the affected package against the failpoint decision guide and rerunning with failpoints enabled plus -count=1. If the diff persists, reproduce minimally with -run and bisect the merged commit range to find the first bad commit.

Why do TiDB tests behave differently with and without failpoints?▼

Many TiDB test behaviors rely on failpoint instrumentation, and the -tags=intest,deadlock build tags do not enable failpoints. If a package has failpoint hits, tests must be rerun with failpoints enabled or the output may differ from expected results.

When should I update TiDB testdata expected outputs?▼

Only sync expected plan or result outputs after proving the cause: either an upstream merge intentionally changed optimizer behavior, or the existing expectation is stale while query semantics are unchanged. Never record testdata before identifying the root cause.

Why does a single TiDB test run differ from the full suite?▼

Differences between single-test and full-suite runs often indicate environment or setup issues rather than logic regressions. Rerun with failpoints enabled and add -count=1 to the go test command to eliminate caching and confirm reproducibility.

How do I find which commit broke a TiDB test?▼

Use git bisect over the merged range: mark the known bad and good commits, then let bisect narrow to the first bad commit. Identify that commit before updating any expected test outputs.

Related Skills

Explore More Software Engineering Skills→
yves-syves-s
24

systematic-debugging

Identify root causes of bugs and test failures through structured investigation.

Community
Advanced
tribedotruntribedotrun
8

mando-pr

Create GitHub pull requests and trigger AI reviewers with CI monitoring.

Official
Advanced
D3HunterD3Hunter

tidb-coding-reference

Standardize TiDB coding contributions with implementation rules and quality guardrails.

Community
Advanced
hey-konghey-kong

sglang-bisect-ci-regression

Extract failure signatures and bisect CI regression commit ranges in SGLang workflows.

Community
Advanced