flakiness-external

Detect and contain test flakiness caused by external network calls and real-time sleeps.

Updated Jun 24, 2026
One-click install
npx skills add https://github.com/Hakkadaikon/hymme --skill flakiness-external-hakkadaikon
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: flakiness-external
Source: https://github.com/Hakkadaikon/hymme/tree/main/skills/flakiness-external
Command: npx skills add https://github.com/Hakkadaikon/hymme --skill flakiness-external-hakkadaikon

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Tests that pass or fail unpredictably because they depend on real external services (HTTP APIs, DNS) or fixed real-time sleeps erode trust in the test suite and slow down CI. This Skill provides systematic detection and containment methods for these two flakiness sources. ## Core Features & Use Cases - External network containment: Detect real outbound communication by cutting the network and running repeated test iterations, then replace external calls with test doubles (MSW, stubs) that fix responses, including error cases like timeouts and 5xx. - Timer and sleep containment: Find fixed sleeps via grep, reproduce failures under CPU load, and replace them with fake timers (vi.useFakeTimers) or condition-based waiting (vi.waitFor, expect.poll). - Completion checklists: Each factor ships with verification criteria, such as network-cut runs passing 20 consecutive iterations and zero remaining fixed sleeps under load. - Use Case: A Vitest suite intermittently fails on CI. Use this Skill to block outbound network access, identify tests hitting real APIs, stub fetch responses, and convert setTimeout-based waits into condition polling until the suite is green across repeated runs. ## Quick Start Use the flakiness-external skill to find and fix the flaky tests in my Vitest suite that depend on real API calls and fixed sleeps.

Frequently Asked Questions about flakiness-external

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find flaky tests caused by real network calls?

Block outbound network access (offline mode, DNS block, or proxy cutoff) and run the suite repeatedly, for example vitest run --repeat=20. Any test that fails under network isolation still depends on real external communication and must be stubbed.

How to replace fixed sleep in Vitest tests?

Replace fixed sleeps with condition-based waiting using vi.waitFor or expect.poll so the test proceeds as soon as the condition holds. For timer-driven code, use vi.useFakeTimers with vi.advanceTimersByTime and restore real timers in afterEach.

Does MSW catch unmocked HTTP requests in tests?

Yes, MSW supports the onUnhandledRequest: "error" option, which makes any unexpected real request fail immediately. You can verify the setting exists by grepping for onUnhandledRequest in your project configuration.

Why do tests pass locally but fail on CI?

Fixed real-time sleeps assume operations finish within a set duration, but slower or loaded CI machines violate that assumption. Reproduce the failure by running tests under CPU limits or load with vitest run --repeat=50 to expose the timing dependency.

When should I use real external services instead of mocks?

Reserve real external communication for a small set of contract or smoke tests isolated from the main suite under separate tags. Managed dependencies you control, like your own database, can use real engines via Testcontainers instead of mocks.