check-test-separation

Detect mixed unit and integration tests in Python pytest projects via AST analysis.

Updated Sep 9, 2024

Search Tags:#pytest #python #code-quality #static-analysis #ast #tests #test-separation

Ready to run

npx skills add https://github.com/axel-kaliff/dotfiles --skill check-test-separation

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: check-test-separation
Source: https://github.com/axel-kaliff/dotfiles/tree/main/claude/.claude/skills/check-test-separation
Command: npx skills add https://github.com/axel-kaliff/dotfiles --skill check-test-separation

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill detects and reports when unit tests and integration tests are improperly mixed in a Python/pytest codebase, helping teams maintain clear test boundaries and reduce flaky or slow test suites.

Core Features & Use Cases

Validates directory placement under tests/unit/ and tests/integration/ and flags unclassified test files.
Detects pytest markers (@pytest.mark.unit / @pytest.mark.integration), integration-signaling imports and call patterns, heavy mocking, and problematic conftest fixtures.
Use cases include pre-commit or CI checks for changed files, local code review of test hygiene, and auditing large repositories for misplaced integration work.

Quick Start

Run the included AST-based checker against changed or specified test files to receive a formatted report of separation violations.

Frequently Asked Questions about check-test-separation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I detect mixed unit and integration tests in a pytest project?▼

Separating unit and integration tests in Python prevents slow, flaky test suites by maintaining clear test boundaries. This AST-based checker analyzes directory placement, pytest markers, integration-signaling imports, and mock usage to flag improperly mixed tests and report violations.

Can I check test separation on only the changed files in my CI pipeline?▼

Yes, you can check test separation on only changed files in a CI pipeline. The checker supports analyzing changed files, specific directories, or single test files, returning a non-zero exit code when error-level violations are found for pre-commit or CI checks.

How does AST analysis identify integration tests in a Python codebase?▼

AST analysis identifies integration tests by inspecting test files for integration-signaling imports, decorator patterns like pytest markers, call patterns, and counting mock usage. It also inspects conftest fixtures to flag unclassified test files and validate directory placement.

What is the best way to enforce test directory placement under tests/unit/ and tests/integration/?▼

The best way to enforce test directory placement under tests/unit/ and tests/integration/ is using an AST-based static analysis checker that flags unclassified test files. This validates directory structure and reports separation violations to maintain clear test boundaries.

Why does heavy mocking in unit tests indicate mixed integration test logic?▼

Heavy mocking in unit tests indicates mixed integration test logic because excessive mock usage signals complex dependency management that belongs in integration tests. The checker counts mock usage and inspects conftest fixtures to detect these problematic patterns and report violations.