null-operation-invariance-audit

Audits permutation nulls to confirm the shuffle perturbs the object the test statistic consumes.

1|Updated Dec 13, 2025
One-click install
npx skills add https://github.com/ZK-Theory/TDL --skill null-operation-invariance-audit-zk-theory
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: null-operation-invariance-audit
Source: https://github.com/ZK-Theory/TDL/tree/main/.agents/skills/null-operation-invariance-audit
Command: npx skills add https://github.com/ZK-Theory/TDL --skill null-operation-invariance-audit-zk-theory

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Permutation and shuffle nulls can silently leave the test statistic unchanged — for example, shuffling rows of a point cloud whose persistence diagram is invariant to row order — making the test structurally unable to reject anything. This Skill audits whether the null operation actually perturbs the object the statistic is computed on, catching invariant or degenerate nulls before they produce meaningless p-values. ## Core Features & Use Cases - Static Trace Check: Determines without running any compute whether the null's perturbed variable even appears in the statistic's construction, catching label permutations against label-blind statistics and row permutations against set-invariant constructions. - Empirical Witnesses: Verifies that at least one non-identity permutation changes the statistic, and that full statistic outputs match in shape, grid, and reduction behavior between observed data and null draws. - Centering/Degeneracy Check: Detects nulls fit from the same sufficient statistic the test statistic factors through (e.g., a Markov-k null whose MLE is the observed transition-count matrix), which force p-values toward 0.5 regardless of real signal. - Use Case: When a persistent-homology battery reports negative-control p-values pinned near 0.5, run this audit to determine whether the shuffle is applied downstream of the embedding, rendering the null structurally powerless. ## Quick Start Audit my permutation null to confirm the shuffle actually changes the object my test statistic is computed on.

Frequently Asked Questions about null-operation-invariance-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I check if a permutation null can actually reject my test statistic?

Trace the statistic as T(input-object) and confirm the shuffled variable appears in that construction upstream of the statistic. Then run one non-identity permutation and assert the statistic value changes; if it does not, the null is invariant and cannot reject.

Why is my permutation test p-value always near 0.5?

A p-value pinned near 0.5 often means the null is invariant to the shuffle or degenerate — fit from the same sufficient statistic the test statistic factors through, such as a Markov-1 null built from the observed transition-count matrix. Non-zero null variance alone does not rule this out.

Can a row shuffle fail to change a persistence diagram?

Yes. A persistence diagram is a set, invariant to row order of the input point cloud, so shuffling already-embedded rows leaves W2(observed, null) at zero for every draw. The shuffle must be applied upstream of the embedding to have any effect.

What causes shape mismatch errors between observed and null statistics?

Mismatches arise when observed and null statistics are evaluated on different data-dependent grids, such as differing counts of active filtration steps. Compute the full statistic on one null draw and assert identical length, shape, and a shared fixed grid before integrating.

When should I use a two-sample design instead of a parametric null?

Use a two-sample design when the null model is fit from the same sufficient statistic the test statistic consumes, making the test structurally near-powerless. Alternatives include a lower-order null rung or a statistic computed on a richer substrate than the null's sufficient statistic.