skills.rest
stephendor
stephendorCOMMUNITY
Explore all skills by this author →
WebsiteGitHub

null-operation-invariance-audit

Validate that permutation test null operations alter the test statistic.

1|Updated Dec 13, 2025
Search Tags:#statistical audit#permutation test#null operation#test statistic#shuffle analysis
One-click install
npx skills add https://github.com/stephendor/TDL --skill null-operation-invariance-audit
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: null-operation-invariance-audit
Source: https://github.com/stephendor/TDL/tree/main/.agents/skills/null-operation-invariance-audit
Command: npx skills add https://github.com/stephendor/TDL --skill null-operation-invariance-audit

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill verifies that a null operation in a permutation/shuffle test actually alters the object the test statistic is computed on, preventing false inferences.

Core Features & Use Cases

  • Test Statistic Invariance Check: Confirms that the shuffle is applied upstream of the statistic computation.
  • Empirical Witness Verification: Ensures at least one non-identity permutation yields a different statistic.
  • Pipeline Analysis: Identifies the shuffle location and the object it affects.
  • Use Case: For auditing permutation null models in stochastic processes, ensuring the null operation is effective.

Quick Start

Run the skill on your permutation test to ensure the null operation affects the test statistic: null-operation-invariance-audit -s your_test_script.py

Frequently Asked Questions about null-operation-invariance-audit

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
What does it mean if a permutation test null operation does not change the test statistic?▼

A null operation that does not alter the test statistic object causes false inferences. This audit checks permutation test null operations to verify the shuffle applies upstream of the statistic computation.

How do I validate that my shuffle analysis null model is effective?▼

You validate a shuffle analysis by running this audit script on your test. It performs an empirical witness verification to ensure at least one non-identity permutation yields a different statistic.

Why does my permutation test produce false negatives during stochastic process auditing?▼

False negatives occur if the shuffle location bypasses the object the test statistic is computed on. Pipeline analysis identifies the shuffle location to confirm the null operation perturbs the correct data.

Can I use this script to check test statistic invariance in any permutation null model?▼

Yes, this script checks test statistic invariance for any permutation null model. It is applicable to stochastic process auditing by confirming the null operation effectively perturbs the tested object.

Do I need to provide a specific script format to run the null operation invariance audit?▼

Yes, you need to provide your permutation test script for validation. The audit executes this script to analyze the pipeline and verify the null operation modifies the test statistic object.

Related Skills

Explore More Software Engineering Skills→
stephendorstephendor
1

markov-null-design

Generate Markov null models with explicit order for trajectory TDA permutation tests.

Community
Intermediate
stephendorstephendor
1

statistical-design-audit

Validate statistical designs for TDL tasks by checking denominators, p-values, and estimands.

Community
Advanced
parcadeiparcadei
3.9k

complete-skill

Execute an end-to-end test suite for the data persistence pipeline.

Community
Advanced
MonumentalSystemsMonumentalSystems
17

statistical-analysis

Translate raw data into statistical conclusions via hypothesis testing and forecasting.

Official
Advanced