multiverse-analysis

What problem does it solve?

Multiverse analysis prevents overconfidence by showing how your conclusions change when you vary defensible analytical decisions, making researcher degrees of freedom visible rather than hidden.

Core Features & Use Cases

• Decision inventory & mapping: Identify selectional, operationalizational, statistical, and computational decision points in an end-to-end research pipeline.
• Type E/N/U classification: Classify decisions as Type E (equivalent), Type N (non-equivalent), or Type U (uncertain) to avoid contaminating the multiverse with wrong estimands or incomparable questions.
• Multiverse execution & specification curves: Run many specifications, summarize convergence/failures, and produce a specification curve (Simonsohn, Simmons & Nelson, 2020) to communicate robustness transparently.
• Mini-multiverse for expensive pipelines: Use a targeted subset when computational cost is high (e.g., LLMs, Bayesian models, networks/ERGM).
• Computational failure detection: Explicitly report which decision combinations fail to converge so failures become part of the evidence.

Quick Start

Run the provided demo to generate a Type E multiverse, print a specification-curve style table, and observe how changing decisions can flip robustness verdicts.