skills.rest
nagisanzenin
nagisanzeninCOMMUNITY
Explore all skills by this author →
GitHub

crucible

Enforce gated scientific pipelines for AI-driven experiments with pre-registration and power analysis.

Updated Jul 9, 2026
Search Tags:#research#evaluation#data analysis#statistics#experimentation#scientific method
One-click install
npx skills add https://github.com/nagisanzenin/crucible --skill crucible-nagisanzenin
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: crucible
Source: https://github.com/nagisanzenin/crucible/tree/main/skills/crucible
Command: npx skills add https://github.com/nagisanzenin/crucible --skill crucible-nagisanzenin

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This skill prevents AI agents from producing unreliable research by enforcing hard gates on pre-registration, sample size calculation, and blind independent judging, ensuring conclusions are defensible rather than built on sand.

Core Features & Use Cases

  • Pre-registration Lock: Hashes and freezes study designs to prevent HARKing (Hypothesizing After Results are Known).
  • Statistical Power Enforcement: Automatically computes required sample sizes to ensure studies are adequately powered.
  • Blind Independent Judging: Orchestrates a separate agent to score outputs without bias, blocking conclusions if the judge is unqualified.
  • Use Case: Use this when conducting A/B tests, model evaluations, or ablations where the integrity of the conclusion is critical for downstream product decisions.

Quick Start

Use the crucible skill to initialize a new study with the goal of comparing the correctness of two different prompt strategies.

Frequently Asked Questions about crucible

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent HARKing when running AI-driven A/B testing and model evaluation experiments?▼

Statistical power enforcement automatically computes the required sample sizes for experimentation, ensuring your A/B tests and model evaluations are adequately powered to detect meaningful effects before data collection begins.

How do I eliminate bias when using AI agents to score experimental outputs in ablation studies?▼

To eliminate bias in ablation studies, you can orchestrate blind independent judging where a separate agent scores outputs without exposure to condition labels, blocking conclusions if the judge lacks qualification.

What is the best way to ensure statistical validity and defensibility in AI-driven research pipelines?▼

The best way to ensure statistical validity is to apply a gated pipeline that hard-blocks conclusions unless pre-registration, sample size calculation, and blind judging requirements are fully satisfied.

Do I need a pre-registered study design to run model evaluation comparisons with this scientific method approach?▼

Yes, pre-registration is required. The pipeline hashes and freezes your study design before execution to prevent HARKing, ensuring your model evaluation conclusions remain defensible.

When should I not use automated statistical power enforcement for data analysis?▼

You should not use automated statistical power enforcement for data analysis when your research goal is exploratory rather than confirmatory, as the gated pipeline requires formal hypotheses and pre-registration to function correctly.

Related Skills

Explore More Education & Research Skills→
product-on-purposeproduct-on-purpose
527

measure-experiment-design

Design end-to-end experiment plans with hypotheses, variants, metrics, and guardrails.

Official
Advanced
chenghaoYangchenghaoYang
2

report

Generate structured technical reports from experiment results in Markdown or LaTeX.

Community
Intermediate
han-so1omonhan-so1omon
1

dispatching-parallel-agents

Deploy isolated agents to resolve unrelated failures in parallel.

Community
Advanced
stanislavjiricekstanislavjiricek
6

phase-preregistration

Draft and manage pre-registration documents for scientific research registries.

Community
Intermediate