skills.rest
manusabbath-arch
manusabbath-archCOMMUNITY
Explore all skills by this author ā†’
GitHub

empirical-project-playbook

Applies pre-registration, anti-mining checks, and build-failing guards to empirical projects built with AI agents.

Updated Sep 11, 2026
Search Tags:#verification#ai agents#pre-registration#ci checks#incident patterns#anti-mining#empirical methodology
One-click install
npx skills add https://github.com/manusabbath-arch/hermes-skills --skill empirical-project-playbook-manusabbath-arch
Or copy as Structured Prompt for Agentā–¼
Please help me install this Agent Skill.
Skill: empirical-project-playbook
Source: https://github.com/manusabbath-arch/hermes-skills/tree/main/skills/empirical-project-playbook
Command: npx skills add https://github.com/manusabbath-arch/hermes-skills --skill empirical-project-playbook-manusabbath-arch

SYSTEM DOCUMENTATION & REQUIREMENTS

šŸ’” This Skill includes references (resource) components.

What problem does it solve? Empirical projects built with AI agents often fool themselves: findings mined post-hoc, features declared done when they merely run, and silent bugs that recur because prose rules never stop them. This Skill provides a decision method with mechanized, build-failing checks so projects that measure something (edge, conversion, performance, a business hypothesis) actually decide with trustworthy data. ## Core Features & Use Cases - Pre-registration and kill criteria: Fix success/failure thresholds before seeing data, with a findings registry lifecycle (PROPOSED ā†’ TESTING ā†’ SUPPORTED ā†’ CONFIRMED / RETRACTED / SUPERSEDED) enforced by code guards. - Mechanized anti-patterns: A catalog of 'treacherous silence' bug families (phantom wiring, writers that never commit, gates that cannot distinguish failed from never-ran) turned into checks that fail the build. - Verification and security standards: Tests that must fail against pre-fix code, hermetic tests, supply-chain gates that can actually fail, and AI-agent security countermeasures (git guards, escape hatches, prompt-injection defenses). - Use Case: When starting a trading, ML, or analytics project with Claude Code or Hermes, load this playbook to produce a pre-registration memo, a checks checklist, and incident-pattern records before writing production code. ## Quick Start Ask the agent to load the empirical-project-playbook skill and create a pre-registration memo with kill criteria for your hypothesis before implementing any feature.

Frequently Asked Questions about empirical-project-playbook

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I prevent data mining when testing a hypothesis with AI agents?ā–¼

Use pre-registration: write a memo fixing kill and success criteria (e.g., kill if EV<0 at nā‰„80) before seeing any data. Then replicate findings on a fresh sample that did not participate in the discovery, enforced by a strict freshness cutoff.

What is the difference between presence and utility when accepting a feature?ā–¼

Presence means the code runs without error; utility means a production query answers the question the feature was built for, with plausible values in post-deploy rows. A feature is only done when the pre-written acceptance query passes against production data.

How do I make CI security gates actually block merges?ā–¼

Remove `|| true` and `continue-on-error` from security steps like bandit and pip-audit so they can fail the build. Pin dependency versions exactly in lockfiles and verify the gate fails when a vulnerability is introduced.

What are phantom wiring bugs and how do I detect them?ā–¼

Phantom wiring occurs when code reads state via `getattr(obj, "x", {})` that nothing ever assigns, so nothing crashes but the caller operates on empty state forever. Detect it by verifying `hasattr` or tracing which module actually assigns the attribute, not by grepping the name.

When should I use this playbook versus a general coding harness?ā–¼

Use this playbook when the project measures something and decides with data; it covers what to measure and how to avoid self-deception. Pair it with an agent-coding harness skill for validation loops and lint baselines, which apply even to non-empirical projects.

Related Skills

Explore More Software Engineering Skillsā†’
soeiroosoeiroo

metrics-definition

Define UX metrics and KPIs that connect design decisions to measurable business outcomes.

Community
Intermediate
rritecrritec
3

e2e-medallion-architecture

Design and orchestrate end-to-end Medallion Architecture solutions in Microsoft Fabric.

Community
Advanced
KrilliacKrilliac
6

sonder-validation-and-qa

Defines evidence doctrine, pytest harness contract, and acceptance ratchets for Sonder Runtime verification.

Community
Advanced
malloydatamalloydata
95

notebook-chat

Execute ad-hoc data queries on in-memory notebook contexts.

Official
Advanced