skills.rest
estebandegetau
estebandegetauCOMMUNITY
Explore all skills by this author →
Website𝕏@estebandegetauGitHub

codebook-yaml

Standardize and validate YAML codebook definitions for LLM fiscal shock annotation workflows.

1|1|Updated Nov 20, 2025
Search Tags:#research#validation#llm#yaml#standards#annotation#codebook
One-click install
npx skills add https://github.com/estebandegetau/Fiscal-shocks --skill codebook-yaml
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: codebook-yaml
Source: https://github.com/estebandegetau/Fiscal-shocks/tree/main/.claude/skills/codebook-yaml
Command: npx skills add https://github.com/estebandegetau/Fiscal-shocks --skill codebook-yaml

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

This Skill codifies a single, enforceable format for YAML codebooks used to drive LLM-based annotation and evaluation, removing ambiguity and ensuring reproducible behavioral testing across C1-C4 codebooks.

Core Features & Use Cases

  • Formalized Structure: Enforces required top-level fields, class definitions, and example formats for prompts/c1_.yml through prompts/c4_.yml.
  • Validation & Testing Guidance: Specifies S0-S3 stage-gate workflow, S1 behavioral tests, and S3 ablation procedures to verify model behavior.
  • Country-Agnostic & Ablation-Ready Rules: Mandates neutral terminology in definitions and independent clarification items for robust ablation studies.
  • Use Case: Create or revise a C2 motivation codebook and verify it passes definition recovery, example recovery, and order-invariance tests before evaluation.

Quick Start

Create a new YAML codebook in prompts/ following the SKILL conventions and run the S1 behavioral tests for immediate validation.

Frequently Asked Questions about codebook-yaml

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I standardize YAML codebooks for LLM annotation workflows?▼

Standardize YAML codebooks by enforcing required top-level fields, class definitions, and specific output_instructions to remove ambiguity and ensure reproducible LLM-driven annotation evaluation.

What validation tests are needed for LLM labeling codebooks?▼

LLM labeling codebooks require S1 behavioral tests including definition recovery, example recovery, and order-invariance tests to verify model behavior before running full evaluations.

How do I ensure my YAML codebook is ablation-ready for model evaluation?▼

Make YAML codebooks ablation-ready by mandating independent clarification items, enforcing field-level constraints, and applying country-agnostic language rules to definitions for robust ablation studies.

What is the stage-gate workflow for developing LLM annotation codebooks?▼

The stage-gate workflow spans S0 through S3 development stages, covering codebook authoring, S1 behavioral testing, and S3 ablation procedures to verify reproducible model evaluation.

Does my YAML codebook need country-agnostic definitions for LLM labeling?▼

YAML codebooks require country-agnostic language rules mandating neutral terminology in definitions to ensure consistent LLM-driven annotation across different regional contexts.

Why do my LLM annotation results vary when using YAML codebooks?▼

Variability often stems from unenforced field-level constraints and ambiguous output_instructions in YAML codebooks, which prevent reproducible behavioral testing across C1-C4 annotation stages.

Related Skills

Explore More Education & Research Skills→
WeiYu-NUSWeiYu-NUS

paper-standardizer

Standardize raw paper metadata from Excel into analysis-ready datasets with a reproducible codebook.

Community
Advanced
estebandegetauestebandegetau
1

review-iteration

Diagnose pipeline run failures and map error patterns across stages and codebooks.

Community
Advanced
mcauduro0mcauduro0
1

senior-prompt-engineer

Analyze and optimize LLM prompts with Python scripts.

Community
Advanced
estebandegetauestebandegetau
1

log-iteration

Log codebook development iterations as YAML trails with version, commit, and decisions.

Community
Intermediate