write-evals
OfficialAutomate probabilistic behavioral evals authoring and review for model output beads
Software Engineering#software development#Claude Code#model validation#eval authoring#probabilistic evaluation
AuthorDektora
Version1.0.0
Installs0
System Documentation
What problem does it solve?
This Skill automates the creation, audit, review, revision, and acceptance of probabilistic behavioral evals for beads that produce model output, streamlining the process and ensuring accuracy.
Core Features & Use Cases
- Eval Authoring: Write evals for beads that produce model output, specifying scenarios, thresholds, and pass criteria.
- Audit: Validate existing evals against their bead/IB for continued validity.
- Review: Interactively walk through open issues in eval sets for resolution.
- Resync: Re-derive evals after the source IB has been resynced.
- Revise: Update existing evals based on engineer feedback.
- Accept: Promote evals from DRAFT to ACCEPTED after a clean audit.
- Dry-Run: Preview eval count, types, and scenarios without drafting full eval files.
- Use Case: Before a coding session begins, use this Skill to author evals for a bead that produces model output, ensuring the model's behavior is within expected parameters.
Quick Start
Run the write-evals skill with the --teaching flag to start the interactive tutorial for authoring evals for a bead.
Dependency Matrix
Required Modules
claude-opus-4-7
Components
scriptsreferences
💻 Claude Code Installation
Recommended: Let Claude install automatically. Simply copy and paste the text below to Claude Code.
Please help me install this Skill: Name: write-evals Download link: https://github.com/Dektora/dekspec-public/archive/main.zip#write-evals Please download this .zip file, extract it, and install it in the .claude/skills/ directory.
Agent Skills Search Helper
Install a tiny helper to your Agent, search and equip skill from 620,000+ vetted skills library on demand.