skills.rest
514-labs
514-labsOFFICIAL
Explore all skills by this author →
Website𝕏@514hqGitHub

dec-bench-create-scenario

Scaffold DEC Bench evaluation scenarios with directory structure and CLI tooling.

1|2|Updated Feb 22, 2026
Search Tags:#automation#workflow#cli#scaffold#registry#scenario#dec-bench
One-click install
npx skills add https://github.com/514-labs/agent-evals --skill dec-bench-create-scenario
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: dec-bench-create-scenario
Source: https://github.com/514-labs/agent-evals/tree/main/.claude/skills/dec-bench-create-scenario
Command: npx skills add https://github.com/514-labs/agent-evals --skill dec-bench-create-scenario

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Create a new DEC Bench evaluation scenario scaffold by collecting user context and scaffolding the required directory structure with the CLI.

Core Features & Use Cases

It is used when defining new scenarios across multiple harnesses, with scenario.json, harness prompts, and shared init data, including per-harness init. The skill guides you through constructing the required prompts, init blocks, and metadata to conform to the DEC Bench contract and registry workflow.

Quick Start

Describe your task and run dec-bench create to scaffold it.

Frequently Asked Questions about dec-bench-create-scenario

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I create a new DEC Bench evaluation scenario?▼

You need to provide task, domain, tier, and starting-state information. The skill then guides you through constructing prompts, init blocks, and metadata to conform to the DEC Bench contract and CLI tooling.

What is a DEC Bench scenario scaffold?▼

A DEC Bench scenario scaffold is an automated directory structure containing scenario.json, harness prompts, and shared init data. It is needed when defining new evaluation scenarios across multiple harnesses.

Can I define per-harness init data when scaffolding a scenario?▼

Yes, you can define per-harness init data. The skill scaffolds scenarios across multiple harnesses, guiding you through constructing shared init blocks and harness-specific prompts for the registry workflow.

What's the best way to scaffold evaluation scenarios across multiple harnesses?▼

The best way to scaffold evaluation scenarios across multiple harnesses is using the dec-bench create CLI command. It automates directory structuring and ensures your scenario.json and harness prompts adhere to the DEC Bench contract.

Do I need to manually create scenario.json for DEC Bench?▼

No, you do not need to manually create scenario.json. The skill automates scaffolding the correct directory structure and metadata generation, ensuring compliance with DEC Bench CLI tooling and registry workflow.

Related Skills

Explore More Software Engineering Skills→
514-labs514-labs
1

dec-bench-quickstart

Validate DEC Bench prerequisites and execute the first benchmark scenario.

Official
Intermediate
514-labs514-labs
1

dec-bench-run

Build and run DEC Bench scenarios, inspect results, and open the audit UI.

Official
Intermediate
514-labs514-labs
1

dec-bench-local-test

Compile the DEC Bench CLI from source and run scenario tests in Docker containers.

Official
Intermediate
514-labs514-labs
1

dec-bench-local-override

Substitute a locally-built artifact into a DEC Bench scenario image using --override during build and run.

Official
Intermediate