SprocketLabOfficial·8 Agent Skills Included

slop-code-bench

Benchmark coding agents under evolving specifications and checkpoints

Evaluates coding agents as specifications change across checkpoints, measuring correctness, code quality, and structural stability. Replaces one-shot benchmarks that miss path dependence, non-convergence, and code erosion over iterative changes. Runs agents in isolated Docker environments, grades results with pytest and LLM judges, and visualizes outcomes in a dashboard.

138|30

↗

npx skills add SprocketLab/slop-code-bench --all -g -y

Available:

Instructs the AI agent on how to run benchmark problems, evaluate checkpoints in Docker, and use the built-in testing and fixing skills correctly.

All Skills in This Repository (8)

Pure Emerald Level Indicators

📦 In Repo

SprocketLab

edge-cases

Compare checkpoint specifications with existing tests to generate edge-case skeleton tests.

reclassify-tests

Mark tests not demonstrated in checkpoint specs with pytest functionality marks.

fix-solution

Automate iterative test-and-fix loops for benchmark problems across checkpoints.

test-ambiguity-detector

Compare tests against specifications to identify ambiguous assumptions.

run-tests

Run eval-snapshot benchmark tests for coding solutions inside Docker.

validate-run

Validate all checkpoints in an agent run directory with parallel validation and summarized results.

audit-spec

Audit checkpoint specifications for redundant guidance and design giveaways.

checkpoint-ambiguity-review

Compare checkpoint specs with tests to identify ambiguous interpretations and propose fixes.

Official

Intermediate

Frequently Asked Questions

FAQPage Schema

How to install slop-code-bench?▼

Run `npx skills add SprocketLab/slop-code-bench --all -g -y` in your terminal to install all skills in this suite globally.

What does SlopCodeBench measure?▼

It measures how coding agents handle iterative specification changes, exposing path dependence, non-convergence, and code quality erosion that single-shot benchmarks cannot detect.

Which coding agents does it support?▼

It supports Claude Code, Codex, Gemini, Cursor CLI, OpenHands, OpenCode, and other agents through a pluggable registry system.

How are benchmark solutions tested?▼

Solutions run inside isolated Docker containers and are graded with pytest checkpoints covering core, functionality, regression, and error-handling test groups.

Can I add my own benchmark problems?▼

Yes. Problems are defined with a config.yaml, checkpoint specs, and pytest tests, and new problems can be contributed to the companion scb-problems repository.