Run OhMyCode coding benchmarks and record pass-fail outcomes with token usage.
npx skills add https://github.com/AlphaLab-USTC/OhMyCode --skill bench-alphalab-ustc
Please help me install this Agent Skill. Skill: bench Source: https://github.com/AlphaLab-USTC/OhMyCode/tree/main/.claude/skills/bench Command: npx skills add https://github.com/AlphaLab-USTC/OhMyCode --skill bench-alphalab-ustc
This Skill helps you measure how well a provider or model performs on realistic coding tasks, with token tracking and repeatable results.
Run the benchmark with your current configuration by asking to benchmark OhMyCode and then review the generated scorecard and results file.
High-intent search queries and answers about installing and using this skill.
Coding benchmarks compare model providers by running a fixed suite of SWE-bench-style tasks through OhMyCode. You can configure different providers and endpoints to execute the same workload and review the generated scorecard for pass-fail outcomes and token usage.
Regression testing for coding models identifies performance drops or efficiency gains after configuration changes. You need it when evaluating models or debugging failures, using benchmark scorecards to capture pass-fail outcomes, token usage, and runtime to distinguish model limitations from system bugs.
Yes, the benchmark harness supports configurable providers, models, and task filters. It requires this configurability to execute SWE-bench-style coding tasks and output machine-readable result files for accurate model evaluation and provider comparison.
Track token usage during model evaluation by running SWE-bench-style coding tasks through the benchmark harness. It records pass-fail outcomes alongside token consumption and runtime, generating a scorecard to analyze efficiency gains or regressions across different providers.
Yes, you can use benchmark scorecards for closed-loop debugging. By analyzing pass-fail outcomes, token tracking, and runtime data from SWE-bench-style tasks, you can distinguish between model limitations and OhMyCode system bugs after task failures.
DiologIREvaluate benchmark tasks for difficulty and fairness using statistics, rubric checks, mutation tests, and solution replays.
RhyTsuiRun automated lint, typecheck, test, and build gates before user review.
qcmuuEvaluate code generation models on HumanEval, MBPP, and MultiPL-E using pass@k metrics.
ValenBorrasCreate repeatable evaluation frameworks for Claude Code sessions with pass/fail criteria.
