Score AI models on the 113-task DeepSWE benchmark via OpenRouter.
npx skills add https://github.com/Matymatyk-business/david-skills --skill run-deep-swe
Please help me install this Agent Skill. Skill: run-deep-swe Source: https://github.com/Matymatyk-business/david-skills/tree/main/skills/run-deep-swe Command: npx skills add https://github.com/Matymatyk-business/david-skills --skill run-deep-swe
This skill enables automated, reproducible scoring of AI models on the 113-task DeepSWE coding-agent benchmark via OpenRouter, allowing fair comparisons and verification of vendor claims.
Run a full 113-task DeepSWE evaluation for a selected model via OpenRouter and review the resulting score.
High-intent search queries and answers about installing and using this skill.
To benchmark AI models on DeepSWE, you can use this skill to run automated, reproducible scoring via the OpenRouter API. It executes single-task, subset, or full 113-task runs end-to-end using a compatible mini-swe-agent driver.
Running DeepSWE evaluations requires OpenRouter API access, a compatible mini-swe-agent driver, and a configured environment with Docker, uv, and a model slug. This setup ensures tasks execute correctly end-to-end within the sandbox.
Yes, you can evaluate a subset of DeepSWE tasks instead of the full 113-task corpus. The skill supports single-task, subset, and full-corpus runs to provide flexible model evaluation options via OpenRouter.
DeepSWE benchmark evaluation is used for automated, reproducible scoring of AI models on a 113-task coding-agent benchmark. It allows fair comparisons and verification of vendor claims through model-agnostic testing.
Yes, this DeepSWE evaluation tool supports optional leaderboard submission. After running your model evaluation through OpenRouter, you can submit the resulting comparable scores directly to the leaderboard.
Using OpenRouter and Docker for model evaluation on DeepSWE ensures model-agnostic benchmarking and sandboxed task execution. This approach guarantees reproducible scoring by isolating the environment and standardizing API access.
AngelP17Generate and edit images from prompts using AI models via OpenRouter API.
surus-latMerge local benchmark results into the HuggingFace LatamBoard leaderboard dataset.
paxlabs-incBenchmark large language models across academic reasoning, coding, and knowledge tasks.
AlphaLab-USTCRun OhMyCode coding benchmarks and record pass-fail outcomes with token usage.
