skills.rest
sourcegraph
sourcegraphOFFICIAL
Explore all skills by this author →
Website𝕏@sourcegraphGitHub

mine-tasks

Mine merged pull requests to generate benchmark tasks with Docker environments.

32|4|Updated Feb 1, 2026
Search Tags:#mcp#task generation#devops automation#code benchmarking#ai agent evaluation#git mining
One-click install
npx skills add https://github.com/sourcegraph/CodeScaleBench --skill mine-tasks
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: mine-tasks
Source: https://github.com/sourcegraph/CodeScaleBench/tree/main/skills/mine-tasks
Command: npx skills add https://github.com/sourcegraph/CodeScaleBench --skill mine-tasks

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes references (resource) components.

What problem does it solve?

This Skill automates the creation of benchmark tasks for evaluating AI coding agents by mining real-world code changes from repositories.

Core Features & Use Cases

  • Automated Task Generation: Mines merged PRs/MRs to create evaluation tasks where AI agents must reproduce known fixes or features.
  • Cross-Platform Compatibility: Works with GitHub, GitLab, Bitbucket, Azure DevOps, and local Git repositories.
  • Ground Truth Creation: Automatically generates ground truth data from code patches.
  • Use Case: You want to benchmark how well an AI coding agent can reproduce a bug fix from a merged pull request in your company's internal GitLab project. This Skill can mine that PR, create a runnable Docker environment for the agent, and provide the expected outcome.

Quick Start

Use the mine-tasks skill to mine 5-10 SDLC tasks from the GitHub repository 'https://github.com/sourcegraph/sourcegraph'.

Frequently Asked Questions about mine-tasks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate benchmark tasks for AI coding agents from merged pull requests?▼

Yes, this skill supports mining merge requests from GitHub, GitLab, Bitbucket, Azure DevOps, and local Git repositories to generate evaluation tasks. It automatically creates ground truth data and runnable Docker environments from the merged code changes across these platforms.

How does mining code repos for AI eval tasks work with MCP-augmented agents?▼

You can run quick evaluations by mining a small sample of 5-10 SDLC tasks from a target repository. This provides a fast subset of benchmark tasks and runnable Docker environments to test AI coding agents without performing full repository mining.

What is needed to benchmark AI coding agents using internal GitLab projects?▼

Benchmarking AI coding agents using internal GitLab projects requires access to the repository's merged merge requests to serve as ground truth. The skill mines these MRs to automatically build the evaluation tasks and expected outcomes needed for the benchmark.

Does this task generation tool support local Git repositories?▼

Yes, this task generation tool supports local Git repositories alongside cloud hosts like GitHub and Bitbucket. It mines the merged pull requests from these local repos to create runnable Docker environments and ground truth data for evaluating AI agents.

Related Skills

Explore More Software Engineering Skills→
ashiqcodeleafashiqcodeleaf

github-code-review

Automate code reviews on local git repositories and GitHub pull requests.

Community
Advanced
Srujan0798Srujan0798

performance

Benchmark system performance by measuring core metrics against established baselines.

Community
Advanced
databricks-solutionsdatabricks-solutions
5

genie-benchmark-generator

Generate and validate benchmark questions with SQL execution and MLflow synchronization.

Official
Advanced
humanerd-drewhumanerd-drew
2

qa-scenario-gen

Generate test scenarios and acceptance criteria from user requirements.

Official
Advanced