Sourcegraph
Official@sourcegraph
Helping developers search, understand, and write code in complex codebases with AI
Agent Skills by Sourcegraph
Showing 25 vetted skills indexed across 2 GitHub repositories.
sourcegraphvalidate-tasks
Validate benchmark task definitions for truncated instructions and missing test scripts.
sourcegraphir-analysis
Compute precision, recall, MRR, nDCG, and MAP for file retrieval comparisons.
sourcegraphsync-metadata
Reconcile task metadata between selected_benchmark_tasks.json and task.toml files.
sourcegraphmcp-audit
Analyze MCP tool usage patterns and performance deltas from task_metrics.json files.
sourcegraphEvaluate Benchmark Traces
Evaluate AI agent benchmark traces for data integrity, output quality, and efficiency.
sourcegraphreextract-metrics
Re-extract task metrics from completed benchmark runs after extraction fixes.
sourcegrapharchive-run
Archive completed benchmark runs older than a specified duration.
sourcegraphscore-tasks
Score benchmark tasks on clarity, verifier quality, and reproducibility with Python scripts.
sourcegraphrepo-health
Validate documentation, configurations, and task definitions before commits or pushes.
sourcegraphrun-status
Retrieve and summarize active benchmark run statuses within a specified time frame.
sourcegraphquick-rerun
Execute a single benchmark task locally with the Harbor runner.
sourcegraphwhats-next
Analyze benchmark run status and recommend prioritized next actions for AI coding agents.
sourcegraphcompare-configs
Compare benchmark results across AI agent configurations to identify performance divergences.
sourcegraphtriage-failure
Diagnose failed CodeScaleBench tasks by analyzing logs, result JSON, and agent transcripts.
sourcegraphcost-report
Analyzes AI token usage and estimated costs for benchmark runs, suites, and configurations with JSON output options.
sourcegraphgenerate-report
Generate aggregate CodeScaleBench evaluation reports from Harbor runs in markdown, JSON, and CSV.
sourcegraphcheck-infra
Validate API tokens, environment variables, Docker status, and disk space for benchmark tasks.
sourcegraphwatch-benchmarks
Monitor benchmark run statuses and generate structured JSON or table summaries.
sourcegraphbenchmark-audit
Audit benchmark suites against the Agent Benchmark Criteria framework.
sourcegraphrun-benchmark
Configure and launch CodeScaleBench benchmark runs with paired-run and curation guardrails.
sourcegraphscaffold-task
Generate Harbor-compatible CodeScaleBench task files and suite configurations.
sourcegraphmine-tasks
Mine merged pull requests to generate benchmark tasks with Docker environments.
sourcegraphbigquery
Run BigQuery SQL queries and export results using the bq CLI.
sourcegraphtmux
Manage concurrent processes in tmux windows and capture output.
Frequently Asked Questions About Sourcegraph
FAQPage SchemaWhat specific tasks can I perform using Sourcegraph's benchmarking capabilities?▼
You can configure and launch CodeScaleBench runs, audit benchmark suites against established criteria, and generate aggregate performance reports. The system enables local task execution via the Harbor runner, failure triage through log analysis, and cost estimation based on token usage metrics.
Which personas benefit most from these benchmarking and analysis features?▼
These capabilities are designed for software engineers, quality assurance leads, and infrastructure architects. Professionals managing large-scale codebases or evaluating coding performance will find the structured reporting, failure diagnostics, and repository health validation essential for maintaining high-quality development standards.
What are the primary prerequisites for running these benchmark tasks?▼
Successful execution requires a configured environment with valid credentials, including API tokens and necessary environment variables. Additionally, the host system must maintain active Docker status and sufficient disk space to support the containerized environments required for task mining and local benchmark execution.