skills.rest
sourcegraph
sourcegraphOFFICIAL
Explore all skills by this author →
Website𝕏@sourcegraphGitHub

triage-failure

Diagnose failed CodeScaleBench tasks by analyzing logs, result JSON, and agent transcripts.

32|4|Updated Feb 1, 2026
Search Tags:#debugging#root cause#logs#failure analysis#benchmark#triage
One-click install
npx skills add https://github.com/sourcegraph/CodeScaleBench --skill triage-failure
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: triage-failure
Source: https://github.com/sourcegraph/CodeScaleBench/tree/main/skills/triage-failure
Command: npx skills add https://github.com/sourcegraph/CodeScaleBench --skill triage-failure

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires json, sys, os, re, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps developers quickly identify the root cause of failed benchmark tasks, understand the failure type, and receive actionable suggestions for remediation.

Core Features & Use Cases

  • Automated Failure Analysis: Parses logs and task outputs to diagnose errors.
  • Root Cause Identification: Categorizes failures into types like infrastructure, agent bugs, or task difficulty.
  • Suggested Fixes: Provides specific commands or code changes to resolve the issue.
  • Use Case: When a benchmark run fails, use this Skill to get an immediate diagnosis and a clear path to fixing the problem, rather than manually sifting through logs.

Quick Start

Use the triage-failure skill to investigate the most recent benchmark failure.

Frequently Asked Questions about triage-failure

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I find the root cause of a failed benchmark task?▼

To find the root cause of a failed benchmark task, you can automatically parse task logs and result JSON to categorize failure types and identify whether errors stem from infrastructure issues, agent bugs, or task difficulty.

What is the best way to analyze CodeScaleBench failure logs?▼

Analyzing CodeScaleBench failure logs is best handled by automated triage scripts that investigate task outputs and agent transcripts to diagnose errors and output specific remediation steps.

Can I get suggested fixes for benchmark failures from task logs?▼

Yes, you can get suggested fixes for benchmark failures by analyzing task logs and agent transcripts, which generates specific commands or code changes needed to resolve the identified issue.

Does automated failure triage work with multiple failed benchmark tasks?▼

Automated failure triage does work with multiple failed benchmark tasks, supporting batch investigation of various result JSON files and logs to diagnose errors across an entire run.

Why do I need to analyze agent transcripts for benchmark debugging?▼

You need to analyze agent transcripts during benchmark debugging to perform behavioral analysis of tasks, helping pinpoint exactly where agent bugs or task difficulty caused the execution to fail.

Related Skills

Explore More Software Engineering Skills→
sourcegraphsourcegraph
32

scaffold-task

Generate Harbor-compatible CodeScaleBench task files and suite configurations.

Official
Advanced
sourcegraphsourcegraph
32

run-benchmark

Configure and launch CodeScaleBench benchmark runs with paired-run and curation guardrails.

Official
Advanced
sourcegraphsourcegraph
32

validate-tasks

Validate benchmark task definitions for truncated instructions and missing test scripts.

Official
Intermediate
chkp-ronizchkp-roniz
4

debugger

Diagnose Python bugs via MCPDebug breakpoints and runtime state inspection in VS Code.

Community
Advanced