triage-failure

Name: triage-failure
Availability: InStock
Author: Sourcegraph

Diagnose failed CodeScaleBench tasks by analyzing logs, result JSON, and agent transcripts.

32|4|Updated Feb 1, 2026

Search Tags:#debugging #root cause #logs #failure analysis #benchmark #triage

Ready to run

npx skills add https://github.com/sourcegraph/CodeScaleBench --skill triage-failure

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: triage-failure
Source: https://github.com/sourcegraph/CodeScaleBench/tree/main/skills/triage-failure
Command: npx skills add https://github.com/sourcegraph/CodeScaleBench --skill triage-failure

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires json, sys, os, re, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill helps developers quickly identify the root cause of failed benchmark tasks, understand the failure type, and receive actionable suggestions for remediation.

Core Features & Use Cases

Automated Failure Analysis: Parses logs and task outputs to diagnose errors.
Root Cause Identification: Categorizes failures into types like infrastructure, agent bugs, or task difficulty.
Suggested Fixes: Provides specific commands or code changes to resolve the issue.
Use Case: When a benchmark run fails, use this Skill to get an immediate diagnosis and a clear path to fixing the problem, rather than manually sifting through logs.

Quick Start

Use the triage-failure skill to investigate the most recent benchmark failure.

Frequently Asked Questions about triage-failure

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I find the root cause of a failed benchmark task?▼

To find the root cause of a failed benchmark task, you can automatically parse task logs and result JSON to categorize failure types and identify whether errors stem from infrastructure issues, agent bugs, or task difficulty.

What is the best way to analyze CodeScaleBench failure logs?▼

Analyzing CodeScaleBench failure logs is best handled by automated triage scripts that investigate task outputs and agent transcripts to diagnose errors and output specific remediation steps.

Can I get suggested fixes for benchmark failures from task logs?▼

Yes, you can get suggested fixes for benchmark failures by analyzing task logs and agent transcripts, which generates specific commands or code changes needed to resolve the identified issue.

Does automated failure triage work with multiple failed benchmark tasks?▼

Automated failure triage does work with multiple failed benchmark tasks, supporting batch investigation of various result JSON files and logs to diagnose errors across an entire run.

Why do I need to analyze agent transcripts for benchmark debugging?▼

You need to analyze agent transcripts during benchmark debugging to perform behavioral analysis of tasks, helping pinpoint exactly where agent bugs or task difficulty caused the execution to fail.