validate-tasks

Name: validate-tasks
Availability: InStock
Author: Sourcegraph

Validate benchmark task definitions for truncated instructions and missing test scripts.

32|4|Updated Feb 1, 2026

Search Tags:#code quality #testing #task management #validation #benchmark #pre-flight check

Ready to run

npx skills add https://github.com/sourcegraph/CodeScaleBench --skill validate-tasks-sourcegraph

💬 Prompt for ChatGPT / Claude Chat▼

Please help me install this Agent Skill.
Skill: validate-tasks
Source: https://github.com/sourcegraph/CodeScaleBench/tree/main/skills/validate-tasks
Command: npx skills add https://github.com/sourcegraph/CodeScaleBench --skill validate-tasks-sourcegraph

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill prevents wasted time and resources by identifying critical issues in benchmark task definitions before initiating lengthy execution runs.

Core Features & Use Cases

Pre-flight Validation: Checks for truncated instructions, missing test scripts, and configuration mismatches.
Error Detection: Identifies template placeholders, non-executable scripts, and incorrect path references.
Use Case: Before running a large suite of code generation tasks, use this Skill to automatically verify that all task instructions are complete, all necessary test scripts are present and executable, and that task configurations align with expected parameters, preventing potential failures mid-run.

Quick Start

Run pre-flight checks on all benchmark tasks using the command python3 scripts/validate_tasks_preflight.py --all.

Frequently Asked Questions about validate-tasks

High-intent search queries and answers about installing and using this skill.

FAQPage Schema

How do I validate benchmark task definitions before running execution?▼

To validate benchmark task definitions, run a pre-flight check using Python 3 to identify truncated instructions, missing test scripts, and configuration mismatches. This prevents execution failures by verifying task integrity before initiating lengthy runs.

What is pre-flight validation for code generation tasks?▼

Pre-flight validation for code generation tasks is the process of checking benchmark definitions for truncated instructions, non-executable scripts, and template placeholders. It ensures task configurations align with expected parameters to prevent mid-run failures.

How do I check for missing test scripts and configuration mismatches in task definitions?▼

You can check for missing test scripts and configuration mismatches by running an automated validation script against the repository structure. This error detection process identifies incorrect path references and verifies that all necessary scripts are executable.

Do I need Python 3 to run pre-flight checks on benchmark tasks?▼

Yes, you need Python 3 installed to execute the validation script for pre-flight checks. You also require access to the CodeScaleBench repository structure to perform the pre-run analysis on task definitions and selection registries.

Why does my benchmark task execution fail mid-run?▼

Benchmark task execution fails mid-run due to critical issues like template placeholders, non-executable test scripts, or truncated instructions. Running a pre-flight validation check detects these configuration mismatches beforehand to prevent wasted resources.

What is the best way to prevent execution failures in benchmark tasks?▼

The best way to prevent execution failures is to perform automated pre-flight validation on benchmark tasks. This verifies that all task instructions are complete, necessary test scripts are present and executable, and configurations match expected parameters.