skills.rest
JayFarei
JayFareiCOMMUNITY
Explore all skills by this author →
Website𝕏@jayfareiGitHub

skill-eval-tasks-v1

Convert reviewed skill episodes into leakage-safe eval task rows for Dtrain, Dsel, and Dtest splits.

92|6|Updated Mar 27, 2026
Search Tags:#machine learning#data conversion#skill evaluation#eval task generation#leakage control
One-click install
npx skills add https://github.com/JayFarei/opentraces --skill skill-eval-tasks-v1
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: skill-eval-tasks-v1
Source: https://github.com/JayFarei/opentraces/tree/main/src/opentraces/workflow_templates/skill-eval-tasks-v1
Command: npx skills add https://github.com/JayFarei/opentraces --skill skill-eval-tasks-v1

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

This Skill automates the conversion of reviewed skill episodes into eval task rows, ensuring leakage-safe data handling for evaluation purposes.

Core Features & Use Cases

  • Evaluation Task Generation: Converts skill-episodes-v1 rows into deterministic eval task rows for Dtrain, Dsel, and Dtest splits.
  • Leakage Control: Ensures that the evaluation tasks are leakage-safe, protecting sensitive information.
  • Use Case: Ideal for generating evaluation tasks for machine learning models, especially when working with proprietary or sensitive data.

Quick Start

Run the skill-eval-tasks-v1 script with the source path to the skill episodes and the output path for the eval tasks.

Frequently Asked Questions about skill-eval-tasks-v1

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I convert reviewed skill episodes into eval task rows?▼

The script converts reviewed skill episodes into eval task rows by processing JSONL input files and outputting structured rows for Dtrain, Dsel, and Dtest splits.

What is leakage-safe data handling in machine learning evaluation?▼

Leakage-safe data handling in machine learning evaluation ensures that sensitive information from training data does not improperly appear in test sets. This Skill enforces leakage control when generating eval task rows to protect proprietary data integrity.

Can I use JSONL files to generate machine learning data splits?▼

Yes, this script handles JSONL input files containing reviewed skill episodes and converts them into deterministic eval task rows for Dtrain, Dsel, and Dtest splits.

How do I generate Dtrain, Dsel, and Dtest splits from skill episodes?▼

You generate Dtrain, Dsel, and Dtest splits by running the conversion script against your JSONL source files, which automatically handles the different data splits to output structured eval task rows.

Does the eval task generation script have any external dependencies?▼

The eval task generation script has zero external dependencies, operating independently through its internal scripts component to process JSONL inputs into structured eval task rows.

Related Skills

Explore More Software Engineering Skills→
infantesromeroadrianinfantesromeroadrian

skill-effectiveness

Aggregate skill usage telemetry from JSONL logs and flag low success rates.

Community
Advanced
m0atm0at
1

scikit-learn

Guide scikit-learn workflows for classification, regression, clustering, and pipelines.

Community
Advanced
JayFareiJayFarei
92

skill-update-report-v1

Generate report rows for SkillOpt updates as JSONL files.

Community
Intermediate
Imad-OuteImad-Oute

pytdc

Access, process, and evaluate pharmaceutical datasets for machine learning experiments.

Community
Intermediate