skill-eval-tasks-v1

What problem does it solve?

This Skill automates the conversion of reviewed skill episodes into eval task rows, ensuring leakage-safe data handling for evaluation purposes.

Core Features & Use Cases

• Evaluation Task Generation: Converts skill-episodes-v1 rows into deterministic eval task rows for Dtrain, Dsel, and Dtest splits.
• Leakage Control: Ensures that the evaluation tasks are leakage-safe, protecting sensitive information.
• Use Case: Ideal for generating evaluation tasks for machine learning models, especially when working with proprietary or sensitive data.

Quick Start

Run the skill-eval-tasks-v1 script with the source path to the skill episodes and the output path for the eval tasks.