skills.rest
open-thoughts
open-thoughtsOFFICIAL
Explore all skills by this author →
WebsiteGitHub

eval-standard-launch

Automate Delphi 6279 RL-scaling-laws evaluation workflows on Leonardo with sbatch submission.

276|37|Updated Dec 5, 2025
Search Tags:#workflow#evaluation#scoring#sbatch#delphi#rl-scaling-laws#offline-compute
One-click install
npx skills add https://github.com/open-thoughts/OpenThoughts-Agent --skill eval-standard-launch
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: eval-standard-launch
Source: https://github.com/open-thoughts/OpenThoughts-Agent/tree/main/.claude/skills/eval-standard-launch
Command: npx skills add https://github.com/open-thoughts/OpenThoughts-Agent --skill eval-standard-launch

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Orchestrates the end-to-end Delphi #6279 RL-scaling-laws evaluation workflow on Leonardo, including pre-download, job submission, and result-tracking.

Core Features & Use Cases

  • End-to-end orchestration of Delphi scaling-law evaluations across SFT/RL/base checkpoints on a single Leonardo run.
  • Handles offline pre-download, RUN_NAME / STAGE conventions, chat-template overrides, and centralized results consolidation into SCORES.md.
  • Supports reproducible workflows for data scientists evaluating model scaling behaviors.

Quick Start

Run the sbatch script on Leonardo with the RUN_NAME and STAGE specified to start the Delphi 6279 evaluation, using the pre-downloaded data.

Frequently Asked Questions about eval-standard-launch

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I automate RL-scaling-laws evaluation workflows on Leonardo?▼

Automating RL-scaling-laws evaluation on Leonardo requires orchestrating pre-download, sbatch job submission with RUN_NAME and STAGE conventions, and result tracking into SCORES.md for SFT/RL/base checkpoints.

What is the best way to run Delphi 6279 evaluations across a 54-run main grid?▼

Running Delphi 6279 evaluations across a 54-run main grid requires applying chat-template overrides and offline cluster constraints to SFT/RL/base checkpoints, then consolidating results into SCORES.md.

How does offline pre-download work for Delphi scaling-law evaluations?▼

Offline pre-download for Delphi scaling-law evaluations fetches necessary data and checkpoints before sbatch submission, ensuring reproducible workflows under offline compute constraints on the Leonardo cluster.

Can I apply chat-template overrides when submitting sbatch jobs on Leonardo?▼

Yes, you can apply chat-template overrides during sbatch submission on Leonardo. The workflow supports overriding chat templates for SFT/RL/base checkpoints across the 54-run main grid.

How do I track and consolidate evaluation results from sbatch jobs?▼

Tracking and consolidating evaluation results from sbatch jobs is done by centralizing the outputs into SCORES.md. This approach maintains reproducible scaling-law evaluation records for data scientists.

Do I need specific RUN_NAME and STAGE conventions for Leonardo cluster submissions?▼

Yes, specific RUN_NAME and STAGE conventions are required for Leonardo cluster submissions. These conventions structure the sbatch submission process and ensure proper tracking of the 54-run main grid checkpoints.

Related Skills

Explore More Data & Analytics Skills→
open-thoughtsopen-thoughts
276

sft-launch-leonardo

Launch, monitor, and clean up LLaMA-Factory SFT jobs on the Leonardo cluster.

Official
Advanced
jonajona
2

ai-scaling-laws-amodei

Forecast AI capability trajectories using scaling laws and the 7-month doubling heuristic.

Community
Basic
1479089714790897
7

slurm

Submit, monitor, and cancel SLURM jobs on remote Linux clusters.

Community
Intermediate
abrahamgreysonabrahamgreyson

gsd:workflow:checkpoint

Create and manage workflow checkpoints with approval and progress documentation.

Community
Advanced