skills.rest
yo-steven
yo-stevenCOMMUNITY
Explore all skills by this author →
GitHub

nel-assistant

Generate NEL YAML evaluation configs interactively across execution backends.

Updated May 23, 2026
Search Tags:#tool calling#model evaluation#vllm#slurm#nel#yaml config#reasoning toggles
One-click install
npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill nel-assistant-yo-steven
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: nel-assistant
Source: https://github.com/yo-steven/skills-exploration-20260522/tree/main/skills/NeMo-Evaluator-Launcher/nel-assistant
Command: npx skills add https://github.com/yo-steven/skills-exploration-20260522 --skill nel-assistant-yo-steven

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

It solves the problem of creating correct, production-ready NeMo Evaluator Launcher (NEL) YAML evaluation configs without manually figuring out all required fields, model-specific settings, and deployment nuances.

Core Features & Use Cases

  • End-to-end config generation: Builds a base NEL config from scratch after gathering the required choices (execution mode, deployment backend, auto-export, model type, and benchmark set).
  • Model-card-driven configuration: Uses WebSearch to read the full model card and extract model-specific sampling parameters, context length, reasoning toggles, tool-calling requirements, and deployment constraints.
  • Interactive refinement & validation loop: Finds missing values, iteratively confirms task lists, supports multi-node advanced setups, and guides dry-run/test runs before full evaluation.
  • Operational guidance: Provides the exact nel run commands for dry-run and sample-limited testing, plus monitoring and log-inspection workflows for both local and SLURM execution.

Quick Start

Ask the assistant to create a new NEL evaluation config for your target model and benchmarks, then run a dry-run with a small sample size to confirm the configuration works.

Frequently Asked Questions about nel-assistant

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I generate a NeMo Evaluator Launcher YAML config for a new model evaluation?▼

To generate a NeMo Evaluator Launcher (NEL) YAML config, interactively specify the execution mode, deployment backend, model type, and benchmark set. The assistant bootstraps the base configuration and uses the model card to automatically extract sampling parameters, context length, and tool-calling requirements.

How does model-card-driven parameterization work for NEL evaluation configs?▼

Model-card-driven parameterization works by using WebSearch to read the target model's card and extract specific settings. It automatically populates the NEL YAML config with exact sampling parameters, context length constraints, reasoning toggles, and deployment limitations required for accurate evaluation.

Can I run NEL evaluations on multi-node SLURM clusters with vLLM?▼

Yes, you can run NEL evaluations on multi-node SLURM clusters with vLLM. The interactive setup supports advanced multi-node configurations, provides exact `nel run` commands for SLURM execution, and guides you through monitoring and log-inspection workflows for both local and cluster environments.

What is the best way to validate an NEL YAML config before running a full evaluation?▼

The best way to validate an NEL YAML config is to execute a dry-run and a sample-limited test. The assistant checks the NEL installation, ensures all placeholders are filled with correct reasoning and tool-calling conventions, and generates the exact commands for both dry-run and limited-sample execution.

Do I need to manually configure reasoning toggles and tool-calling parameters in NEL?▼

You do not need to manually configure reasoning toggles and tool-calling parameters in NEL. The assistant extracts these requirements directly from the model card and applies the correct logging parameter conventions to the generated YAML configuration automatically.

Related Skills

Explore More Software Engineering Skills→
yo-stevenyo-steven

evaluation

Generate NEL configurations and execute LLM accuracy benchmarks on local or SLURM backends.

Community
Advanced
sahunosahuno

snakemake

Automate creation and debugging of Snakemake workflows for SLURM-based HPC.

Community
Advanced
ovachieverovachiever
52

serving-llms-vllm

Deploy OpenAI-compatible LLM APIs with vLLM for high-throughput inference.

Community
Advanced
NVIDIANVIDIA
3.4k

compare-results

Compare baseline and candidate model evaluation results to determine quantization impact.

Official
Advanced