Automate operator-only evaluation loops over local runtime artifacts.
npx skills add https://github.com/JetXu-LLM/DocMason --skill operator-eval
Please help me install this Agent Skill. Skill: operator-eval Source: https://github.com/JetXu-LLM/DocMason/tree/main/skills/operator/operator-eval Command: npx skills add https://github.com/JetXu-LLM/DocMason --skill operator-eval
The hidden operator-only evaluation loop over local runtime/eval artifacts can be difficult to run safely and reproducibly within standard user workflows.
Ask the agent to run the operator-eval workflow for the current evaluation request and return results to the main agent.
High-intent search queries and answers about installing and using this skill.
Automate local evaluation loops by reading, validating, and executing a single operator request from the local surface to safely run regression reviews while keeping artifacts isolated from normal user flows.
An operator-only evaluation workflow is a hidden, isolated loop that processes local runtime artifacts to perform replayable evaluation runs, candidate promotion, and baseline freezing without exposing these operations to normal users.
Freeze evaluation baselines by executing operator requests over local runtime artifacts, which preserves provenance and separates runtime activity logs from standard user data during the process.
Yes, you can run replayable evaluation runs by keeping all evaluation artifacts under runtime/eval directories, ensuring isolation from normal user data and workflows.
Operator-only eval workflows require local file access, shell execution capabilities, and structured JSON output inspection, keeping all actions strictly out of standard user flows and confined to local runtime environments.
Yes, operator-eval inspects structured JSON output from local runtime artifacts to automate candidate promotion, returning operator-facing results and artifacts directly to the main agent.
JetXu-LLMRetrieve ranked evidence bundles from the DocMason knowledge base.
NVIDIACompare baseline and candidate model evaluation results to determine quantization impact.
JetXu-LLMReview runtime query-session and retrieval-trace logs to surface actionable insights.
chiptoe-svgInstall Telegram admin slash commands for NanoClaw instances.
