skills.rest
Shallow-W
Shallow-WCOMMUNITY
Explore all skills by this author →
GitHub

monitor-experiment

Monitor ML experiments across SSH, Vast.ai, and Modal deployments.

Updated Apr 21, 2026
Search Tags:#log collection#weights and biases#ml training#cost awareness#experiment monitoring#json results#remote ssh
One-click install
npx skills add https://github.com/Shallow-W/llm-wiki --skill monitor-experiment-shallow-w
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: monitor-experiment
Source: https://github.com/Shallow-W/llm-wiki/tree/main/.claude/skills/monitor-experiment
Command: npx skills add https://github.com/Shallow-W/llm-wiki --skill monitor-experiment-shallow-w

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

Monitoring long-running machine learning experiments is time-consuming and error-prone when you need to confirm progress, retrieve logs, and compile final metrics.

Core Features & Use Cases

  • Progress discovery across compute backends: Checks what is currently running via SSH screen sessions, Vast.ai instances, and Modal app lists/logs.
  • Automated output collection and artifact scanning: Pulls the latest screen output and inspects results directories for JSON artifacts to retrieve final structured results.
  • Optional Weights & Biases metrics pull: When enabled, fetches loss/learning-rate/eval metrics and produces a run summary with a direct dashboard link.
  • Interpretation and notification support: Compares results to baselines, flags anomalies (e.g., NaNs/divergence), and optionally sends a Feishu notification if configured.

Quick Start

Use monitor-experiment with your server alias or screen name to check whether your run is finished and to retrieve its latest logs and metrics.

Frequently Asked Questions about monitor-experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor remote SSH screen sessions for ML training progress?▼

The Skill checks running compute backends via SSH screen sessions, Vast.ai instances, and Modal app lists to pull the latest screen output and retrieve final structured results for ML training monitoring.

How do I extract JSON result artifacts from a running ML experiment?▼

To extract JSON result artifacts, this Skill automates output collection by inspecting your results directories to locate and read JSON artifacts, retrieving final structured metrics for your experiment.

Can I pull Weights and Biases metrics and send a Feishu notification for my experiment?▼

Yes, you can optionally enable Weights & Biases metric pulls to fetch loss and eval metrics with dashboard links, and configure Feishu notifications to receive run summaries and anomaly alerts.

Does this support Vast.ai instances and Modal deployments for experiment monitoring?▼

Yes, this Skill supports monitoring Vast.ai instances and Modal deployments by checking app lists and logs to discover progress across these specific compute backends.

How do I compare ML experiment metrics against a baseline and track cost awareness?▼

This Skill compares retrieved metrics against a baseline, flags anomalies like NaNs or divergence, reports run status deltas, and includes cost awareness for your ML training experiments.

Related Skills

Explore More Data & Analytics Skills→
Mang30Mang30

monitor-experiment

Monitor running experiments and collect outputs, JSON artifacts, and W&B metrics.

Community
Intermediate
raja21068raja21068
2

monitor-experiment

Monitor running experiments and summarize JSON metrics with baseline deltas.

Community
Advanced
Mang30Mang30

training-check

Detect training quality failures in Weights & Biases runs.

Community
Advanced
ovachieverovachiever
52

weights-and-biases

Automate ML experiment tracking and model registry management via the wandb API.

Community
Advanced