monitor-experiment

Monitor running ML experiments and collect logs, metrics, and JSON results.

Updated May 25, 2026
One-click install
npx skills add https://github.com/duypham2801/ThS_LLM --skill monitor-experiment-duypham2801
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: monitor-experiment
Source: https://github.com/duypham2801/ThS_LLM/tree/main/.claude/skills/monitor-experiment
Command: npx skills add https://github.com/duypham2801/ThS_LLM --skill monitor-experiment-duypham2801

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill includes scripts (resource) components.

What problem does it solve?

It helps you identify whether training/evaluation jobs are still running and quickly gather their latest outputs and metrics so you can decide what to do next without guesswork.

Core Features & Use Cases

  • Live job visibility: Checks running sessions (screen), verifies cloud/Modal app status, and determines whether work is complete or stalled.
  • Result retrieval: Hard-copies recent log output from each session and inspects result directories for JSON artifacts to fetch the latest metrics.
  • Metric-driven status: Optionally pulls Weights & Biases curves and run summaries to confirm convergence, detect divergence/OOM risk, and link to the dashboard for deeper review.
  • Actionable reporting: Produces a comparison-style results summary and an interpretation with suggested next steps.

Quick Start

Tell the skill which server/screen name to monitor (for example: "monitor-experiment on screen session <name>") so it can check running jobs and return the latest logs and available JSON/W&B metrics.

Frequently Asked Questions about monitor-experiment

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I monitor machine learning training jobs running in remote SSH screen sessions?

To monitor machine learning training jobs in remote SSH screen sessions, you can locate running processes, tail recent log outputs, and read JSON result artifacts to verify training progress and retrieve metrics.

Can I pull Weights and Biases run summaries and convergence curves from remote training instances?

Yes, you can pull Weights and Biases history and run summaries to confirm convergence, detect divergence or OOM risk, and link to the dashboard for deeper review of remote training instances.

How do I check if a training or evaluation job on Vast.ai or Modal is still running or stalled?

You can check cloud or Modal app status by verifying running sessions and determining whether work is complete or stalled, helping you decide next steps without guesswork.

What is the best way to aggregate experiment results and logs from multiple remote training environments?

The best way to aggregate experiment results is by hard-copying recent log output from each session and inspecting result directories for JSON artifacts to fetch the latest metrics across environments.

Do I need to manually inspect logs to detect divergence or OOM risk during model training?

No, you do not need to manually inspect logs to detect divergence or OOM risk; the skill optionally pulls W&B curves and run summaries to interpret failures and convergence automatically.

Why does my machine learning experiment monitoring not return actionable next steps for stalled jobs?

If your monitoring lacks actionable next steps, ensure it produces a comparison-style results summary and an interpretation with suggested next steps based on the gathered logs and metrics.