skills.rest
GenerativeAgents
GenerativeAgentsOFFICIAL
Explore all skills by this author →
GitHub

wandb-primary

Manage Weights & Biases projects by analyzing runs, traces, and reports.

4|2|Updated Nov 12, 2024
Search Tags:#monitoring#report generation#wandb#run analysis#ml project management#trace inspection#run relaunch
One-click install
npx skills add https://github.com/GenerativeAgents/training-llm-application-development --skill wandb-primary-generativeagents
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: wandb-primary
Source: https://github.com/GenerativeAgents/training-llm-application-development/tree/main/day3/.agents/skills/wandb-primary
Command: npx skills add https://github.com/GenerativeAgents/training-llm-application-development --skill wandb-primary-generativeagents

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires wandb, weave, pandas, numpy, and includes scripts (resource) and references (resource) components.

What problem does it solve?

This Skill streamlines the management of Weights & Biases (W&B) projects, providing a comprehensive suite of functionalities for analyzing, querying, and launching W&B projects.

Core Features & Use Cases

  • Project Overview: Gain insights into the status and performance of all training runs in a project.
  • Run Analysis: Perform detailed analysis on individual runs, including counting runs, summarizing runs, and inspecting single runs.
  • Trace Inspection: View and analyze the traces of GenAI calls within a project, providing insights into the execution flow and resource usage.
  • Report Creation: Generate comprehensive reports on the project's performance using W&B Reports API.
  • Monitor Setup: Set up monitors to track the health and performance of training runs.
  • Run Relaunch: Relaunch existing runs with updated configurations or hyperparameters.
  • Code Modification and Launch: Modify code and launch new training jobs with minimal overhead.
  • Queue Management: Create, list, and inspect launch queues for efficient job submission.
  • Artifact Management: Track and manage the artifacts produced by runs, including datasets, models, and checkpoints.
  • Cost Tracking: Monitor the costs associated with LLM inference and training runs.
  • Evaluation: Analyze and summarize evaluation results for LLM applications.
  • Use Case: Suppose you are working on a complex ML project with multiple training runs. Use this Skill to quickly access the status of runs, compare run metrics, and generate detailed reports to understand your project's performance.

Quick Start

Run the following command to get an overview of the training runs in your project:

python skills/wandb-primary/scripts/summary_project.py

Frequently Asked Questions about wandb-primary

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I get an overview of all training runs in a W&B project?▼

To get a W&B project overview, you can run a summary script to query the W&B API and gain insights into the status and performance of all training runs within that project.

Can I generate W&B reports automatically for my ML experiments?▼

Yes, you can generate W&B reports automatically. This functionality uses the W&B Reports API to create comprehensive documents summarizing your ML project performance and run metrics.

How do I inspect traces for GenAI calls in my W&B project?▼

You can inspect GenAI traces in your W&B project by using trace inspection features, which allow you to view and analyze the execution flow and resource usage of GenAI calls.

What is the best way to relaunch a W&B run with updated hyperparameters?▼

The best way to relaunch a W&B run with updated hyperparameters is to use the run relaunch feature, which restarts existing runs with new configurations while managing launch queues efficiently.

Do I need specific Python libraries to manage W&B projects programmatically?▼

Yes, you need the wandb and weave libraries installed, alongside pandas and numpy, to interact with W&B APIs for run analysis, artifact management, and monitoring tasks.

Can I track LLM inference costs within my W&B training runs?▼

Yes, you can track LLM inference costs. The cost tracking functionality monitors the financial expenses associated with both LLM inference and your standard ML training runs.

Related Skills

Explore More Software Engineering Skills→
wandbwandb
65

wandb-primary

Query, analyze, and launch W&B projects with run histories and artifacts.

Official
Advanced
carterwayneskhizeinecarterwayneskhizeine
19

weights-and-biases

Log metrics, configs, and artifacts to W&B for experiment tracking.

Community
Intermediate
jacardljacardl

weights-and-biases

Track ML experiments with automatic logging and real-time dashboards.

Community
Advanced
gztcodegztcode

weights-and-biases

Automate ML experiment tracking and collaboration with Weights & Biases.

Community
Advanced