What problem does it solve?
Manually identifying performance bottlenecks in scripts, running processes, GPUs, memory, and system interconnects is time-consuming and prone to oversight, especially for complex compute workloads like machine learning training and LLM serving.
Core Features & Use Cases
- Multi-target profiling: Supports profiling Python scripts, running processes (by PID or service name), GPUs, memory, interconnects, and full frameworks like vLLM serving.
- Flexible measurement methods: Combines external industry-standard tools (cProfile, nvidia-smi, torch.profiler, nsys etc.) with custom code instrumentation for deep, targeted analysis.
- Structured reporting: Generates clear breakdowns of CPU overhead, memory usage, communication latency, and GPU compute utilization, with prioritized actionable recommendations.
- Use case: If your LLM serving pipeline has unexpectedly high latency, use this skill to profile NCCL communication overhead and GPU kernel utilization to pinpoint the root cause.
Quick Start
Use the system-profile skill to analyze the performance of your training script train.py and identify the top bottlenecks.