What problem does it solve?
Identifying performance bottlenecks in training scripts, serving systems, and multi-GPU workloads requires juggling many profiling tools and manual instrumentation, which is slow and error-prone.
Core Features & Use Cases
- Multi-target profiling: Profile Python scripts, running processes (PID), specific functions, or entire frameworks like vLLM serving end-to-end.
- External tools plus code instrumentation: Combines cProfile, py-spy, torch.profiler, nsys, nvidia-smi, and NCCL debugging with custom instrumentation for CPU-GPU transfer bandwidth, memory redundancy, and collective latency.
- Structured reports with changelog: Produces bottleneck tables ranked by impact plus a mandatory changelog of every instrumentation edit so changes can be reviewed and reverted.
- Use Case: Ask to profile a slow training script and receive hotspot analysis, GPU utilization metrics, communication-to-computation ratios, and ranked optimization recommendations saved to ./profile_output/.
Quick Start
Profile train.py and identify the GPU and memory bottlenecks with actionable recommendations.