veomni-profile

What problem does it solve?

This Skill facilitates comprehensive profiling and performance optimization of deep learning training processes, enabling users to identify bottlenecks and improve efficiency.

Core Features & Use Cases

• Profile analysis: Parse Chrome traces and memory snapshots to evaluate kernel execution, memory usage, and communication overhead.
• Performance optimization: Provide insights into bottlenecks such as slow kernels, inefficient communication, and memory issues to guide improvements during model training.
• Use Case: During training, generate profiling data, analyze the results, and optimize script configurations to maximize throughput and resource utilization, applicable to large-scale model training workflows.

Quick Start

Configure the profiling parameters in your training setup, run the training process with profiling enabled, then analyze the output Chrome traces and memory snapshots using the provided scripts to identify and address performance issues.