graphsignal-sdk

You can profile Python AI inference workloads by setting up the Graphsignal SDK to capture high-resolution profiling timelines, per-step tracing, and system metrics for faster diagnosis and optimization. It provides auto-instrumentation for stacks like vLLM and PyTorch.

Automatic tracing for vLLM and SGLang is set up by installing the Graphsignal SDK, configuring your API key via graphsignal.configure or the GRAPHSIGNAL_API_KEY environment variable, and running your application with the graphsignal-run CLI to enable instrumentation.

GPU profiling with CUDA CUPTI requires Python 3.9 or higher and optional CUDA CUPTI extras. You must install the SDK with graphsignal[cu12] or graphsignal[cu13] to enable GPU profiling and integration for your specific CUDA version.

Yes, you can manually annotate custom code paths during model inference using the SDK's manual tracing and profiling APIs. This allows you to measure and observe specific custom code sections beyond the automatic instrumentation.

The best way to observe PyTorch inference traces and system metrics is to use an auto-instrumentation SDK that captures high-resolution timelines and per-step data, applying it directly to your Python-based inference stack for automatic observability.