python-memory-profiling

Diagnose Python memory growth using tracemalloc, Memray, guppy3, psutil, and Filprofiler.

2|Updated Feb 26, 2024
One-click install
npx skills add https://github.com/rudolfolah/profiling-code --skill python-memory-profiling-rudolfolah
Or copy as Structured Prompt for Agent
Please help me install this Agent Skill.
Skill: python-memory-profiling
Source: https://github.com/rudolfolah/profiling-code/tree/main/.claude/skills/python-memory-profiling
Command: npx skills add https://github.com/rudolfolah/profiling-code --skill python-memory-profiling-rudolfolah

SYSTEM DOCUMENTATION & REQUIREMENTS

💡 This Skill requires guppy3, memray, psutil, filprofiler.

What problem does it solve? Investigating memory growth in Python programs is confusing because tracemalloc, Memray, Heapy, psutil, and Filprofiler each measure different things, and mixing their numbers leads to wrong conclusions. This Skill guides reproducible memory profiling of the repository's Python example program and explains which tool answers which question. ## Core Features & Use Cases - Tool selection guidance: A decision table maps each memory question (retained Python lines, allocation call stacks, object census, RSS footprint, peak-memory flame graph) to the right profiler and exact repository command. - Reproducible workflows: Pinned Python 3.11.5 setup, deterministic Memray output paths with --force, CI-safe Fil runs with --no-browser, and artifact handling rules for ignored files like memray-*.bin and fil-result/. - Interpretation and pitfalls: Explains why tracemalloc bytes, Memray allocations, Heapy object sizes, and RSS cannot be added together, and how native allocations, allocator arenas, and import-time noise distort results. - Use Case: A developer notices the example program's RSS climbing after parsing a Wikipedia page. They follow the safe investigation sequence: psutil for process impact, tracemalloc for line-level retained bytes, then Memray --native to check whether a C extension explains the gap. ## Quick Start Ask the assistant to profile the memory usage of python/program.py and explain which profiler to run first and why.

Frequently Asked Questions about python-memory-profiling

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I profile memory usage of a Python program?

Start with psutil to measure the process RSS footprint, then use tracemalloc for line-level Python allocations, Memray for allocation call stacks and flame graphs, guppy3 for an object census, and Filprofiler for peak-memory visualization. Each tool answers a different question.

What is the difference between tracemalloc and Memray?

tracemalloc traces Python-allocator blocks with source-line attribution but misses native allocations and is not RSS. Memray records allocation and deallocation events including native stacks with --native, producing flame graphs of memory live at peak.

Why does tracemalloc not match my process RSS?

tracemalloc only records Python traced allocations, while RSS includes C extension buffers, shared libraries, allocator arenas, memory-mapped files, and interpreter state. Use psutil for OS-visible footprint and Memray --native to find native allocation paths.

How do I capture import-time memory allocations with tracemalloc?

Start tracing before imports using the interpreter option python -X tracemalloc=25 run_tracemalloc.py or the PYTHONTRACEMALLOC=25 environment variable. The runner script imports program before tracemalloc.start(), so import-time allocations are otherwise missed.

When should I use Filprofiler instead of Memray?

Use Filprofiler when you want an interactive SVG visualization of the call stacks responsible for the high-water mark, especially with native extensions. It is an offline profiler with substantial overhead, so it is unsuitable for production monitoring or multiprocessing workloads.

Can I add tracemalloc, Memray, and psutil numbers together?

No. A tracemalloc size, a Memray allocation, a Heapy object size, and RSS measure different populations at different points in time. Use a process-level tool like psutil to establish impact, then an allocation or object tool to explain it.