python-performance-optimization

Use cProfile to profile Python code by wrapping execution with profiler.enable() and profiler.disable(), then print results with pstats sorted by cumulative time. You can also run python -m cProfile -o output.prof script.py from the command line.

Use memory_profiler with the @profile decorator to track line-by-line memory allocation in a function. Run the script with python -m memory_profiler script.py to see where memory grows and identify leaks.

cProfile measures time at the function level, showing call counts and cumulative time per function. line_profiler measures execution time for each individual line, giving finer granularity when you already know which function is slow.

Yes, py-spy attaches to a running Python process by PID without modifying code. Use py-spy top for live monitoring or py-spy record -o profile.svg --pid <pid> to generate a flamegraph.

List comprehensions avoid repeated append method lookups and are optimized in the interpreter's C implementation. Benchmarks with timeit typically show comprehensions running measurably faster than equivalent loop-and-append code.

Use generators when processing large datasets where you only need one item at a time. Generators yield values lazily and use constant memory regardless of dataset size, while lists store all elements in memory.