First Steps into Memory Management in Python
2 days ago
- Python uses two memory allocation methods: pymalloc for objects ≤512 bytes and malloc (PyMem_RawMalloc) for larger objects.
- Pymalloc requests arenas from the OS; each arena has 64 pools, each with fixed-size blocks; objects occupy free blocks in pools.
- Memory becomes real from the OS perspective only when its page is first written; pymalloc only asks for a new arena when none has space.
- Arenas are only returned to the OS when all their pools are empty; a single live object keeps its entire arena from being released.
- Larger objects (>512 bytes) use malloc; freeing typically keeps memory for reuse, and only very large blocks go back to the OS.
- Memory deallocation uses reference counting: each object's count decreases when references are removed; at zero, the object is freed immediately.
- The garbage collector handles circular references that reference counting cannot; it periodically checks container objects using generational collection (gen0, gen1, gen2).
- Practical insight: you can control object lifetime by managing variable scope, using del, or passing objects directly to functions.
- Studying Python internals is deep and rewarding; the author found it enjoyable and learned a lot about memory management.