Page Table Memory Consumption
4 days ago
- Linus Torvalds argued that tree-structured page tables allow efficient TLB prefetching by placing neighboring entries adjacent in memory, unlike hash tables which scatter them.
- While early Linux adopted multi-level page tables prioritizing latency, later cases (e.g., Shakeel Butt 2020) showed memory efficiency becomes critical when many processes map the same memory, each requiring their own page tables.
- Historical examples (Andrea Arcangeli 2002, Khalid Aziz 2022, Qi Zheng 2021) demonstrate page table memory can grow enormously with many processes or improper memory freeing, leading to OOM crashes even with ample RAM.
- Database workloads (Percona Postgres, ClickHouse) show page tables can balloon to tens of gigabytes with many connections sharing large buffers, mitigated by using huge pages.
- NUMA machines introduce a trade-off between keeping page tables local for low latency versus replicating them across nodes at the cost of synchronization overhead.