Battle of the Mallocators
a year ago
- #performance
- #memory-allocation
- #RocksDB
- Use jemalloc or tcmalloc with RocksDB to avoid OOM, as glibc malloc leads to higher RSS.
- RocksDB's allocation pattern (malloc on block read, free on eviction) stresses allocators, causing RSS to grow larger than expected with glibc malloc.
- InnoDB avoids per-block malloc/free, reducing allocator stress and keeping RSS more predictable.
- Peak RSS for MyRocks is smallest with jemalloc (1.22x buffer pool), slightly larger with tcmalloc (1.31x), and much larger with glibc malloc (3.62x).
- Performance (QPS) is slightly better with jemalloc and tcmalloc compared to glibc malloc for both InnoDB and MyRocks.
- VSZ can be larger with jemalloc, but this may not necessarily indicate a problem (could be address space usage).
- Benchmark setup: MySQL 8.0.40 (InnoDB) and FB MySQL 8.0.32 (MyRocks) on a Hetzner server with 128G RAM.
- For MyRocks, glibc malloc caused OOM with a 50G buffer pool, while jemalloc/tcmalloc handled 100G without issues.