Bilingual

PostgreSQL and the OOM Killer: Why You Must Use Strict Memory Overcommit
20 days ago
- Strict memory overcommit is used to protect PostgreSQL databases from catastrophic OOM kills by converting destructive failures into graceful errors.
- PostgreSQL's architecture means an OOM kill can corrupt shared memory, leading to connection drops, transaction aborts, and lengthy crash recovery.
- A kernel bug (introduced in Linux 6.5) caused phantom committed memory inflation, leading to unexpected ENOMEM errors under strict overcommit.
- The bug was fixed in Linux 6.8 by correcting a condition in move_vma() that reversed error handling for memory accounting.
- Ubicloud's heuristic for setting the commit limit is: 80% of physical memory (excluding hugepages) plus a fixed 2 GB buffer for sidecar processes.
- Monitoring memory usage and understanding workload characteristics is essential before enabling strict overcommit to avoid frequent ENOMEM errors.
FreeBSD ate my RAM
19 days ago
- Memory usage reporting is complex due to OS caching mechanisms like disk caching in RAM for performance, which can be freed when needed.
- Virtual Memory (VM) systems manage RAM pages in queues (active, inactive, laundry, wired, free), making it hard to define 'used' vs. 'free' memory accurately.
- Tools like fastfetch, btop, and htop use different heuristics to report memory usage, leading to discrepancies in displayed values.
- btop had bugs on FreeBSD, including integer overflow with 32-bit variables and incorrect cache reporting due to using a legacy sysctl parameter.
- Fixes were implemented for btop and htop to correctly account for ZFS ARC cache and filesystem buffers, improving memory monitoring accuracy.
- The author contributed patches to open-source projects, highlighting the importance of understanding OS internals for accurate system monitoring.
Rust Service Isn't Leaking – It Could Be the Allocator
18 days ago
- Memory usage remained high after processing events despite no memory leak in Rust code, due to allocator behavior.
- Glibc's ptmalloc allocator retains freed memory in arenas, causing high RSS as freed chunks may not be returned to the OS, especially with thread arenas and fragmentation.
- Switching to jemalloc resolved the issue because it uses size-class-segregated slabs, allowing independent memory reclamation via background threads and decay timers.
- Mimalloc did not work well for bursty workloads because cross-thread frees delayed reclamation, and RSS remained high without continuous allocation activity.
Dense Arena Interning: The Engine of Compiler Performance
10 days ago
- Compiler performance is improved by reducing repeated string and structure checks using Dense Arena Interning.
- Dense Arena Interning converts strings and structures into dense integers during lexing or type construction, enabling O(1) lookups and comparisons in later compiler phases.
- Arena allocators provide stable memory, allowing pointer-based comparisons and efficient memory management without reallocation.
- Dense IDs enable flat array symbol tables, improving cache locality and turning symbol resolution into O(1) array indexing.
- Interning canonicalizes complex types, allowing shallow pointer comparisons instead of deep structural checks, further speeding up type and symbol comparisons.
On Strings in Rust
10 hours ago
- Rust strings involve two main types: `str` (string slice) and `String` (owned string), both guaranteed to be UTF-8 encoded.
- `str` is a dynamically sized type (DST) behind a wide pointer, while `String` wraps a `Vec<u8>` and supports heap allocation and mutation.
- Deref coercion and dot operator semantics allow methods like `chars()` on `String` by automatically dereferencing to `str`.
- The `len()` method returns byte count, not character count, due to UTF-8 variable-length encoding.
- Operator overloading (e.g., `+` for concatenation) and trait implementations like `PartialEq` enable ergonomic string operations.
- In-place methods like `make_ascii_uppercase()` work on `&mut str` but cannot handle non-ASCII characters that change byte length.
- Other string types include `OsString`/`OsStr`, `CString`/`CStr`, and experimental `ByteString`/`Char` for specific use cases.

Hasty Briefsbeta

#memory management

PostgreSQL and the OOM Killer: Why You Must Use Strict Memory Overcommit

FreeBSD ate my RAM

Rust Service Isn't Leaking – It Could Be the Allocator

Dense Arena Interning: The Engine of Compiler Performance

On Strings in Rust