memory optimization

Bilingual

Tokio Gives Progress, Not Ordering: Scheduling 1M Tasks
2 months ago
- Tokio's multi-threaded scheduler uses local and global queues without guaranteeing tasks are polled in creation order.
- Spawning tasks early does not ensure early polling or completion; tasks from different events compete for worker queues and can get delayed.
- Without bounds on task creation, large numbers of live tasks increase peak memory because earlier tasks and their parents remain alive longer.
- Using a Semaphore to limit concurrent events provides event-level fairness, reducing memory spikes while maintaining throughput.
- Application code must enforce its own fairness boundaries; Tokio treats all tasks equally and does not know about application-level units like events.
Span-First C#: Designing Around Span<T>
2 months ago
- Span<T> is a ref struct that provides a type-safe, bounds-checked view over contiguous memory without owning or copying the memory.
- It can wrap arrays, strings, stackalloc blocks, and unmanaged memory via MemoryMarshal.
- Span<T> is stack-only, preventing heap allocation and ensuring safety from GC relocation.
- Four related types (Span<T>, ReadOnlySpan<T>, Memory<T>, ReadOnlyMemory<T>) cover mutable/read-only and stack/heap axes.
- Memory<T> is the heap-storable counterpart for async scenarios, while Span<T> is for synchronous use.
- ref struct restriction by Span<T> enforces lifetime management at compile time, disallowing field storage, boxing, async await, and closures.
- C# 13 allows ref struct as generic type parameter constraint, but does not lift field, closure, or async restrictions.
- Slicing produces a new Span<T> over a subrange without copying, using index and range syntax.
- stackalloc with Span<T> provides stack-allocated buffers with no GC pressure, but is limited to the current stack frame and usually under 1-2 KB.
- Span-first parsing avoids allocations by walking indices over the original buffer and only materializing strings or numbers at final consumption.
- SearchValues<T> (NET 8+) optimizes repeated IndexOfAny calls for tokenizers and delimiter scanning.
- Span-first API design uses ReadOnlySpan<T> as the canonical overload, with array/string overloads as thin convenience wrappers.
- MemoryMarshal allows reinterpreting spans across type boundaries, e.g., casting byte buffers to struct spans without copying.
- Common failure mode: using Span<T> with async methods; fix by using Memory<T> at async layer and Span<T> at synchronous leaf.
- Span-first parsing shows significant performance and memory benefits over string-based parsing, with zero allocations and lower latency.
- Safety caveats include compiler catching direct escapes of stackalloc spans but not all indirect ones, and slice bounds checks being cheap but not free.
- Dictionary keyed by strings cannot use ReadOnlySpan<char> directly without ToString(), but NET 9 offers GetAlternateLookup for span-keyed lookup.
- Span<T> lacks IEnumerable<T> implementation, so LINQ methods are unavailable and must be hand-rolled.
Why Huge Pages matter for Postgres
2 months ago
- Huge pages are large memory units (2MB or 1GB) that allow one page-table entry to cover much more memory compared to standard 4KB pages, reducing the number of translations the CPU's TLB must cache.
- Postgres is sensitive to page size because its shared_buffers cache is mapped by each backend's page tables; with 4KB pages, large caches can consume excessive RAM for page tables and cause TLB misses, slowing reads.
- Using huge pages (e.g., 2MB) dramatically reduces page-table memory usage per connection—from ~31MB to ~0.5MB per connection in tests—and improves TLB coverage, keeping the hot working set in cache and boosting throughput.
- ClickHouse Managed Postgres ensures huge pages by reserving memory early (25% of RAM), configuring huge_pages = 'on' to prevent silent fallback, and sizing shared_buffers to fit the reserved pool via measurement rather than assumption.
- Tests on an EC2 instance showed that with 200 connections, 4KB pages led to 6.12GB of page tables versus only 111MB with 2MB huge pages, and throughput increased by 12% with huge pages due to reduced translation overhead.

memory optimization

Tokio Gives Progress, Not Ordering: Scheduling 1M Tasks

Span-First C#: Designing Around Span<T>

Why Huge Pages matter for Postgres