Bilingual

<antirez>
7 hours ago
- The author uses LLMs routinely for coding but notes they are still far behind human intelligence.
- The example involves fixing a bug in Redis Vector Sets: checking reciprocal links in HNSW graphs to avoid use-after-free errors.
- A naive O(N^2) reciprocal check doubled loading time; Gemini suggested binary search on sorted pointer arrays.
- The author proposed a more creative XOR accumulator with a hashed key using a random seed to detect non-reciprocal links efficiently.
- Gemini helped analyze the XOR approach but couldn't generate the original idea, highlighting human creativity's edge over LLMs.
- The author concludes that while LLMs are useful as a 'smart duck' for discussion, humans still excel at out-of-the-box thinking.
<antirez>
7 hours ago
- Linked lists are educational, teaching fundamental concepts like pointers, O(1) and O(N) complexity.
- Linked lists are augmentable: can be made doubly linked, skip lists, unrolled lists, or embedded (e.g., in Linux kernel).
- Linked lists are composable: can split or glue two lists in O(1), useful in Redis threaded operations.
- Linked lists are useful: used in Redis and Linux kernel, and resemble natural processes like adding items in order.
- Linked lists are simple to implement from memory with low risk of errors.
- Linked lists are conceptual, with metaphors like self-referencing nodes (infinite loops) and NULL pointers (loneliness).
<antirez>
7 hours ago
- Redis Streams fill the gap between lists, sorted sets, and Pub/Sub for modeling message sequences or events.
- The concept was inspired by Kafka consumer groups and a user's idea of a transaction log style data type.
- Implementation uses radix trees and listpacks for space efficiency and indexed access by ID or time.
- Key commands include XADD (add entry), XRANGE (range queries), and XREAD (blocking read from multiple streams).
- Capped streams are supported via MAXLEN with optional approximate trimming using '~'.
- Consumer groups (in progress) allow multiple clients to share messages with acknowledgment and retry features.
- Memory usage is low; RDB loading is fast (e.g., 5 million entries in 0.3 seconds).
- Streams will be backported to Redis 4.0 within two months for production use.
<antirez>
7 hours ago
- Amplitude published an article on scaling analytics using the Set data type, explaining why they avoided Redis due to overhead.
- Building custom vertical solutions offers control and debugging benefits but requires ongoing development and can become problematic if original developers leave.
- Redis already uses memory-efficient encodings for small data types (e.g., intset for numeric sets) but lacks a similar representation for small string sets.
- Antirez proposes a new hash table design using linear arrays per bucket with a high load factor to improve memory efficiency and cache locality.
- The new design must handle incremental rehashing, scanning, and serialization while potentially replacing both small encodings and general hash tables.

Hasty Briefsbeta

data structures

<antirez>

<antirez>

<antirez>

<antirez>