redis streams

<antirez>

2 months ago

Redis Streams, introduced in Redis 5, are often mistakenly seen only as a Kafka-like messaging solution, but they are a general-purpose data structure.
Streams overcome limitations of CSV logs by providing efficient range queries, auto-generated time-based IDs, compact memory usage, and the ability to remove old entries.
Internally, Streams use delta-compressed macro nodes linked by a radix tree, enabling fast seek, range queries, and capped streams.
Entry IDs are monotonically incrementing and time-based, allowing easy range queries by time (e.g., XRANGE).
Streams can replace traditional patterns like Sorted Set + Hash for modeling objects (e.g., tennis matches), achieving over 13x memory savings (16.8 MB vs 220 MB for 1 million entries).
This memory efficiency makes it viable to store billions of items in reasonable RAM, enabling new use cases like time series and small object tracking.
Streams are not limited to streaming; they can be used for any scenario requiring ordered, append-only data with unique IDs and low memory overhead.