8 hours ago
- Redis Streams, introduced in Redis 5, are often mistakenly seen only as a Kafka-like messaging solution, but they are a general-purpose data structure.
- Streams overcome limitations of CSV logs by providing efficient range queries, auto-generated time-based IDs, compact memory usage, and the ability to remove old entries.
- Internally, Streams use delta-compressed macro nodes linked by a radix tree, enabling fast seek, range queries, and capped streams.
- Entry IDs are monotonically incrementing and time-based, allowing easy range queries by time (e.g., XRANGE).
- Streams can replace traditional patterns like Sorted Set + Hash for modeling objects (e.g., tennis matches), achieving over 13x memory savings (16.8 MB vs 220 MB for 1 million entries).
- This memory efficiency makes it viable to store billions of items in reasonable RAM, enabling new use cases like time series and small object tracking.
- Streams are not limited to streaming; they can be used for any scenario requiring ordered, append-only data with unique IDs and low memory overhead.