The Evolution of Caching Libraries in Go and Ristretto's zero hit rate mystery
10 months ago
- #Performance
- #Caching
- #Go
- Caching libraries in Go are divided into on-heap and off-heap types, each with distinct advantages and limitations.
- On-heap caches, while introducing GC overhead, solve many drawbacks of off-heap caches, such as poor eviction policies and lack of features.
- Ristretto emerged as a leading on-heap cache in Go, inspired by Java's Caffeine, but has several disadvantages including questionable design choices and bugs.
- Theine, developed in 2023, introduced adaptive W-TinyLFU and became one of the best caching libraries in Go, though it has some scalability and memory overhead issues.
- Otter v1 was developed to address Ristretto's hit rate degradation, leveraging xsync and S3-FIFO, but had its own set of limitations.
- Sturdyc, created in 2024, implemented advanced features like loading and refreshing but lacked efficiency and a good eviction policy.
- Otter v2 was designed to overcome all previous limitations, offering high throughput, excellent hit rates, and a flexible API, effectively becoming a Go adaptation of Caffeine.