Beating Redis with a Dictionary and Redis
10 months ago
- #performance optimization
- #Frappe Framework
- #Redis caching
- Frappe Framework's dynamic design requires runtime computations stored in Redis cache for performance.
- Redis cache invalidation ensures all processes see updated configurations and schemas.
- Flamegraph analysis reveals significant overhead with Redis calls consuming half the request time.
- Redis's IPC, de-serialization, and lack of CPU cache benefits contribute to slowdowns.
- Using a global dictionary for caching is fast but lacks coordinated invalidation across processes.
- Redis client-side caching with invalidation messages provides a solution for local caching with global consistency.
- Key considerations include limiting tracked keys, managing local cache size, and having contingency plans.
- Implementation results show dramatic speedups: 132x in microbenchmarks and 1.47x in ERP workloads.
- Frappe v16 aims for over 2x speedup with these and other performance improvements.