A concrete explanation of how a cache works
4 days ago
- Processor speed outpaces memory speed, so a small, fast cache inside the CPU stores frequently used data to reduce slow main memory accesses.
- The principle of locality—temporal (re-accessing recent data) and spatial (accessing nearby data)—guides which data is cached.
- Cache hierarchy includes L1 (split into instruction I$ and data D$), L2 (larger, unified), and L3 (shared across cores in multi-core systems).
- Cache performance is measured by hit latency, miss latency, and average access time; improvements come from changing cache size, speed, or organization.
- Cache is essentially a hash table in hardware, using SRAM, with blocks indexed by part of the address, tags to reduce collisions, and a valid bit.
- Tag overhead (space and time) is managed by parallel access: tag array is checked while data array is accessed ahead of time.
- Associative caches (direct-mapped, fully associative, set associative) balance hit latency and collision rate; set associative (e.g., 2-way) is common.
- Write policies include write-through (immediate memory update) and write-back (update on block replacement, using a dirty bit); write misses often use write-allocate.
- Software restructuring (e.g., swapping loops for spatial locality, chunking data for temporal locality) can improve cache efficiency, though modern compilers handle this.