18 hours ago
- The talk explains how chips work from basic logic gates to AI accelerators, focusing on the multiply-accumulate (MAC) unit as the fundamental primitive for matrix multiplication.
- The Dadda multiplier uses AND gates and full adders to efficiently perform multiplication, with the number of full adders equaling the product of bit widths.
- Data movement costs dominate chip design—a mux for selecting register file inputs can require 24× the gate count of the MAC itself, motivating systolic arrays to maximize compute per communication.
- Systolic arrays store weight matrices locally and vector inputs temporarily, reducing bandwidth needs by reusing data across cycles.
- Clock cycles synchronize chip logic via pipeline registers, but loops (e.g., running sums) impose speed limits because feedback loops cannot be pipelined without changing computation.
- FPGAs use lookup tables (LUTs) and muxes to emulate arbitrary gates, but are 10× less efficient than ASICs due to overhead from configurability.
- Deterministic latency in chips often relies on scratchpads (explicit memory access) rather than caches (hardware-managed), which introduce non-determinism.
- CPU cores are larger than GPU cores due to branch predictors and extensive caches; GPUs strip these for more parallel compute units.
- The brain is slower and uses unstructured sparsity, while chips are clocked high for throughput; energy consumption scales with gate switching (dynamic power).
- A GPU can be seen as a collection of many small TPUs (tensor cores in SMs), each with local memory, while a TPU has fewer, larger systolic arrays with a central vector unit.