The talk explains how chips work from basic logic gates to AI accelerators, focusing on the multiply-accumulate (MAC) unit as the fundamental primitive for matrix multiplication.
The Dadda multiplier uses AND gates and full adders to efficiently perform multiplication, with the number of full adders equaling the product of bit widths.
Data movement costs dominate chip design—a mux for selecting register file inputs can require 24× the gate count of the MAC itself, motivating systolic arrays to maximize compute per communication.
Systolic arrays store weight matrices locally and vector inputs temporarily, reducing bandwidth needs by reusing data across cycles.
Clock cycles synchronize chip logic via pipeline registers, but loops (e.g., running sums) impose speed limits because feedback loops cannot be pipelined without changing computation.
FPGAs use lookup tables (LUTs) and muxes to emulate arbitrary gates, but are 10× less efficient than ASICs due to overhead from configurability.
Deterministic latency in chips often relies on scratchpads (explicit memory access) rather than caches (hardware-managed), which introduce non-determinism.
CPU cores are larger than GPU cores due to branch predictors and extensive caches; GPUs strip these for more parallel compute units.
The brain is slower and uses unstructured sparsity, while chips are clocked high for throughput; energy consumption scales with gate switching (dynamic power).
A GPU can be seen as a collection of many small TPUs (tensor cores in SMs), each with local memory, while a TPU has fewer, larger systolic arrays with a central vector unit.