The PowerPC Has Still Got It (Llama on G4 Laptop)
10 days ago
- #AI
- #Vintage Computing
- #PowerPC
- A 2005 PowerBook G4 was used to run a modern large language model (LLM), demonstrating its capabilities despite being outdated.
- Apple's PowerPC microprocessors, used two decades ago, were custom chips before the M-series, with mixed opinions on their superiority.
- Andrew Rossignol modified llama2.c, an open-source LLM inference engine, to run the TinyStories model (110M parameters) on the PowerBook G4.
- The PowerPC's big-endian architecture required data conversion and manual memory alignment, differing from modern little-endian systems.
- Performance was slow (0.77 tokens/second), but using AltiVec (PowerPC's vector processing) improved speed slightly to 0.88 tokens/second.
- Despite hardware limitations (1GB RAM, 32-bit CPU), the experiment showcased the PowerBook G4's ability to run modern AI with optimizations.