- The project runs a language model on a BBC Micro (6502 CPU) with only 25KB of user memory (9KB code, 13KB weights).
- The model uses BitNet ternary quantization (-1,0,1) packing 4 parameters per byte, storing 52k parameters in 13KB.
- A Mamba (SSM) architecture is chosen over attention or GRU because it requires fixed-size state and avoids instability issues on the 6502.
- Inference is coded in C (CC65) and transferred to the BBC Micro via a custom audio cable and PlayUEF.