Llama 2 LLM on DOS
a year ago
- #LLM
- #Retrocomputing
- #DOS
- DOS Llama 2 LLM client runs on vintage DOS machines like Thinkpad T42 (2004) and Toshiba Satellite 315CDT (1996).
- The project is open-sourced with executables available on GitHub.
- Porting llama2.c to DOS required overcoming challenges like floating point operations, memory mapping, and DOS filename limits.
- Benchmarks show inference speeds across various vintage and modern systems, with modern systems being faster but some unable to load larger models due to memory issues.
- The project demonstrates the potential of running LLMs on vintage hardware, thanks to the foundational work by Andrej Karpathy.