Five Years of Tinygrad

4 months ago

First commit to tinygrad was on October 17, 2020.
The company has grown to 6 people and the codebase is 18,935 lines (excluding tests).
The goal is to compete with NVIDIA by focusing on a sovereign software stack before taping out a chip.
Tinygrad is removing LLVM dependencies to run on AMD GPUs with pure Python.
The project outperforms PyTorch on many workloads and aims to be around 20,000 lines when complete.
Tinygrad follows the 'Elon process' for software: 'Make the requirements less dumb. The best part is no part.'
The tiny corp operates openly with a Discord and GitHub, funding through a computer sales division ($2M revenue/year).
AMD contract secured via public Twitter negotiation to get MI350X on MLPerf for Llama 405B training.
Hiring is based on contributions to the repo, with minimal meetings and a focus on improving tinygrad.
Mission: 'Commoditize the petaflop.'

Hasty Briefsbeta