The Open/Closed Problem in AI

8 hours ago

The author attended the MLSys conference and observed a focus on improving efficiency in training and deploying LLMs.
The Open/Closed problem is illustrated through the evolution of computer graphics: from open CPUs to closed fixed-pipeline GPUs, back to open programmable GPUs, and now to closed specialized ASICs for AI inference and training.
Current AI systems use open-loop learning, where models are trained externally and don't learn after deployment, unlike the closed-loop learning of the human brain, which updates itself internally based on sensory predictions.
The AI industry is heavily investing in optimizing open-loop learning through hardware like ASICs and kernel improvements, which inadvertently makes closed-loop learning more difficult to achieve.
Specialized hardware for inference assumes static models, with read-only parameters and separated compute and memory, contrasting with the needs of closed-loop learning, which requires dynamic weights, fine-grained updates, and fused memory-compute.
The author warns that focusing on open-loop efficiency may hinder breakthroughs in closed-loop learning, as hardware becomes increasingly specialized around the current paradigm, limiting experimentation and innovation.

Hasty Briefsbeta