May 2021 Gwern.net Newsletter
a day ago
- Gwern.net's May 2021 newsletter covers updates on AI hardware, diffusion models, and optogenetics, among other topics.
- Notable AI advances include TPUv4 chips with >1 ExaFLOPS performance, ZeRO-Infinity for scaling deep learning to trillions of parameters, and diffusion models surpassing GANs in image synthesis.
- In genetics, highlights include the complete human genome sequencing, optogenetic therapy for partial vision recovery, and ultra-safe cells with synthetic viral resistance.
- Meta-science reveals that subliminal advertising claims are a myth, and a study shows nudge effects are overestimated by 6.2x due to publication bias.
- A proposed 'Master Synthesis' suggests intelligence results from small, simplicity-weighted programs discovered via search over smooth loss landscapes, with human intelligence linked to overall neural resource pools.
- Other updates include a simple explanation of diffusion models (DDPMs), a tool for running GPU processes in idle time, and the release of a 6B-parameter GPT-3 model by EleutherAI.