World Emulation via DNN
a year ago
- #neural-networks
- #machine-learning
- #virtual-reality
- Author created a neural world from a forest trail, explorable in a web browser.
- The neural world is generated by a neural network without traditional game elements like geometry or scripts.
- The project was inspired by a previous attempt to mimic 2D video game worlds with neural networks.
- Data collection involved recording 15 minutes of video and motion from a forest trail using a phone.
- Initial attempts resulted in poor quality ('forest-flavored soup'), leading to several upgrades in the neural network.
- Upgrades included more control information, memory, and processing at multiple scales.
- Further improvements involved making the network bigger, adjusting the training objective, and training longer.
- The final model uses a dataset of 22,814 frames and took ~100 GPU-hours to train.
- Neural worlds are likened to photographs, capturing reality without artistic intervention.
- Future potential includes more lifelike, automated details like trees bending or birds singing.
- Neural worlds could become a new creative medium, distinct from traditional video games.