Booting 5000 Erlangs on Ampere One 192-core
14 days ago
- #ARM64
- #Virtualization
- #Erlang
- Booting 5000 Erlang VMs on an Ampere One 192-core server with 1TB RAM using KVM acceleration.
- Introduction of 'little_loader', a new bootloader for ARM64 qemu devices, enabling A/B upgrades and other Nerves features.
- Significant performance improvements with KVM, reducing boot times to single-digit seconds and lowering memory usage by ~500MB per VM.
- Achieved running 3389 simultaneous connected devices before hitting memory limits, with potential to scale further after memory tuning.
- Memory tuning adjustments for both Linux and BEAM VM, including allocator changes and Linux kernel parameter tweaks, allowed running 5100 devices.
- The setup provides a practical way to test Nerves applications without physical hardware, useful for development and testing.
- Future work includes exploring NUMA and KVM interactions, performance graphing, and further tooling development for Nerves.