Generalized Orders of Magnitude
8 days ago
- #Parallel Computing
- #Machine Learning
- #Numerical Computation
- Introduction of generalized orders of magnitude (GOOMs) for stable computation over large dynamic ranges of real numbers.
- Implementation of GOOMs with an efficient custom parallel prefix scan for native execution on parallel hardware like GPUs.
- Demonstration of GOOMs outperforming traditional approaches in three experiments: compounding real matrix products, estimating spectra of Lyapunov exponents, and capturing long-range dependencies in deep recurrent neural networks.
- GOOMs combined with efficient parallel scanning offers a scalable and numerically robust alternative to conventional floating-point numbers for high-dynamic-range applications.