A Number with a Shadow

6 hours ago

Floating point arithmetic can lead to errors when computing derivatives due to catastrophic cancellation at small step sizes.
Dual numbers provide a way to compute exact derivatives by pairing a value with its derivative, avoiding approximation errors.
Dual numbers follow arithmetic rules derived from calculus, such as the product rule and chain rule, to propagate derivatives accurately.
A Rust implementation of dual numbers demonstrates how to compute exact derivatives with minimal code overhead.
Generic programming in Rust allows the same functions to compute both values and derivatives by simply changing the type parameter.
The 'tang' library leverages dual numbers and generic programming to provide exact derivatives across various mathematical operations and decompositions.
Dual numbers offer performance benefits over finite differences, being both faster and more accurate.
The 'tang' library supports a wide range of operations, from linear algebra to symbolic computation and GPU acceleration.
Despite being generic, 'tang' remains competitive in performance with specialized libraries like 'nalgebra' and 'glam'.
The library is used in training large models and simulations, blurring the line between simulation and learning by providing exact derivatives.

Hasty Briefsbeta