floating-point

Bilingual

Faster floating point math with Rust's new API
2 months ago
- Floating point math is slower than integer math due to compiler conservatism to preserve exact results.
- Rust 1.98 introduces algebraic operators (e.g., `algebraic_add`) that allow compiler optimizations for floats.
- Pairwise summation with algebraic operators can match or beat NumPy speed while maintaining accuracy.
- Algebraic operations enable SIMD optimizations, reducing CPU instructions per value significantly.
- Combining strict and algebraic operators in algorithms balances performance and numerical precision.
Always use feenableexcept() when doing floating point math - Bert Hubert's writings
2 months ago
- Always use feenableexcept() in C/C++ floating point code to catch exceptions like division by zero, invalid operations, and overflow.
- By default, ISO C silently allows infinities and NaNs, which can propagate and produce undetected bogus results in calculations.
- Using feenableexcept(FE_DIVBYZERO | FE_INVALID | FE_OVERFLOW) on Linux with -lm causes SIGFPE on dubious operations, crashing the program for safety.
- Compiler optimization (e.g., SIMD) may be inhibited by exception trapping; enabling -fno-trapping-math allows optimization but prevents using feenableexcept().
- Additional exceptions FE_UNDERFLOW and FE_INEXACT exist; FE_INEXACT is common and usually not useful for debugging.
Matt Godbolt's blog
2 months ago
- SIMD auto-vectorization works efficiently for integer sums by processing 8 integers per loop iteration and then summing the 8 subtotals with fix-up code.
- Floating-point addition is not associative due to rounding, so compilers cannot safely regroup additions the same way, leading to scalar per-element addition even when processing 8 floats per iteration.
- Workarounds include using -Ofast or -funsafe-math-optimizations (global, risky) or using a GCC attribute to assume associativity for a specific function; std::reduce with unsequenced execution policy did not work in testing.
Printing floating point numbers in binary
2 months ago
- Hex to binary conversion works for floating point numbers similarly to integers, by converting each hex digit to binary.
- Python's float.hex() method returns a hex representation with a 'p' exponent indicating a power of 2, not 16.
- To get the binary representation, convert the hex digits after the point to binary using bin() on the integer value, then adjust the fraction point by the exponent.
- Padding with zeros may be necessary when converting hex digits to ensure correct bit alignment.
Log is non-monotonic in PHP and Lua
2 months ago
- The author engineered an example (log(2.93, 10+2^-49) vs log(2.93, 10)) that produces surprising results in PHP and Lua, but not in Rust or C#, due to floating-point implementation differences.
- Languages like PHP and Lua special-case log bases 10 and 2 by using dedicated log10/log2 functions for better precision, but this creates a discontinuity when compared to the general log(x)/log(base) method.
- The inconsistency arises because the two evaluation methods (log10 vs ln/ln) can produce errors in different directions, breaking the assumption that log(x, a) < log(x, b) when a > b.
- The author suggests that Lua should add separate log10/log2 functions and remove special casing from log, but Lua 5.2 did the opposite. PHP's special-casing is aimed at users who don't read the manual.
- Additional edge cases are discussed, such as log(x, 1) returning NaN in PHP and C# but not in Lua or Python, and the use of M_PI approximation in PHP's default log.

floating-point

Faster floating point math with Rust's new API

Always use feenableexcept() when doing floating point math - Bert Hubert's writings

Matt Godbolt's blog

Printing floating point numbers in binary

Log is non-monotonic in PHP and Lua