Floor and Ceil versus Denormals on CPU and GPU

4 days ago

The article discusses the behavior of floating-point functions like floor, ceil, trunc, and round, focusing on a specific case: floor(-1.175493930432748e-38).
The input number is a subnormal (denormal) value, represented hexadecimally as 0x807FFFFD, which is close to zero but negative.
Mathematically, floor(-1.175493930432748e-38) should be -1.0, but the actual result depends on whether the platform preserves or flushes denormals to zero.
If denormals are preserved, the result is -1.0; if flushed to zero, it becomes -0.0, leading to nondeterminism across different hardware (e.g., CPU vs. GPU).
The author tested various compiler flags and settings, finding that platforms like Microsoft Visual Studio preserve denormals by default, while others like NVIDIA GPUs flush them for performance.
The article notes that DirectX specifications require GPUs to flush denormals on input and output of floating-point operations.
To ensure consistent behavior, the author provides custom HLSL functions (DeterministicFloor and DeterministicCeil) using bit manipulation to handle denormals deterministically across platforms.

Hasty Briefsbeta