The Trouble with 'Ntile()'
2 days ago
- The tidyverse packages generally improve on base R's rough edges, and the author is a fan.
- The post warns against using dplyr::ntile() for statistical analysis due to its tie-breaking rule based on row number, which is non-random and statistically indefensible.
- ntile() produces equal-size groups, not quantile-based groups, and can misallocate tied values at break points, leading to incorrect binning.
- The correct approach for quantile binning is to use quantile() to define breaks and cut() to assign observations, ensuring ties are not broken and groups are based on distributional properties.
- The choice of left-closed or right-closed intervals (edge case handling) is arbitrary but should be consistent, with examples showing different results.
- The tie-breaking behavior of ntile() is inherited from the SQL NTILE function, which is also not suitable for statistical analysis, especially in regulatory contexts like pharmaceuticals.