Are polynomial features the root of all evil? (2024)
a year ago
- #machine-learning
- #polynomial-features
- #regularization
- Polynomial features in machine learning are often misunderstood and considered problematic due to misconceptions.
- High-degree polynomials can be effectively controlled using regularization, contrary to common belief.
- The standard polynomial basis is not suitable for estimation tasks, leading to overfitting and oscillations.
- Alternative polynomial bases like Chebyshev and Legendre are better for interpolation but still problematic for fitting noisy data.
- Bernstein polynomials offer a solution by providing coefficients with uniform 'units', making regularization straightforward.
- Bernstein polynomials are widely used in computer graphics but less known in machine learning.
- Using Bernstein polynomials, high-degree polynomials can fit data without overfitting, as demonstrated with degree 50 and 100 examples.
- The reputation of high-degree polynomials as inherently problematic is a myth that can be debunked with proper basis selection and regularization.