The Information Theory Behind Why AI Writing Sucks
14 hours ago
- #Alignment Techniques Impact
- #AI Writing Uniformity
- #Authorial Voice Distribution
- The author notices AI writing has a uniform voice compared to human authors, who each have unique stylistic spaces.
- A unique authorial voice is a specific probability distribution; recognizable due to low-frequency, high-impact choices.
- AI's uniform voice stems from alignment techniques like RLHF, which push models toward a safe 'Annotator Consensus Dialect' to minimize rating variance.
- Prompting AI to mimic a style shifts the distribution's mean but applies variance-collapsed mechanics, smoothing out structured irregularities inherent to human writing.
- Methods like temperature scaling to widen AI's distribution introduce stochastic noise rather than the conditional eccentricity of human authors, keeping AI output detectably uniform.