- Additional data does not always reduce uncertainty; surprising data can increase posterior variance in Bayesian models.
- In a beta-binomial model, observing a success increases posterior variance if the prior odds against success are more than 2:1.
- In a normal-normal model, posterior variance always decreases regardless of the observation's surprise level.
- The third moment of a distribution determines whether positive or negative surprises raise uncertainty in continuous learning scenarios.
- References and proofs show that uncertainty increase with surprising signals is mathematically established but not widely known.