Think of a Number

a year ago

Sam Altman's claims about AGI being imminent are criticized as irresponsible hype.
Current AI can handle undergraduate mathematics but fails at PhD-level tasks, indicating no true AGI.
The author proposes creating a secret database of hard number theory problems to test AI's mathematical understanding.
Questions should require non-negative integer answers, be beyond undergraduate level, and not easily guessable or found online.
The experiment aims to distinguish between AI's pattern-matching and genuine mathematical thinking.
The author seeks collaboration from PhD-level number theorists to contribute challenging problems.
AI companies will be invited to test their models against the database, with results made public.
The project contrasts with FrontierMath by ensuring questions are not pre-exposed to AI models.
Example questions would resemble those in FrontierMath but be harder and more uniformly difficult.
The goal is to assess whether AI can truly think mathematically, beyond stochastic parroting.

Hasty Briefsbeta