Recreating the Bell Labs Switch Experiment with Agents

4 months ago

The book 'The Art of Doing Science and Engineering' by Richard Hamming discusses the challenges of scientific and engineering work, including human difficulty in accepting randomness.
A psychologist friend of Hamming conducted an experiment with a random switch machine to study how theories evolve, revealing participants' inability to recognize randomness.
The experiment showed that people tend to create theories even when there is no pattern, highlighting a gap in education about randomness.
An experiment was conducted to test if LLMs (Large Language Models) could recognize randomness in a similar setup, with results showing they also struggled to identify the lack of pattern.
Different models (Gemma 3 12b, GPT 5.2, Opus) were tested, and all failed to conclude the randomness, instead creating complex theories about switch patterns.
The experiment's architecture involved creating a mystery tool, an agent, and an orchestrator to simulate the switch box scenario.
The results suggest that LLMs, like humans, have difficulty accepting randomness and tend to overcomplicate explanations.

Hasty Briefsbeta