pelicanmaxxing

Are AI Labs Pelicanmaxxing?

3 months ago

An informal benchmark testing AI models' ability to generate an SVG of a pelican on a bicycle has become widely discussed, leading to speculation that labs may optimize for it (pelicanmaxxing).
The author conducted an experiment generating 1,008 SVGs across 7 frontier models using 48 prompts (8 animals × 6 vehicles), scoring them with an LLM judge and analyzing features.
Results show pelicans are drawn no better than other animals, bicycles no better than other vehicles, and the specific pelican-bicycle combination shows no significant performance boost after adjusting for difficulty.
Direction analysis reveals all pelican-on-bicycle images face right, but this is consistent with common patterns across other combinations and not evidence of memorization.
The study concludes there is little evidence of pelicanmaxxing, though limitations include using a single judge, possible SVGmaxxing (general SVG optimization), and a limited budget.