I'm afraid of spiders. So I made AI look at 2k of them
a day ago
- The author, who is afraid of spiders, tested nine AI models on 2,000 spider photos to see how accurately they could identify species.
- The benchmark used a multiple-choice format with 20 possible names per photo, based on research-grade iNaturalist observations.
- Gemini 3.8 Flash achieved the highest accuracy at 49.85%, followed by GPT-6 Astra (47.50%) and Claude Fable 5.1 (43.10%).
- Most models made mistakes within the same family, with high family-level accuracy (e.g., Gemini 87% of misses still in correct family).
- Muse Spark 1.3 was the cheapest model ($0.93 for 782 correct) and placed fourth, offering a cost-effective trade-off.
- The author notes that getting nearly half correct is respectable given the difficulty of distinguishing similar species, especially without microscopic details.