In the realm of foraging, the quest for safe mushrooms can be perilous, especially when relying on artificial intelligence. A recent examination by Polish software engineer Piotr Migdał has unveiled significant shortcomings in the mushroom identification capabilities of leading AI models.
Testing AI Models
Migdał’s experiment involved a dataset comprising 55 mushroom species, both safe and deadly, sourced from Denmark and Poland. He evaluated the performance of various AI models, including ChatGPT and Qwen, by processing 1,040 photographs. Each model was tasked with identifying the most likely species along with four alternatives for each image.
Performance Metrics
The results were concerning. The top-performing model, Gemini-3.8-flash, achieved only a 65 percent accuracy rate for its first guess. While it correctly identified species within its top five guesses 85 percent of the time, the implications of a 35 percent error rate are significant for those relying on AI for safe foraging decisions.
In stark contrast, Qwen3.8-27b underperformed, with a mere 13 percent accuracy on its first guess and only 24 percent within its five guesses. The models’ mistakes were not trivial; for instance, a deadly webcap was misidentified as a chanterelle, a common error that has led to fatalities among foragers.
Dangerous Misidentifications
Among the most alarming findings, the death cap mushroom was incorrectly classified as edible 16 percent of the time, while the fool’s funnel and fatal dapperling were misidentified 48 percent and 31 percent of the time, respectively. The model with the highest rate of false positives, Qwen3.8-27b, labeled poisonous mushrooms as safe 36 percent of the time. Meta’s Muse-spark-1.2 performed better with an eight percent false positive rate, though this was attributed to its tendency to refrain from guessing.
Expert Caution
Migdał emphasized the risks associated with trusting AI for mushroom identification, stating, “do not eat a mushroom because AI told you it is safe.” He advocates for a cautious approach, noting that even experienced foragers can struggle to distinguish between safe and dangerous species. While the dataset used in the study is available for further exploration, the findings serve as a stark reminder of the limitations of AI in critical safety applications.
This article was produced by NeonPulse.today using human and AI-assisted editorial processes, based on publicly available information. Content may be edited for clarity and style.
