AI chatbots can't be trusted for news, EU study warns
What's the story
A recent study by the European Broadcasting Union has raised concerns over the reliability of AI chatbots in delivering accurate news. The study evaluated four popular AI assistants - OpenAI's ChatGPT, Microsoft's Copilot, Google's Gemini, and Perplexity. It found that these systems often provided erroneous information about major news events. In fact, nearly half of all responses contained at least one significant issue related to accuracy or relevance.
Confusion
AI assistants confused parody with real news
The study found that responses from the AI assistants were often confused with parodies or contained incorrect dates. Some even went as far as to invent events altogether. Outdated information was a common problem in the 3,000 responses provided by these systems. For instance, when asked about the current Pope, ChatGPT, Copilot, and Gemini incorrectly stated it was Francis instead of Leo XIV who had succeeded him after his death.
Performance comparison
Gemini least accurate among tested AI assistants
Among the four AI assistants tested, Gemini was found to be the least accurate. The system had significant issues in 76% of its responses, more than double that of the other assistants. This poor performance was mainly attributed to its sourcing capabilities. Public media outlets from 18 European countries had posed similar questions on current affairs to these AI chatbots between May and June this year for this study.
Expert opinion
'AI assistants not reliable for news consumption'
Jean Philip De Tender, the European Broadcasting Union's Deputy Director General, and Pete Archer, BBC's Head of AI, both emphasized that "AI assistants are still not a reliable way to access and consume news." Their statement comes in light of the study's findings which highlight major accuracy issues with these systems. The results suggest that while AI chatbots can be helpful tools for information retrieval, they should not be solely relied upon for accurate news reporting.