ChatGPT, Gemini, Copilot and Perplexity: how often they get answers wrong, in independent studies
What broadcasters, universities, fact-checkers and consumer groups found when they checked AI assistants' answers, each study dated and linked.
45% of answers had at least one significant issue.
- Wrong or missing sources are the most common problem. In the largest study, 31% of answers had a significant sourcing issue. 52% of Google's AI Overviews in another test made a claim that the pages they cited did not support.
- The assistants rarely say they don't know. They declined 0.5% of questions in the largest study, down from 3% a few months earlier. Asked where 200 news passages came from, ChatGPT got 134 wrong and showed doubt in only 15 of its answers.
- It depends on the assistant and the question. In the same study, Gemini had a significant issue in 76% of its answers and Perplexity in 30%. Chatbots answered 88 to 96% of ordinary news questions correctly, and as few as 19% when the question held a false premise.
- Some numbers improve, and answers change. On the BBC's questions, answers with a significant issue fell from 51% to 37% in a few months. Full Fact found figures repeated months after they changed, and different answers to the same question on different days.
The 9 studies, newest first
Each number is the study's own, with its own measure. They test different questions in different ways, so they cannot be ranked against each other.
- Full Fact, the UK fact-checking charity
7,625 answers checked. Full Fact gives no error rate.
What they tested
Wrong on Sunday, right on Monday. Why AI chatbots have a confidence problem. They tested Claude, ChatGPT, Gemini, DeepSeek and Grok, through their developer access. The same 22 questions on UK topics, asked every day from 15 July to 22 September 2026.
The chatbots were right much of the time. The errors were of a few kinds: figures that had gone out of date (an asylum support rate of £8.86 a week, months after it rose to £9.95), DeepSeek naming Keir Starmer as prime minister in September, after the change on 20 July, and different answers to the same question on different days.
Answers through developer access are close to, but not the same as, what people see in the apps.
Source: Full Fact.
- NPR, with questions from NewsGuard
3 in 4 chatbot answers pushed back on the false story, on average.
What they tested
AI chatbots may be better than search engines in guarding against foreign propaganda. They tested ChatGPT, Gemini, Copilot, Meta AI, Grok and Claude, and the AI summaries of Google, Bing and DuckDuckGo. 30 questions built on 15 false stories spread by China, Iran and Russia.
The AI summaries in search failed more often than the chatbots. Google's AI Overviews pushed back most of the time, and Bing's summaries failed to most of the time.
Google said it disagreed with the method.
Source: NPR.
- Stanford University researchers
19 to 70% of questions answered correctly when the question held a false premise, against 88 to 96% for ordinary ones.
What they tested
Evaluating Commercial AI Chatbots as News Intermediaries. They tested Gemini 3 Flash and Pro, Grok 4, Claude 4.5 Sonnet, GPT-5 and GPT-4o mini. 2,100 questions about the day's BBC News stories, asked from 9 to 22 February 2026, in six of the BBC's language services.
Every model did worst in Hindi: 79%, against 89 to 91% in the other languages. About seven in ten errors came from the search step: the chatbot did not find the right page, or the page it found said something different.
A research paper that has not yet been peer reviewed.
Source: Stanford University researchers.
- Oumi, for The New York Times
9% of AI Overviews gave a wrong answer, with Gemini 3.
What they tested
AI Overviews: correct, and supported by their sources?. They tested Google's AI Overviews, with Gemini 2 and then Gemini 3. About 4,000 factual questions from SimpleQA, a public test set, run through Google Search in late 2025 and early 2026. The main count covers 1,609 AI Overviews.
52% had at least one claim that the pages they cited did not support. On a smaller set of 158 questions, wrong answers fell from 11% with Gemini 2 to 6% with Gemini 3, while answers with an unsupported claim rose from 37% to 56%.
Google said the test set does not reflect real searches and has errors of its own. Oumi sells the tools it used to judge the answers.
Source: Oumi.
- Which?, the UK consumer group
55 to 71 out of 100: the lowest and highest scores, Meta AI and Perplexity.
What they tested
AI tools giving risky consumer advice. They tested ChatGPT, Gemini, Google's AI Overviews, Copilot, Meta AI and Perplexity. 40 questions about money, the law, health and consumer rights. Which? experts scored each answer out of 100 for accuracy, relevance, clarity, usefulness and responsibility.
Google's AI Overviews scored 70, Gemini 69, Copilot 68 and ChatGPT 64. Asked about a £25,000 ISA allowance (the real limit is £20,000), ChatGPT and Copilot did not correct the mistake and gave advice that could break the tax rules.
Source: Which?.
- European Broadcasting Union and BBC, with 22 public broadcasters in 18 countries
45% of answers had at least one significant issue.
What they tested
News Integrity in AI Assistants. They tested the free versions of ChatGPT, Copilot, Gemini and Perplexity. 2,709 answers to questions about the news, in 14 languages, collected in late May and early June 2025. Journalists at each broadcaster reviewed them.
Sourcing was the most common problem, in 31% of answers, then accuracy (20%) and missing context (14%). Gemini had a significant issue in 76% of its answers, Copilot in 37%, ChatGPT in 36% and Perplexity in 30%. The assistants declined just 17 of 3,113 questions (0.5%). On the BBC's own questions, the share with a significant issue fell from 51% in the first round to 37%.
Source: European Broadcasting Union and BBC, The full report (PDF).
- NewsGuard
35% of answers repeated a false claim in August 2025, up from 18% a year before.
What they tested
AI False Claims Monitor: one year on. They tested ten leading chatbots, among them ChatGPT, Gemini, Copilot, Claude, Meta AI, Grok and Perplexity. Each month, questions about false claims that were spreading in the news.
A year earlier the chatbots often declined to answer. By August 2025 they answered every question, and more of the answers were wrong. In January 2026, eleven chatbots repeated false claims in more than 28% of answers.
NewsGuard sells ratings of news websites.
Source: NewsGuard, January 2026 round.
- Tow Center for Digital Journalism, Columbia University
60%+ of the questions were answered wrongly, across the eight tools.
What they tested
We compared eight AI search engines. They're all bad at citing news. They tested ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Copilot, Grok 2, Grok 3 and Gemini. 1,600 questions. Each gave a passage from one of 200 news articles and asked for the article's headline, publisher, date and link.
Perplexity got 37% wrong and Grok 3 got 94% wrong. ChatGPT got 134 of its 200 answers wrong and showed doubt in only 15 of them. More than half of the answers from Gemini and Grok 3 cited links that were made up or broken.
Source: Tow Center for Digital Journalism.
- BBC
51% of answers had a significant issue.
What they tested
Representation of BBC News content in AI Assistants. They tested ChatGPT, Copilot, Gemini and Perplexity. 100 questions about the news, with the assistants allowed to read the BBC's website. BBC journalists who know each topic reviewed the answers.
19% of the answers that cited BBC content added factual errors, such as wrong numbers or dates. 13% of the quotes taken from BBC articles were changed, or were not in the article at all.
Source: BBC, The Register's report.
What it means for a brand
- Put the facts where assistants lookMany errors start at the search step: the chatbot finds the wrong page, or one that says something different. Keep prices, products, opening hours and contact details on your own site, in plain text, on pages AI crawlers may read. Which crawlers to let in.
- Fix old facts where they liveFull Fact found chatbots repeating figures months after they changed. Update or remove old pages, listings and press releases that still carry an old price, address or product.
- Don't judge by one answerThe same study found a significant issue in 30% of one assistant's answers and 76% of another's, and Full Fact saw answers change from day to day. One answer you saw may not be the one your customers see.
How we chose the studies
We looked for studies that checked the answers of the widely used AI assistants against the facts, published from 2025 on, with their method in public: by broadcasters, universities, fact-checkers, consumer groups and newsrooms. We left out studies by the AI companies themselves, and by companies that sell AI visibility or brand-monitoring services. Where a study's author sells something related, we say so under that study.
Each number is as the study reported it, in its own words: "significant issue", "wrong", "repeated a false claim". Most studies asked about the news, which changes fast and is hard to get right, so their numbers may not hold for other kinds of questions. Assistants change often, and a study describes the versions it tested, when it tested them. Checked on 29 Oct 2026.