Why AI chatbots invent references – and how to spot them

By Terje Lien, teacher · 14 October 2026 · 2 min read

Ask an AI chatbot for five sources supporting a claim, and you will get five neat references with authors, year, journal and page numbers. Often some of them exist. Sometimes none do.

Why it happens

A language model does not look anything up in a library while it writes. It predicts which words are likely to come next. A reference has a very predictable pattern – surname, initial, year in brackets, title, journal – and the model is very good at patterns. The result is a reference that looks right whether or not the work exists.

The error is hard to spot because invented references are often built from real parts: a researcher who exists, a journal that exists, a topic the researcher has actually written about. Only the combination was never published.

It has real consequences

In 2023 a US federal court sanctioned two lawyers who had filed a brief citing court decisions that turned out to have been invented by ChatGPT (Mata v. Avianca). Similar cases have since appeared in courts, government reports and academic papers in several countries. In every case the problem was the same: nobody checked whether the sources existed before the text went out.

Warning signs

• The title reads like a summary of exactly your question – almost too good.
• The DOI goes nowhere, or to a completely different paper.
• Volume and page numbers do not fit the journal's actual issues that year.
• The book is not in any library catalogue, even though it is supposedly from a major publisher.
• The author exists, but the text does not – check their publication list.

What to do

Use AI for ideas and search terms, never as a source in itself. Every reference in work you hand in should be one you have seen yourself – in full text, or at least in a library catalogue. The quickest way to sort a long list is to paste it into Refvera: tick off what is found, and you know exactly which ones to check by hand.

Remember that "not found" does not prove a source is invented. It is a reason to look further – nothing more.