I mentioned in my earlier post about ChatGPT safety that there’s a problem with factual accuracy in the large language models (LLMs) that sit behind the latest popular chatbots.
Unfortunately, these models aren’t good at admitting when they don’t know something, and regularly spew out inaccurate ‘facts’ with unwarranted confidence. These invented facts are becoming known as hallucinations due to their illusory nature. (Some people have raised objections to that terminology, due to concerns that it’s making AI systems sound more human, but I haven’t come across a good alternative yet.)
Some of the so-called guard rails that have been added to ChatGPT in particular do attempt to address this. A common response is to cause the chatbot to respond that “As an AI…” and then state that it can’t comment on certain things. However, this applies only in a small number of pre-determined cases, such as moral judgements or current events.
More often than not, LLM responses are clear and unequivocal. Often, unequivocally wrong.
Oxford Semantic Technologies drew attention to a simple, moderately harmless example of this kind of hallucination in their blog post about animated films. Apparently, if you ask ChatGPT about the first full-length animated film, it gives the answer of Snow White. But if you ask it about an earlier, Argentine animated film called El Apóstol , it also ‘knows’ (i.e. can recite) the release date of that film, which was some 20 years earlier.
This highlights one of the main problems with the idea that generative AI systems ‘know’ anything at all. There’s no reasoning engine sitting behind the language model to reconcile conflicting ‘facts’ into a unified world view. A human encountering these two ideas in parallel would quickly update their mental model (“ah, Snow White was only the first animated film in English, that’s why it’s more widely known!”), but this isn’t possible for a statistical model.
The Guardian and the Washington Post have both highlighted recent examples of ChatGPT inventing whole articles, and attributing them to their respective publications.
In the case of the imaginary Guardian article, a journalist was perplexed to be contacted by someone asking where they could find a copy of an article which had never been written or published, although it was close in topic to their usual areas of interest. ChatGPT had simply made it up.
The Washington Post has a much more sinister example, with a sexual abuse scandal being invented from thin air, but naming a real professor as the culprit. Again, ChatGPT’s text generation model credited this story to an article, this time in the Washington Post — that didn’t exist. (We should probably add libel to the list of legal problems that ChatGPT could accidentally cause.)
When you think about how LLMs work, it’s not really surprising that this kind of thing can happen. The model is running on pure statistics, using only the probabilities of which words and phrases are likely to come next.
My approach to getting non-experts to understand ChatGPT is to start from your phone’s autocomplete or predictive text function. Most people have seen, if not used, their phone’s attempt to guess which word they’re going to type next. You might have joined in with one of those social media memes where everyone completes the same sentence with their own phone’s suggested next words. ChatGPT and other LLMs are doing the same thing, on a much bigger scale.
Sometimes, as in the Snow White example, the models are mimicking something that has genuinely (but incorrectly) been said by a lot of people.
Sometimes, it’s saying things that are ‘a bit like’ stuff people say — people quote newspaper articles, people talk about sexual abuse allegations — without anchoring that to actual real-world facts or events.
I was recently asked what prompts could be used to get ChatGPT to be more accurate, and avoid the need for extensive fact-checking. Unfortunately, that’s impossible. Prompts can’t fix a problem that exists at the level of the underlying technology, so if you want to use ChatGPT, you need to understand the nature of these problems and where they come from.
