Avoiding AI hallucinations in learning

We speak of a hallucination when a language model makes a statement that is false or invented but phrases it as if it were certain. Typical examples are invented sources and quotations, wrong figures, mixed-up sections of laws, non-existent technical terms or answers to questions the model actually knows nothing about.

The cause lies in how the models work. A language model generates text by calculating, word by word, what is likely to follow. It has no built-in understanding of true and false. If it has not learned reliable patterns for a question, it still produces a plausible answer, and plausibility is not accuracy.

In many applications a wrong AI answer is annoying. In learning it is dangerous, because learners trust the material and commit it to memory. A wrongly learned deadline, an invented rule or a twisted definition lingers until it causes damage at work. And because AI answers sound so convincing, learners hardly notice errors; after all, they are only just learning the topic.

There is also a loss of trust. Once learners discover a wrong answer, they distrust the right ones too. A learning platform that flags errors openly and corrects them quickly keeps that trust better than one that hides errors behind smooth wording.

Hallucinations cannot be ruled out entirely, but they can be greatly reduced. The following measures work best in combination.

The most widespread method is called retrieval-augmented generation. Before the model answers, the relevant passages from the course content and stored sources are retrieved and passed to the model. The model is instructed to answer only on that basis and to say when the basis is insufficient.

This reduces hallucinations considerably but does not eliminate them. The model can summarise sources incorrectly or fill gaps with its own knowledge. A second stage that checks the finished answer against the sources once more before it reaches the learner is therefore sensible.

A learning system should rather say “I can’t answer that reliably on the basis of this course” than give a smooth wrong answer. That runs counter to the urge to always be helpful, but in a learning context it is the right standard. Learners also learn something important about AI from it: that its answers have limits.

Guardrails keep the system in its role. Questions outside the course topic, attempts to manipulate it or requests for individual legal, medical or financial advice are recognised and politely declined. That means fewer situations in which the model would have to answer without a basis.

In Universe, answers in the course go through a verification chain that checks statements against sources; anything that cannot be supported is flagged as such. When slides are created, two language models independently check verifiable claims against archived sources before graphics and narration are produced. Guardrails keep the avatar, tutor chat and voice dialogue within the course, fend off injected instructions and log attempts. Content errors can be reported and tracked as errata.