The Five Types of AI Error

Every sentence was fluent, confident, and completely wrong.

The interaction it described does not exist. The enzyme pathway it cited was incorrect. The clinical consequence it warned about does not happen. But if I had not already known the correct answer, I would have had no reason to doubt it.

That is the central problem with AI errors. They do not look like errors. They look like knowledge. And in this lesson, I want to show you exactly what these errors look like so you can recognise them when they appear.

Let me take you back to something we covered in Module 1. Language models do not “know” things. They generate text based on patterns learned from enormous amounts of training data. Most of the time, those patterns produce accurate output. But sometimes the patterns lead somewhere plausible but wrong.

The term “hallucination” is widely used, and I will use it because you will encounter it everywhere. But it is slightly misleading. The AI is not having a bad day. It is doing exactly what it was designed to do — predicting the most likely next word based on everything it has seen. The problem is that the most likely next word is not always the correct next word.

These are the simplest to understand. The AI states something that is simply not true.

Here is an example. You ask about the maximum dose of amitriptyline for neuropathic pain. The AI tells you 75mg at night. The actual maximum dose in the BNF is 75mg, but the usual starting dose and titration schedule the AI provides are slightly wrong. It suggests starting at 25mg and increasing weekly, when the BNF recommends starting at 10mg and increasing more cautiously.

The numbers are close. Close enough to sound right. But in prescribing, close is not the same as correct. 10mg and 25mg are very different starting doses, particularly for an elderly patient.

Factual errors are the most common type and the most dangerous, because they often involve specific clinical details that directly affect patient care.

This one is remarkably common and catches people out regularly.

You ask AI to support a clinical recommendation with evidence. It cites a study. It gives you a journal name, a year of publication, a first author, and a title. The formatting is perfect. It looks exactly like a real reference.

The paper does not exist. The author has never written an article with that title. The journal is real, but the study was never published in it.

I have seen AI invent references from the British Medical Journal, The Lancet, and the New England Journal of Medicine. The citations look impeccable. They follow the correct formatting conventions. They reference real journals and sometimes even real authors. But the papers themselves are entirely fabricated.

Language models are trained on data up to a certain date. They do not have access to information published after their training cutoff. And they do not tell you when their knowledge stops.

Here is what this looks like in practice. You ask about the NICE guideline for managing a particular condition. The AI gives you a detailed answer based on a guideline that was updated 18 months ago. The recommendations it describes were correct at the time. But the guideline has since been revised, and the current recommendations are different.

The AI does not know this. It cannot know this. It presents the old guidance with the same confidence it would present current guidance.

This is particularly dangerous for clinical topics where guidance changes frequently: anticoagulation, diabetes management, cardiovascular risk assessment. If NICE has updated a guideline since the AI’s training data was collected, you will get the old version without any warning.

This is the American default problem we have discussed throughout this course, and it deserves a concrete example.

You ask AI about managing a patient with an HbA1c of 53. If the AI is thinking in American units, 53 mmol/mol is roughly 7%, which is above target for most diabetic patients. The AI might recommend intensifying treatment.

But if the AI confuses the unit systems, or if it gives you a target based on American Diabetes Association guidelines rather than NICE, the thresholds and recommendations may be different. American guidelines use percent. UK guidelines use mmol/mol. The numbers are different. The targets are different. The treatment escalation criteria are different.

Context errors are particularly insidious because the clinical content is often broadly correct. The AI is not making things up — it is giving you accurate information for the wrong country, the wrong healthcare system, or the wrong clinical context.

These are the subtlest and often the hardest to spot.

The AI presents a set of symptoms accurately. It then draws a conclusion that does not follow from the information it has presented. The individual sentences make sense. The overall reasoning does not.

For example, you describe a patient with bilateral lower limb oedema, raised jugular venous pressure, and breathlessness on exertion. The AI correctly identifies these as cardinal signs of heart failure. It then recommends starting a calcium channel blocker — which can actually worsen heart failure by causing further fluid retention.

The AI knew the symptoms. It knew the diagnosis. But it connected the wrong treatment to the right diagnosis. The logic failed at the final step.

Logical errors often happen when the AI combines correct information from different contexts. It might describe a treatment that is appropriate for a different presentation of the same condition, or apply a guideline recommendation to a patient group it was not designed for.

These are the hardest errors to catch because each individual statement may be correct. The mistake is in how they are combined.

In the next lesson — our final lesson in this module — I am going to show you how to catch these errors before they reach your patients. Practical verification techniques you can build into your daily workflow.