Cynobi AI Academy
Course 1 · AI Foundations · Lesson 05

Hallucination

The most dangerous moment in AI isn't when it fails — it's when it answers beautifully, confidently, and wrong. Why that happens, where it hides, and the four habits that catch it.

Free · Course 1 ~6 min video Type · Misconception

The one mental model

Confident is not correct. The model blends everything it has read by pattern — by how often something was repeated and how smoothly it was phrased — not by truth. When false sources carry enough weight, they taint the answer, and the false claim comes out woven in so smoothly you can't see the seam. Its certainty tells you nothing about whether it's right.

The true story that opens this lesson

In 2023, two New York lawyers filed a federal court brief citing six supporting cases — all supplied by ChatGPT, all completely invented. The judge fined them $5,000 and the story made headlines worldwide. The detail to remember: before filing, one of them asked ChatGPT if the cases were real. It said yes.

Key terms

Hallucination
The model stating false information with total confidence. Not lying, not a glitch — a by-product of how it generates text.
Same-trick principle
Fluency and fabrication come from the same mechanism: predicting the next likely token. The machinery can't feel the difference between landing on fact and landing on fiction.
Hallucination hotspots
Specifics: names, dates, numbers, statistics, quotes, links, citations. Precise slots a sentence demands be filled — exactly where invention leaks in.
Grounding
The fix (Lesson 08): give the model real documents to read so it can point at where the answer came from, instead of generating from memory alone.

The misconception to drop

✕
“It sounds sure, so it's probably right — and if it were wrong, it would warn me or say it doesn't know.”
✓
The confidence needle never moves. A right answer and a wrong answer arrive in the same tone, same fluency, same certainty. And “I don't know” is rare because the model has no inner list of facts to check itself against — there's a blank, and the pattern demands it be filled.

Put it to work — the four habits

1
Treat every specific — every name, number, date, quote and link — as unverified until you've checked it.
2
Ask for sources — then actually open them. A real link takes ten seconds to confirm.
3
Use it where it shines — drafts, structure, explanations, ideas — and slow down exactly where it's weakest: precise facts.
4
Scale your checking to the stakes. A movie recommendation? Let it slide. A legal brief? You know how that ends.

Ask the AI Tutor

Pause the video and ask anything from this lesson — the tutor answers from this lesson’s material.

Why does AI make things up? Is the model lying to me? Why doesn't it just say “I don't know”? Which parts of an answer should I double-check?