Plausible isn't the same as true.
That's the most useful thing students can learn about AI.

A man was trying to book a flight after his grandmother died.

He asked Air Canada's chatbot about bereavement fares, and the chatbot told him he could book the flight first, then apply for the reduced bereavement fare afterward. That sounded like official guidance. It was on Air Canada's own website, written clearly, and specific enough to act on. So he followed it.

The problem was that the chatbot was wrong. Air Canada's actual policy did not allow the discount to be claimed that way, and the airline later argued that the correct information was available elsewhere on its website. The British Columbia Civil Resolution Tribunal disagreed and held Air Canada responsible for the misleading information its chatbot had provided. (McCarthy)

That story stayed with me because it shows a very ordinary version of AI failure. The chatbot did not need to be dramatic or obviously broken. It just needed to sound helpful, confident, and official at a moment when someone had a real reason to trust it.

AI feels magical until students understand that it is not thinking in the way people think. It is generating plausible responses from patterns, and plausible is not the same as true. That is the core problem this post is about: students do not need a technical explanation of AI, but they do need to understand it well enough to question what it gives them.

Why it matters

AI mistakes are different from ordinary mistakes because they often arrive in a very polished package. A student may know to be suspicious of a messy website, a random forum post, or a badly written answer. But AI output can look calm, complete, and professional even when it is wrong. That is what makes it so easy to over-trust. The Air Canada chatbot did not look confused. It gave a clear answer, and the clarity was part of the problem.

This is where students need a basic mental model of how AI works. Not a technical explanation with equations or engineering detail, but enough to understand that a generative AI system is not checking truth in the same way a person might. It is producing likely responses based on patterns in data. That can be incredibly useful for drafting, explaining, summarizing, and brainstorming, but it also means AI can invent details, miss context, or sound certain when it should not.

For teachers, the point is not to tell students that AI is unreliable and should be avoided. That would be too simple, and students would not believe it anyway. A better target is helping students build the habit of asking: What kind of task is this? What would I need to check? What might the AI be missing? Once students understand AI as a powerful pattern tool rather than a thinking authority, they are in a much better position to use it well.

Addressing it in your class

That habit is the aim of How AI Works, a group activity for ages 14 to 18 that fits well in AI literacy, digital literacy, media literacy, technology, advisory, or study skills lessons. Students do not need to become computer scientists to use AI critically. They do need a clear, practical understanding of what generative AI is doing, why it can produce useful answers, and why it can also get things wrong with confidence. The goal is to replace the "AI is magic" feeling with a more useful habit: trust it where it is strong, question it where it is weak, and check it when the stakes matter.

What the activity covers

The activity is built in three parts. In Part 1, students test an AI tool directly by asking a specific factual question they can verify. They then check the answer against a reliable source and try a second question about a less well-known person, place, or event. The point is not to catch AI out for fun. It is to help students notice the gap between confidence and accuracy.

In Part 2, students look at why AI gets things wrong. They work through examples of hallucination, including a fake book, fabricated legal citations, incorrect details about a minor public figure, outdated information about recent events, and errors in a less common language. For each one, they discuss what teachers and students would need to check before trusting that kind of output.

In Part 3, students apply that understanding to common AI tasks. They compare where AI tends to do well, such as summarizing, drafting, brainstorming, and explaining concepts, with where it falls short, such as fact-checking, niche topics, recent events, and multi-step reasoning. The point is to help students build a practical verification habit: use AI as support, but keep responsibility for accuracy, judgment, and final decisions.

The lesson also includes a teacher guide, student worksheet, discussion prompts, differentiation ideas, extension options, and an assessment rubric. The rubric focuses on understanding how AI works, hallucination analysis, strengths and limitations, group discussion, and quality of reflection, so students are assessed on how clearly they understand the tool rather than whether they memorize technical terms.

How to run it well

The main thing is to keep the explanation simple enough to be useful. Students do not need a lecture on model architecture, training runs, or tokens. They need a working idea they can remember: AI is very good at producing plausible responses, but plausible does not always mean true, complete, fair, or appropriate for the task.

Where groups may stall is in separating "AI got this wrong" from "AI is useless." Push them away from that all-or-nothing thinking. A calculator can be used badly. A search engine can return poor results. A person can give confident but wrong advice. AI is another tool that needs judgment around it. "Can AI be trusted?" is the wrong question. Ask instead: "What would I need to verify before I use this?"

The best discussion often comes from asking students to compare two tasks: one where AI is genuinely helpful and one where it could mislead them. For example, using AI to brainstorm revision questions is low risk. Using it to explain a medical symptom, cite sources for an essay, or summarize a topic they do not understand well is different. That is where students start to see the central habit: the less you know about the topic, and the higher the stakes, the more carefully you need to check the output.