When a decision comes from an algorithm, it feels neutral.
That doesn't mean it is.

Recently I saw a story about a lawsuit involving Workday, a company whose software is used by many employers to screen job applicants.

The allegation was not that one hiring manager had made a biased decision. It was that algorithm-based screening tools had discriminated against applicants based on characteristics like race, age, and disability. Workday denies the claims, and the case is still working its way through the courts. But the story stuck with me because it captures something students need to understand about AI: when a decision comes from a system, it can feel more objective than when it comes from a person. That does not mean it is. (Fisher Phillips)

That is the danger with AI bias. A human decision can be questioned as a human decision. An algorithmic decision often arrives with a different kind of authority. It feels technical, data-driven, neutral. But AI systems are trained on data from the real world, and the real world already contains unfairness. If we are not careful, AI does not remove that unfairness. It can automate it.

That is the tension this activity is built around: AI can feel objective, but that illusion can make bias harder to see, harder to challenge, and easier to scale. And it left me wondering: where might AI bias already be affecting my life without me knowing?

Why it matters

AI bias is often harder for students to see than human bias because it can arrive dressed as math. A person can explain a decision badly, show unfair assumptions, or be challenged directly. An algorithm gives a score, a ranking, a recommendation, or a rejection. That can make the decision feel neutral, even when the system has simply learned old patterns and repeated them with new authority.

That is why the Workday case is such a useful starting point. The claims are still being tested in court, and Workday denies wrongdoing, but the case raises a question students will keep encountering: what happens when an algorithm helps decide who gets seen, selected, supported, or screened out? Similar concerns have appeared in criminal justice, healthcare, hiring, facial recognition, and credit scoring. ProPublica's COMPAS analysis found that Black defendants who did not reoffend were almost twice as likely as white defendants to be misclassified as higher risk. Research on a widely used healthcare algorithm found that, at the same risk score, Black patients were considerably sicker than white patients, meaning the system could recommend less extra care for Black patients with comparable or greater medical need.

Regulators are beginning to respond to these risks in different ways. In the EU, AI systems used in employment and worker management are treated as high-risk under the AI Act, which means they face stricter rules rather than being treated as ordinary software. That matters because hiring, healthcare, education, criminal justice, and credit are not low-stakes settings. If an AI system gets a music recommendation wrong, the harm is small. If it gets a loan, job, care, or sentencing recommendation wrong, the consequences can follow a person for years. (Clifford Chance)

The classroom value is not in telling students that every AI system is unfair. That would be too simple. The value is in helping them understand why fairness is difficult to define and why "we did not include race or gender" does not automatically mean a system cannot discriminate. Zip code, school attended, healthcare spending, job history, or patterns in old decisions can all act as proxies. Once students understand that, they can ask better questions: What data was used? Who was harmed? Who benefits? Can the decision be explained? And who gets to challenge it?

Addressing it in your class

That gap is the focus of AI Bias and Fairness, a group activity for ages 14 to 18 that fits well in digital literacy, media literacy, technology, social studies, civics, ethics, or future-of-work lessons. Students explore why AI systems can produce unfair outcomes even when they appear neutral, then analyze real cases where algorithmic decisions affected people in hiring, healthcare, criminal justice, facial recognition, image generation, and credit scoring. The goal is not to make students suspicious of every AI system. It is to help them understand the right questions to ask before accepting an algorithmic decision as fair.

What the activity covers

The activity is built in three parts. In Part 1, students begin with the idea of objectivity. They discuss whether an AI system feels more reassuring than a human decision-maker, and whether removing human judgment can sometimes make a decision less fair rather than more fair. They also look at where bias comes from: historical data, proxy variables, feedback loops, and the way algorithmic decisions can appear more neutral than they really are.

In Part 2, students analyze real cases of AI bias. These include COMPAS in criminal sentencing, facial recognition error rates, Amazon's hiring tool, healthcare allocation in the US, image generation stereotypes, and AI-powered credit scoring. For each case, students identify where the bias came from, who was harmed, and what might reduce the harm. This keeps the conversation grounded in real systems rather than abstract warnings.

In Part 3, students work through fairness tensions where both sides have a serious argument. Should an AI be used if it is more accurate overall but less accurate for one group? Should systems be required to explain their decisions, even if that means using simpler models? If AI bias reflects bias in society, is the AI the problem or is society the problem? These questions are deliberately difficult, because fairness is not just a technical setting you can switch on.

The lesson also includes a teacher guide with timing, facilitation notes, differentiation ideas, and an assessment rubric. The rubric focuses on understanding AI bias, case analysis, engagement with fairness tensions, group discussion, and quality of reflection, so students are assessed on the depth of their reasoning rather than whether they land on a neat answer.

How to run it well

This lesson needs a careful tone. The cases involve race, gender, healthcare, policing, hiring, and inequality, so students may bring strong views or personal experiences into the room. Set the expectation early that the goal is not to win a debate about whether AI is good or bad. The goal is to understand how a system that looks neutral can still produce unfair outcomes.

Where groups may stall is on the idea of proxy variables. Students often understand obvious bias quickly: if a system uses race or gender directly, the problem is clear. Seeing how a system can exclude those variables and still discriminate through things that correlate with them is the trickier part — zip code, school attended, job history, or past spending on healthcare. That is usually the moment when the lesson clicks, because students realize that "we didn't use that data" is not the same as "the system can't be biased."

The best discussion often comes from the fairness tensions in Part 3. Ask students: "If two definitions of fairness conflict, who should get to decide which one matters most?" That question moves the conversation beyond "fix the bias" and into the harder civic issue underneath it. Fairness is not only a technical problem. It is a human decision about what kind of mistakes we are willing to tolerate, who carries the cost, and who gets a say.