5.3 Computing Bias

What you need to know

• Computing bias occurs when a computing innovation systematically produces unfair or skewed outcomes for certain groups.
• Bias can enter through the data: if training data underrepresents a group, the system performs worse for that group (e.g., facial recognition trained mostly on one demographic).
• Bias can enter through design choices: which features are included, what the defaults are, what's considered "normal" — reflecting the assumptions of the developers.
• Bias can be intentional or unintentional. Most exam scenarios are unintentional — a well-meaning team that didn't test with diverse users.
• Bias may exist at every level of development, from problem selection to data collection to testing.
• Programmers should take action to reduce bias: audit data for representativeness, test with diverse users, involve diverse teams (this is why 1.1 emphasizes diverse collaboration), and examine outcomes by group.
• Bias in a system used at scale — hiring, lending, policing — can amplify existing inequality dramatically.

Worked example

A company builds a résumé-screening tool by training it on a decade of past hiring decisions. Those past decisions favored one gender for engineering roles. The tool learns the pattern and down-ranks résumés with signals associated with the other gender. Nobody at the company intended that — the bias was in the historical data. Detecting it required checking the tool's outcomes by group, and fixing it required different data, not just different code.

Going deeper

The nuance, edge cases, and connections that turn a 3 into a 5.

• Computing bias is when a system produces systematically unfair results for certain groups. The CED's key claim: bias in a computing innovation reflects the biases in its data, its design, and its developers — and it can be unintentional.
• Bias from data: systems trained on historical data learn historical patterns, including unfair ones. A hiring tool trained on past hires learns past discrimination. A facial recognition system trained mostly on one demographic works worse on others. The data doesn't have to be labeled "biased" — it just has to be unrepresentative.
• Bias from design: defaults, categories, and assumptions encode the designers' worldview. A form with only two gender options, a voice assistant tested only with one accent, a health app that assumes a particular body type — each excludes people the designers didn't picture.
• Bias from developers: teams that lack diversity have blind spots. This is why 1.1 emphasizes diverse collaboration — it's a direct countermeasure to bias.
• Bias can exist at every level: which problem to solve (whose needs matter?), what data to collect, how to clean it, what to test, who to test with, and how to interpret results. It's not one step you can check off.
• Amplification at scale: a biased human decision affects one person. A biased algorithm used for millions of decisions affects millions — consistently and invisibly. That's why algorithmic bias is treated as more dangerous than individual bias.
• Reducing bias is an active responsibility: audit data for representation, test outcomes across groups, involve affected communities, build diverse teams, and be willing to not ship something that can't be made fair. The CED says programmers "should take action to reduce bias."
• Bias is often invisible from inside: developers who share the blind spot don't see the problem. External testing and diverse perspectives are how it gets found.

Mistakes that cost points

• Thinking bias requires intent. Most computing bias is unintentional. "The developers didn't mean to" doesn't make a system unbiased.
• Believing a better algorithm alone fixes biased data. If the data is unrepresentative, the algorithm learns from unrepresentative data. The data must be fixed.
• Assuming computers are objective. "An algorithm can't be biased because it's math" is a common belief and a wrong answer. Algorithms encode human choices.
• Missing the connection to collaboration. "Why would a diverse team reduce bias?" — because they catch each other's blind spots. This links 5.3 back to 1.1.

Practice questions

Written in the style of the real exam. Try each one before revealing the answer.