Data bias is a systematic distortion in the data used to build or evaluate a model. The data may over-represent some people or situations, miss others, record things inaccurately or carry the prejudices of the past. Because machine learning systems learn patterns from examples, biased data usually produces biased models, which is why data bias is a leading cause of algorithmic bias and a central topic in fairness in AI and AI ethics.
Why it matters
A model can only reflect what it has seen. If a dataset lacks older patients, speakers of certain dialects or rare conditions, the model may perform poorly for them even when overall accuracy looks high. Data bias can also be invisible, since average metrics hide weak spots, and it can be amplified when model outputs are used to make new data.
Common types
- Sampling bias. The sample does not represent the target population, for example survey responses gathered only online.
- Selection bias. The process that selects cases favors some of them, such as studying only people who completed a program.
- Survivorship bias. Only successes remain visible. Abraham Wald noticed that damage on returning aircraft showed where planes could be hit and still survive, so armor belonged elsewhere [1].
- Historical bias. Data accurately record a world that was unfair, such as past hiring patterns.
- Measurement bias. Instruments or proxies record some groups less accurately, such as a sensor that works poorly on dark skin.
- Label bias. Human annotators bring their own judgments or errors; see data labeling.
- Reporting and exclusion bias. Some events are recorded more often, as in policing data that reflect where officers patrol.
- Confirmation and automation bias. Collecting or trusting data in ways that support existing beliefs.
- Temporal bias. Data from one period are used in another; see concept drift.
Examples
- Benchmarks for facial analysis have been dominated by lighter-skinned faces, and systems tested on them performed worse for darker-skinned women [3].
- Speech recognition tools trained mostly on some accents have shown higher error rates for others.
- Medical studies that enrolled mainly men have limited what is known about how conditions and drugs affect women.
- Language models trained on web text reflect the viewpoints and languages most common online; see large language model.
- Crime prediction trained on arrest records may reproduce patrol patterns rather than actual offending.
Detecting data bias
- Profile the data. Check group counts, missing values and distributions against the population you care about.
- Compare with reference sources, such as census figures.
- Slice evaluation. Report accuracy and error rates by subgroup; see model evaluation.
- Audit labels. Measure agreement between annotators and review disputed items.
- Look for proxies. Test whether harmless-looking features predict sensitive traits.
- Use documentation. Datasheets describe how data were collected, why and with what limits [2].
Reducing data bias
- Collect better data. Plan sampling, seek out under-represented groups and involve community members.
- Rebalance. Oversample rare groups, reweight examples or generate additional data carefully; see data augmentation and synthetic data.
- Clean and relabel. Fix errors, clarify guidelines and use multiple annotators.
- Choose targets carefully, avoiding proxies that encode unfairness.
- Document. Publish datasheets and model cards so users know limits.
- Monitor. Compare live data to training data over time.
Techniques have costs. Synthetic data may copy the original biases, and removing sensitive columns can hide problems rather than solve them.
Data bias, privacy and law
Measuring bias often needs sensitive attributes, which privacy laws may restrict. Organizations must balance protection with the need to test fairness; see AI privacy. Regulations such as the European Union’s AI Act require high-risk systems to use training data that are relevant, representative and as free of errors as possible; see AI regulation.
A worked example
A hospital trains a model to flag skin lesions from photographs. The team collects 50,000 images from its own dermatology clinic, where most patients have lighter skin. The model scores well on a held-out test set drawn from the same clinic, but when a partner clinic with a more diverse population tries it, sensitivity drops sharply for darker skin tones. The problem was not the algorithm but the data: sampling bias in who was photographed, plus label bias because specialists were trained mainly on lighter-skin examples. The fix required new image collection, review by a wider group of clinicians, separate reporting of results by skin tone and a notice in the documentation about the limits of the original data. This pattern, in which strong overall numbers hide poor performance on groups missing from the data, is one of the most common ways data bias reaches real users.
Data bias across the data life cycle
Bias can arise before any data are collected, when someone decides what to measure and what to ignore. It continues during collection, when sensors, forms and web scraping reach some people more than others, and during labeling, when annotators interpret ambiguous cases. It can enter in preparation, for example when rows with missing values are dropped and those rows are not random, and in splitting, when similar records leak between training and test sets and inflate results; see training data and testing data. It can also arise after deployment, when a system’s choices change who appears in future data. Reviewing every stage, rather than only the final dataset, finds more of these problems.
Questions to ask about any dataset
Useful checks are simple to state. Who is included and who is missing? How were examples chosen and by whom? What was the purpose of the original collection, and is the new use similar? Who labeled the data, with what instructions? Are some groups measured less accurately? How old is the data and has the world changed? What would a harmful error look like, and who would bear it? Writing these answers into a datasheet gives later users a way to judge whether the data fit their purpose, and it creates a record that auditors and regulators can examine.
FAQs
What is data bias? It is a systematic distortion in data that makes a model less accurate or fair for some people or situations.
What are examples of data bias? Sampling bias, selection bias, survivorship bias, historical bias, measurement bias and label bias.
How is data bias different from algorithmic bias? Data bias is one source of algorithmic bias. The latter also comes from model design and deployment.
How can data bias be reduced? Through more representative collection, rebalancing, careful labeling, documentation and ongoing monitoring.
Can large datasets remove bias? Not automatically. Size does not fix skewed sampling or historical prejudice.
More in AI Ethics, Safety and Governance
Key facts
- Survivorship bias is illustrated by Abraham Wald's analysis of returning World War II aircraft, in which armor was recommended where returning planes had no damage1.
- The Datasheets for Datasets proposal (2018) recommended documenting how data were collected, for what purpose and with what limits2.
- A 2018 study of facial analysis found that benchmark datasets were heavily composed of lighter-skinned faces3.
Related concepts
Quick checkWhat is data bias?Show answer
Data bias is a systematic distortion in the data used to build or evaluate a model, such as unrepresentative sampling, skewed labels or measurement error, that causes inaccurate or unfair results for some people or situations.
Sources
- Jordan Ellenberg, How Not to Be Wrong: The Power of Mathematical Thinking (account of Abraham Wald and survivorship bias), https://www.penguinrandomhouse.com/books/312474/how-not-to-be-wrong-by-jordan-ellenberg/
- Timnit Gebru et al., Datasheets for Datasets, https://arxiv.org/abs/1803.09010
- Joy Buolamwini and Timnit Gebru, Gender Shades, https://proceedings.mlr.press/v81/buolamwini18a.html
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
