Training data is the collection of examples that a machine learning model learns from. During training, the model processes these examples and adjusts its internal parameters so that its outputs match patterns in the data. The quality, size, variety and fairness of training data strongly shape what a model can do and where it fails, which is why practitioners often say that a model is only as good as its data. Training data is distinct from validation data, used to tune a model, and testing data, used to measure final performance. Together they are drawn from a dataset.
Overview
A machine learning model starts with no knowledge of a task. In supervised learning, each training example pairs an input, such as a photograph, with the correct output, called a label, such as “cat.” The learning algorithm, usually based on gradient descent and backpropagation, measures how far the model’s predictions are from the labels and nudges the parameters to reduce the error. After many passes through the data, the model captures regularities that let it handle new inputs it has not seen.
Not all learning needs labels. Unsupervised learning finds structure in unlabeled data, and self-supervised learning creates its own training signal from the data, as when a language model learns to predict the next word in a text. Reinforcement learning learns from rewards gathered through interaction instead of from a fixed set of examples. Most large language models are pretrained on huge text collections, then fine-tuned on smaller, curated data; see LLM training.
Types of training data
- Labeled data. Examples with correct answers, created by people or by automated processes. Used in classification and regression.
- Unlabeled data. Raw examples without labels, such as web text or images. Often plentiful and cheap.
- Structured data. Tables of records with defined columns, such as customer transactions.
- Unstructured data. Text, images, audio and video.
- Synthetic data. Artificially generated examples, produced by simulation or by generative models, used to cover rare cases or protect privacy; see synthetic data.
- Human feedback data. Rankings or ratings of model outputs, used in instruction tuning and alignment.
- Interaction data. Logs of user behavior used in recommendation and personalization.
Data quality
Quality matters at least as much as quantity. Common quality problems include:
- Label errors. Wrong or inconsistent labels teach the model the wrong thing. Even benchmark data sets have measurable error rates.
- Noise and outliers. Corrupted values or irrelevant examples reduce accuracy.
- Duplicates. Repeated examples can bias the model and, if they also appear in test data, inflate scores.
- Missing values. Gaps need handling through removal or imputation.
- Irrelevant or low-quality content. For text, spam, machine-generated text and boilerplate lower quality.
- Staleness. Data that no longer reflects the world makes models outdated.
Cleaning and curating data is often the largest part of a machine learning project. Filtering, deduplication, balancing and careful labeling guidelines usually improve results more than small changes to a model. See also feature engineering.
How much training data is needed?
There is no fixed number. Simple models on easy problems can learn from hundreds of examples, while large neural networks use millions or billions. Research on language models found that performance improves predictably as data, model size and compute grow together, a finding known as scaling laws [3]. The ImageNet data set that powered the 2012 deep learning breakthrough had about 1.2 million labeled training images across 1,000 categories [2]. When data is scarce, practitioners use transfer learning, few-shot learning, data augmentation, active learning and synthetic data to get more from less. More data does not always help: if it is noisy or unrepresentative, it may add little or cause harm.
Bias, representativeness and fairness
A model learns the patterns in its training data, including unwanted ones. If certain groups, languages, skin tones or situations are underrepresented, the model may perform worse for them. If historical decisions in the data reflect discrimination, the model may reproduce it. Examples include facial analysis systems with higher error rates for some groups and hiring tools that learned to favor patterns from past, male-dominated hiring. Responsible practice includes examining who and what the data represents, measuring performance across subgroups, documenting the data’s origin and limits, and correcting imbalances. Frameworks such as datasheets for datasets encourage this kind of documentation.
Privacy, consent and copyright
Training data often includes personal information and creative works. Legal and ethical issues include consent from the people whose data is used, compliance with privacy laws such as the General Data Protection Regulation, the risk that models memorize and reveal personal data, and copyright in books, images, code and music. Courts and legislators in several countries are working through how copyright law applies to training, and the answers differ by jurisdiction and are changing. Techniques such as anonymization, differential privacy and federated learning reduce exposure of personal data. Organizations should track the licenses and provenance of their data.
Overfitting and data splitting
A model that memorizes its training data performs poorly on new data, a problem called overfitting. To detect it, data is divided into separate parts. The model is trained on the training portion, tuned on the validation portion and judged on the test portion, which it must never see during development. Common splits allocate most data, such as 70 to 80 percent, to training. Keeping these portions properly separate, including avoiding leakage of information from test data into training, is vital for honest evaluation.
Data-centric AI
The approach called data-centric AI argues that improving data, with better labels, consistent definitions and coverage of important cases, is often a more effective route to better models than changing algorithms. Practices include error analysis to find which kinds of examples the model fails on, adding targeted data for those cases, and versioning data as carefully as code.
FAQs
What is training data? It is the set of examples that a machine learning model learns from by adjusting its parameters.
What is the difference between training data and test data? Training data is used to teach the model. Test data is held back and used only to measure how well the finished model performs on unseen examples.
How much training data do I need? It depends on the task and model. Simple problems may need hundreds of examples, large neural networks need millions or more, and techniques like transfer learning reduce the need.
What is labeled data? It is data in which each example includes the correct answer, such as an image tagged with what it shows.
Why does bias in training data matter? Models learn patterns in their data, so unrepresentative or prejudiced data can lead to unfair or inaccurate results for some groups.
More in AI Data and Infrastructure
Key facts
- In supervised learning, training examples consist of inputs paired with correct outputs, called labels1.
- The ImageNet data set used for the 2012 breakthrough in image classification contained about 1.2 million labeled training images in 1,000 categories2.
- Research on scaling laws for language models found that performance improves predictably as model size, training data and compute increase together3.
Related concepts
Quick checkWhat is training data?Show answer
Training data is the set of examples, such as images with labels or passages of text, that a machine learning model learns from; the model adjusts its internal parameters to fit patterns in this data so that it can make predictions on new inputs.
Sources
- Stuart Russell and Peter Norvig, Artificial Intelligence: A Modern Approach, Pearson, https://aima.cs.berkeley.edu/
- Olga Russakovsky et al., ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision, https://arxiv.org/abs/1409.0575
- Jared Kaplan et al., Scaling Laws for Neural Language Models, https://arxiv.org/abs/2001.08361
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
