Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Overfitting

nounAI Evaluation and Reliabilityalso overfit, over-fitting, high variance

In one line

Overfitting is a modeling error in which a machine learning model learns the training data too closely, including its noise, so that it performs well on training examples but poorly on new, unseen data.

Overfitting is what happens when a machine learning model learns its training data too well. Instead of capturing the underlying pattern, it memorizes quirks and random noise in the examples, and so performs excellently on data it has seen but poorly on data it has not. A student who memorizes the answers to last year’s exam may score perfectly on that paper and fail a new one that tests the same ideas. Overfitting is among the most common problems in machine learning, and detecting and preventing it is a central purpose of validation data and testing data. Its opposite is underfitting, and the goal of good modeling is generalization.

An intuition

Imagine fitting a curve through ten noisy points that really follow a straight line. A straight line misses each point slightly, but captures the trend. A wiggly curve of high degree can pass exactly through every point, giving zero training error, yet it swings wildly between points and predicts new values badly. The wiggly curve has fitted the noise, not the signal [1].

Bias, variance and model complexity

The bias-variance trade-off explains the phenomenon. Bias is error from overly simple assumptions, which causes underfitting. Variance is error from sensitivity to the particular training sample, which causes overfitting. A very flexible model has low bias but high variance: trained on a slightly different sample, it would produce a very different function [1]. As complexity increases, training error keeps falling, while error on new data first falls and then rises. The best model sits near the bottom of that second curve.

Modern deep learning complicates the classical picture. Very large networks can fit their training data perfectly and still generalize well, and test error can even fall again as size grows, a pattern called double descent. Overfitting remains a practical concern, but model size alone does not determine it.

Causes

  • Too little data relative to model flexibility.
  • Too complex a model, with many parameters or features.
  • Noisy or mislabeled data, which the model memorizes; see data labeling.
  • Training too long, so the model starts to fit noise.
  • Irrelevant features, which give the model spurious clues.
  • Data leakage, which makes validation look better than reality; see testing data.
  • Duplicates across training and evaluation sets.
  • Overuse of the validation set, by trying hundreds of settings until one looks good by chance.

How to detect it

  • Gap between training and validation scores. High training accuracy with much lower validation accuracy is the classic sign.
  • Learning curves. Training loss keeps falling while validation loss flattens or rises.
  • Cross-validation. Large variation in scores across folds; see cross-validation.
  • Poor performance after deployment, often from data drift combined with fragile fits.
  • Suspiciously perfect results on small data sets.

How to prevent or reduce it

### More and better data

Collecting more training data, cleaning errors and using data augmentation are the most reliable cures.

### Simpler models

Reduce the number of layers, parameters or features, or choose a simpler algorithm. Feature selection and dimensionality reduction cut irrelevant inputs.

### Regularization

Penalties on model complexity discourage extreme solutions. L2 regularization (weight decay) shrinks weights, and L1 pushes some to zero, giving sparse models. See regularization.

### Early stopping

Stop training when validation error stops improving, and keep the best saved version [2].

### Dropout and noise

Dropout randomly switches off neurons during training, so the network cannot rely on any single path. Adding noise to inputs or weights has a similar effect.

### Ensembles

Averaging many models, as in random forests and bagging, reduces variance.

### Pruning and constraints

Trim decision trees, limit tree depth, or prune networks; see model compression.

### Transfer learning

Starting from a pretrained model lets a small data set be enough; see transfer learning.

### Cross-validation and proper splits

Use them to choose settings honestly, keeping a final untouched test set.

Overfitting in large language models

Language models can memorize passages of their training text, which raises privacy and copyright concerns, and they can overfit when fine-tuned on small data sets, losing general abilities. Teams monitor validation loss during fine-tuning, use low learning rates and small adapters such as LoRA, and deduplicate training text.

Overfitting to benchmarks

A related problem occurs at the community level: when many researchers tune models against the same public test set, the field can collectively overfit it, producing gains that do not carry over to fresh data; see AI benchmark.

A practical diagnosis workflow

When a model disappoints on new data, a methodical approach separates overfitting from other causes. First, plot training and validation loss against training time. If both are high and close, the model is underfitting. If training loss is low and validation loss is much higher, the model is overfitting. Second, plot a learning curve, training on increasing fractions of the data. If validation performance is still rising with more data, collecting more examples will help. If it has flattened while training performance stays far above it, the model has too much capacity for the signal available. Third, check for leaks and duplicates, since an apparent good result that collapses in production often has such a cause. Fourth, read errors on validation examples to see whether the model relies on spurious cues, such as a hospital’s watermark on X-rays that correlates with disease in the training set.

Examples of overfitting in the wild

  • Medical imaging. A model trained on scans from one hospital learned to recognize that hospital’s equipment markings, which happened to correlate with disease prevalence, and failed at other sites.
  • Finance. Trading strategies tuned to past price data, a practice called backtest overfitting, often fail on live markets because the pattern was chance.
  • Image recognition. Classifiers have learned to identify wolves by snow in the background instead of the animal.
  • Competitions. Entrants who tune endlessly against a public leaderboard sometimes drop sharply when scored on the hidden final data.

Balancing the cure

The remedies should not be applied blindly, because too much regularization or too simple a model leads to underfitting. The aim is to find the point where validation error is lowest, checking it on testing data at the end. A reliable habit is to start with a simple baseline, add complexity step by step, and stop when validation performance stops improving.

FAQs

What is overfitting? It is when a model learns the training data, including its noise, so well that it performs poorly on new data.

How do you know if a model is overfitting? Training performance is much better than validation performance, and validation loss rises while training loss falls.

How can you prevent overfitting? Use more data, simpler models, regularization, dropout, early stopping, ensembles and cross-validation.

What is the difference between overfitting and underfitting? Overfitting means a model is too closely tuned to training data. Underfitting means it is too simple to capture the pattern at all.

Can deep learning models overfit? Yes, especially with limited data, though very large models sometimes generalize well despite fitting training data perfectly.

Key facts

  • An overfitted model has low error on training data and high error on new data, so the gap between training and validation performance grows1.
  • The bias-variance trade-off describes how very flexible models have low bias but high variance, and overfit noise in the data1.
  • Regularization, early stopping, dropout and more training data are standard defenses against overfitting2.

Related concepts

Quick checkWhat is overfitting?Show answer

Overfitting is a modeling error in which a machine learning model learns the training data too closely, including its noise and quirks, so that it performs well on the training examples but poorly on new, unseen data.

Sources

  1. Trevor Hastie, Robert Tibshirani and Jerome Friedman, The Elements of Statistical Learning, Springer, https://hastie.su.domains/ElemStatLearn/
  2. Ian Goodfellow, Yoshua Bengio and Aaron Courville, Deep Learning, MIT Press, https://www.deeplearningbook.org/

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue