Model collapse is a degenerative process in which generative models trained repeatedly on data produced by earlier models gradually lose diversity and accuracy. Each generation learns from an imperfect copy of the one before, and the errors accumulate until the model forgets rare patterns in the original data and produces bland, repetitive or distorted output. The concern grew as the internet filled with text and images made by large language models, some of which may end up in the training sets of future models.
Origin of the concept
The idea that learning from generated data can harm a model was explored in several papers in 2023, with names such as the curse of recursion and model autophagy disorder. A widely cited study by Ilia Shumailov and colleagues, published in Nature in 2024, gave a clear demonstration with language models, variational autoencoders and Gaussian mixture models. The authors trained each generation on text sampled from the previous one and observed that quality dropped over successive rounds.
How model collapse happens
Generative models approximate the distribution of their training data. Any approximation has error, and sampling favors common patterns over rare ones. When the next model trains on those samples, rare cases become even rarer, and the model’s estimate narrows.
- Statistical approximation error: a finite sample misses rare events, which then vanish from later generations.
- Functional approximation error: the model cannot represent the true distribution perfectly and introduces biases.
- Sampling bias: common settings such as low temperature produce typical outputs, which reduces variety further.
Researchers describe an early stage in which the tails of the distribution disappear, often unnoticed because overall performance can look stable, and a late stage in which the model’s outputs converge to a narrow, low-variance range that may bear little resemblance to the original data.
Evidence and limits
Experiments show collapse most clearly when each new model is trained only on the previous model’s output and the real data are discarded. Later studies reached a more nuanced conclusion: when synthetic data accumulate alongside the original real data, rather than replace it, errors stay bounded and collapse is avoided. Practical systems also mix, filter and curate data. Whether collapse threatens the largest commercial models is therefore debated, and it depends on the share of synthetic data, the filtering applied and the quality of the generator.
Why it matters
- Loss of diversity: minority viewpoints, dialects, rare facts and unusual styles may be underrepresented.
- Quality decline: models could become less accurate and more generic.
- Data scarcity: as the supply of fresh human-written text is consumed, developers turn to synthetic data, which increases the risk if done carelessly.
- Value of human data: authentic data created before the spread of generative tools, and ongoing human contributions, may become more valuable.
Prevention and mitigation
- Keep real data: retain and keep adding verified human-created data to each training run.
- Accumulate, not replace: add synthetic data to the pool instead of swapping out the original corpus.
- Filter and verify: use quality filters, deduplication and checks of correctness, especially for mathematics and code where answers can be verified.
- Provenance and watermarking: label or detect machine-generated content so that it can be tracked or excluded.
- Diverse generation: sample from varied prompts and settings to preserve variety.
- Distillation with care: model distillation deliberately trains on a larger model’s output and works well when designed and tested carefully.
Related ideas
Model collapse is related to overfitting, to catastrophic forgetting, in which a model loses earlier skills when trained on new data, and to feedback loops in recommender systems that narrow what users see. It also connects to concerns about hallucination, because errors repeated across generations can harden into apparent fact.
A worked example
Imagine a language model that learns from a corpus where one percent of sentences describe an unusual dialect. The first model reproduces the dialect a little less often than the corpus does. A second model trained on the first model’s text sees even fewer examples and reproduces it still less. After several generations the dialect has almost vanished from the output, even though no one removed it on purpose. If the original corpus had been kept in every training set, the dialect would have stayed represented.
Model collapse beyond text
The same dynamics have been studied in image generators, where repeated training on generated pictures can narrow the range of faces, textures and styles and amplify artifacts. In recommender systems and search, content produced to match what algorithms favor can create loops in which popular patterns grow and unusual ones fade. In scientific modeling, simulations used to create training data can pass their own biases downstream if never checked against measurements.
Detecting the early signs
Because early collapse can hide behind stable average scores, teams watch for specific signals: falling diversity of outputs measured by metrics such as the number of distinct n-grams, shrinking coverage of rare categories, rising repetition, drift in style and worse results on held-out tests that stress unusual cases. Keeping a fixed, human-written evaluation set that is never used for training provides a stable reference for noticing change.
Policy and data ecosystem
The topic has also prompted discussion of how to preserve access to trustworthy human-created data, including archives of pre-generative-AI content, licensing agreements with publishers and standards for labeling machine-generated media. Some researchers argue that interaction data from real users and domain experts will become a key resource for future training, while others expect that carefully verified synthetic data will fill much of the gap.
Open research questions
Several questions remain unsettled. Researchers are still working out how much synthetic data a model can safely absorb, how the answer changes with model size, and whether collapse in small experiments predicts what happens at the scale of frontier systems. They also study whether some kinds of generated content, such as verified solutions to mathematics problems, are more benign than others, and how to measure the loss of rare knowledge before it affects users. Progress on these questions will shape how future training data pipelines are designed.
FAQs
Is model collapse already happening? Collapse has been shown in controlled experiments. Whether it affects major products is debated, since developers mix data and filter it.
Can synthetic data be used safely? Yes, when it is mixed with real data, filtered and verified, and when it is used for purposes such as training smaller models or creating checked examples.
How is it different from overfitting? Overfitting is memorizing a particular training set, while collapse is the progressive loss of diversity across generations of models.
More in Large Language Models
Key facts
- A 2024 paper in Nature by Shumailov and colleagues showed that language models trained recursively on generated data lose the tails of the original distribution and degrade.1
- The researchers described early collapse as loss of information about rare events and late collapse as outputs that converge to a narrow, low-variance distribution.1
- Later studies found that collapse can be avoided when synthetic data is accumulated alongside real data rather than replacing it.2
Related concepts
Quick checkWhat is model collapse?Show answer
It is the gradual degradation of generative models trained repeatedly on the outputs of earlier models, in which rare patterns disappear and outputs become less diverse and less accurate.
Sources
- Shumailov et al., AI models collapse when trained on recursively generated data, Nature, https://www.nature.com/articles/s41586-024-07566-y
- Gerstgrasser et al., Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data, https://arxiv.org/abs/2404.01413
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
