A foundation model is a large artificial intelligence model trained on a very broad collection of data, usually with self-supervised learning, that can be adapted to many different tasks. Instead of building a separate model for each job, developers start from one general model and adapt it by prompting, fine-tuning or adding small components. The term was proposed in 2021 by researchers at Stanford University to describe a shift in AI in which a few very large models serve as the base, or foundation, for countless applications. Well-known examples include large language models such as GPT, image-text models such as CLIP and image generators such as Stable Diffusion.
History of foundation models
The idea builds on transfer learning, in which knowledge gained on one task helps with another. In computer vision, models pretrained on ImageNet became the standard starting point in the 2010s. In language, word embeddings and then contextual models such as ELMo and BERT in 2018 showed that pretraining on raw text produced representations useful for many tasks. The transformer architecture of 2017 allowed training at much larger scale, and GPT-2 in 2019 and GPT-3 in 2020 showed that a single language model could perform many tasks from instructions or a few examples. In August 2021, the Stanford Center for Research on Foundation Models published a report of over 100 authors that named the category and discussed its opportunities and risks. Since then, models such as PaLM, LLaMA, Claude, Gemini, DALL-E, Whisper and many others have extended the idea to text, images, audio, video, code, proteins and robotics.
Key characteristics
- Scale: foundation models have billions or trillions of parameters and are trained on huge datasets with large amounts of computing power.
- Self-supervised pretraining: they learn from the structure of unlabeled data, for example by predicting the next word or filling in a masked part of an image.
- Generality: one model handles many tasks, often without task-specific training.
- Emergence: new abilities, such as following instructions or solving simple reasoning problems, appear as scale increases and are not explicitly programmed.
- Homogenization: many applications rely on the same few base models, so their strengths and flaws are shared widely.
How foundation models are built and adapted
Building one involves several stages. First, developers collect and filter large datasets, such as web text, books, code, images with captions or audio. Second, they pretrain the model with a self-supervised objective, which can take weeks or months on thousands of accelerators. See pretraining. Third, they adapt the model to be useful and safe, using instruction tuning, reinforcement learning from human feedback and safety training. Users and developers then adapt it further through prompting, fine-tuning on domain data, retrieval of external documents or adding tools and agents.
Types of foundation models
Language models generate and analyze text and code. Vision models and vision-language models, such as CLIP, link images and text. Multimodal models handle several types of input and output at once. Generative image, video and audio models are foundation models for media, see diffusion model. Scientific foundation models predict protein structure, weather or molecular properties, and robotics foundation models learn general control policies from many demonstrations.
Applications
Foundation models support chat assistants, search and question answering, coding assistants, document summarization, translation, image and video creation, speech recognition and synthesis, drug discovery, medical documentation, customer service and educational tutoring. Companies build products on top of models supplied by providers, often through an API, or run open-weight models on their own systems.
Benefits
They lower the cost of building AI products, because a developer can start from a strong base and need little data for each task. They give strong performance across many tasks, accessible interfaces through natural language and rapid progress, as improvements to the base model flow to all applications built on it.
Risks and challenges
- Hallucination and errors: models can produce plausible but false statements.
- Bias and fairness: biases in training data can be reproduced and spread to many downstream uses.
- Concentration of power: the cost of training limits who can build frontier models, which raises concerns about competition and control.
- Misuse: capabilities can aid fraud, disinformation, cyberattacks and other harms.
- Privacy and copyright: training on web data raises questions about personal information and the rights of creators.
- Environmental cost: training and serving large models uses significant energy.
- Single point of failure: a flaw in a widely used base model affects every application built on it.
Governance
Governments and industry have started to regulate and standardize practice. The European Union AI Act includes obligations for providers of general-purpose AI models, including documentation and, for models with systemic risk, evaluations and incident reporting. In the United States, voluntary commitments and executive actions have addressed testing and safety, and international efforts, such as the Bletchley and Seoul summits, have discussed shared approaches. Companies publish model cards and system cards that describe capabilities, limits and safety testing, and independent groups run benchmarks and red-team exercises.
Open and closed models
Some developers release model weights openly, allowing anyone to download, study and adapt them, while others provide access only through an interface. Open models support research, customization and local use, while closed models allow providers to monitor use and update safeguards. The balance between openness and safety is an active debate.
A worked example
A hospital wants a tool to summarize clinic notes. Instead of training a model from scratch, it starts with a general language model. It tests prompts on a sample of de-identified notes, adds retrieval from approved clinical guidelines and fine-tunes a smaller model on its own data to cut cost. Clinicians review each summary, and the hospital tracks errors and bias, so the system stays a support tool rather than a decision-maker.
Evaluating foundation models
Because a foundation model can do so many things, evaluating it is harder than testing a single-purpose system. Developers use broad benchmarks of knowledge, reasoning, coding, mathematics and language understanding, along with safety tests that probe for harmful output, bias and leaking of private data. Human evaluators compare answers, and red teams try to make the model fail. Scores on public benchmarks can be misleading when test items leak into training data, so careful evaluators also build new, private tests and measure performance on the exact task a product needs.
FAQs
Is every large language model a foundation model? Most are, since they are trained broadly and adapted to many tasks, but the term also covers image, audio and multimodal models.
Why are they called foundation models? Because many applications are built on top of them, like a building on a foundation, and flaws in the base can affect everything above.
Can small companies build foundation models? Frontier models are very expensive, but smaller organizations can adapt open models for their needs.
More in Generative AI
Key facts
- The term foundation model was introduced in August 2021 by researchers at the Stanford Center for Research on Foundation Models.1
- Foundation models are trained with self-supervision on broad data and then adapted, for example by prompting or fine-tuning.1
- Examples include large language models such as GPT, image-text models such as CLIP and image generators such as Stable Diffusion.2
Related concepts
Quick checkWhat makes a model a foundation model?Show answer
It is trained at scale on broad, mostly unlabeled data, and can be adapted to many different tasks, so one model serves as the base for many applications.
Sources
- Bommasani et al., On the Opportunities and Risks of Foundation Models, https://arxiv.org/abs/2108.07258
- Stanford CRFM, Foundation Models, https://crfm.stanford.edu/
Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue
