Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Transfer Learning

nounMachine Learningalso knowledge transfer in machine learning, pretrained model reuse

In one line

Transfer learning is a machine learning technique in which a model trained on one task is reused as the starting point for a different but related task.

Transfer learning is a machine learning technique in which a model trained on one task is reused as the starting point for a different but related task. Instead of training a new model from scratch, which needs large amounts of data and computing time, a practitioner takes a model that has already learned useful patterns and adapts it. A network that learned to recognize thousands of everyday objects, for example, can be adapted to recognize types of plant disease from a few hundred photos. Transfer learning is now standard practice in computer vision and natural language processing and is the basis of most modern large language models.

The idea behind transfer learning

People transfer knowledge all the time. Someone who plays the piano learns the organ faster, and someone who knows Spanish finds Italian easier. Transfer learning applies the same principle to machines. The early layers of a deep network tend to learn general features. In an image model, they detect edges, colors and textures. Later layers learn more specific features, such as the shape of a wheel or a face. General features are useful across many tasks, so they can be reused, while the specialized layers are replaced or retrained for the new job. This is why a model trained on a broad dataset gives a head start even when the new task looks quite different.

How transfer learning works

The process usually has four steps.

  1. Choose a pretrained model. The source model has been trained on a large dataset, such as ImageNet for images or a large body of text for language.
  2. Reuse its layers. Most of the network is kept, and the final output layer is removed because it was designed for the original task.
  3. Add a new head. A small set of new layers matched to the target task, such as a classifier with the right number of categories, is attached.
  4. Train on the target data. The new head is trained on the smaller dataset, and the pretrained layers are either held fixed or adjusted gently.

The result is a model that usually reaches good accuracy faster and with far less labeled data than training from scratch.

Feature extraction vs. fine-tuning

There are two main strategies. In feature extraction, the pretrained layers are frozen, meaning their weights do not change, and the network acts as a fixed tool that turns each input into a rich set of features. Only the new head learns. This is quick, cheap and works well when the target dataset is small or very similar to the source. In fine-tuning, some or all of the pretrained layers continue to train, usually with a small learning rate so that useful knowledge is not destroyed. Fine-tuning can reach higher accuracy when there is more data or when the target task differs from the source, but it takes more computing and carries a greater risk of overfitting. A common compromise is to unfreeze only the last few layers.

Types of transfer learning

Researchers classify transfer learning by how the source and target relate.

  • Inductive transfer: the tasks differ, such as moving from general image classification to medical image classification, and some labeled target data is available.
  • Transductive transfer or domain adaptation: the task is the same but the data distribution differs, for example a sentiment model trained on movie reviews and applied to product reviews.
  • Unsupervised transfer: neither domain has labels, and the shared structure is used for tasks such as clustering.
  • Multitask and zero-shot variants: a model trained on many tasks, or on instructions, performs a new task with no or very few examples.

Applications of transfer learning

In computer vision, models such as ResNet and EfficientNet pretrained on ImageNet are fine-tuned for medical scans, satellite images, manufacturing inspection and wildlife monitoring. In natural language processing, models such as BERT and the GPT family are pretrained on huge text collections and then adapted to tasks such as question answering, translation, summarization and sentiment analysis. In speech, pretrained audio models cut the amount of transcribed speech needed. Transfer learning also helps in robotics, where a policy learned in simulation is transferred to a real machine, and in low-resource settings, such as languages with little written data, where knowledge learned from large languages is carried over.

Advantages of transfer learning

The approach cuts the amount of labeled data needed, which is often the main cost. It speeds up training, since the model begins from a useful state instead of random values. It usually improves accuracy, especially on small datasets, and it makes advanced AI accessible to teams without huge computing budgets. It also reduces energy use compared with training many large models from scratch, because one pretrained model can serve many projects.

Limitations and risks

Transfer does not always help. If the source and target tasks are too different, the pretrained features can be irrelevant or even harmful, a problem called negative transfer. Fine-tuning can cause catastrophic forgetting, in which the model loses useful earlier knowledge. Pretrained models also pass on their weaknesses. Biases in the source data, security flaws and copyright or privacy issues can carry over into every model built on top. Practitioners therefore check the source model’s documentation and license, test for bias and verify performance on realistic data from the target setting.

Transfer learning vs. training from scratch

Training from scratch can be best when there is a very large, specific dataset and the task is unlike anything in public models. Otherwise, transfer learning is usually the better first choice because it is faster, cheaper and more reliable. In practice, teams begin with a pretrained model, set a baseline with feature extraction, then try fine-tuning to see whether it improves results enough to justify the cost.

FAQs

What is an example of transfer learning? Taking an image model trained on ImageNet and retraining its final layers to identify skin conditions from a small set of medical photos is a typical example.

Is fine-tuning the same as transfer learning? Fine-tuning is one way of doing transfer learning, in which the pretrained model’s weights are further trained on the new task.

Why does transfer learning need less data? The pretrained model already knows general patterns, so the new task only has to learn the differences.

Key facts

  • Transfer learning reuses knowledge from a source task to improve learning on a target task, often with less data.1
  • Two common strategies are feature extraction, which keeps pretrained layers frozen, and fine-tuning, which continues training them.2
  • Models pretrained on ImageNet became a standard starting point for image tasks, and pretrained language models play the same role for text.3

Related concepts

Quick checkWhat is the difference between feature extraction and fine-tuning in transfer learning?Show answer

In feature extraction the pretrained layers stay frozen and serve as a fixed feature generator, while in fine-tuning some or all of those layers keep training on the new task.

Sources

  1. Pan and Yang, A Survey on Transfer Learning, https://ieeexplore.ieee.org/document/5288526
  2. TensorFlow, Transfer learning and fine-tuning, https://www.tensorflow.org/tutorials/images/transfer_learning
  3. Google, Machine Learning Glossary, https://developers.google.com/machine-learning/glossary

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue