Deep learning

Deep learning is a method of machine learning in which computer models with many processing layers learn to represent data at several levels of abstraction. It has improved results in speech recognition, visual object recognition and object detection, as well as in areas such as drug discovery and genomics.

History of deep learning

Deep learning grew out of research on artificial neural networks. Juergen Schmidhuber’s 2015 survey in the journal Neural Networks notes that much of the relevant work dates from the previous millennium, and it recounts the history of backpropagation, the training method that remains central today.

A turning point came in 2012. Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton trained a large convolutional network to classify 1.3 million high-resolution images into 1,000 classes, and it beat the previous best results on the ImageNet test set. The authors reported top-1 and top-5 error rates of 39.7 percent and 18.9 percent. Their network had five convolutional layers, 60 million parameters and 500,000 neurons, and it ran on graphics processors to make training practical.

Yann LeCun, Yoshua Bengio and Hinton summarized the field in a 2015 review in Nature. The review describes that image-classification result as a breakthrough that nearly halved the error rate for object recognition and spurred rapid adoption in computer vision. It also credits a joint 2012 paper by several speech laboratories as the first major industrial application of deep learning.

By the time Schmidhuber’s survey appeared, deep artificial neural networks, including recurrent ones, had won numerous contests in pattern recognition and machine learning. His review covers supervised learning, unsupervised learning, reinforcement learning and evolutionary computation.

How deep learning works

A deep learning model is built from layers. Each layer transforms the output of the layer before it, so early layers pick up simple features and later layers combine them into more abstract ones. For an image, the first layers might respond to edges, and later layers to parts of objects.

Training relies on backpropagation. As LeCun, Bengio and Hinton explain, the algorithm shows the machine how to change the internal parameters that compute each layer’s representation from the representation in the previous layer. The model makes predictions, measures its error against known answers, and adjusts its parameters to reduce that error across many examples.

Schmidhuber separates shallow from deep learners by the depth of their credit assignment paths, which are chains of learnable links between an action and its effect. A deeper path means more stages in which the model can transform the data.

Schmidhuber’s survey also covers unsupervised learning, in which models find structure without labeled answers, and reinforcement learning, in which they learn from rewards.

Types of deep learning models

Convolutional neural networks suit grid-like data. The Nature review says they have brought breakthroughs in processing images, video, speech and audio. Recurrent networks suit sequences, and they have shone a light on text and speech.

The transformer, introduced in a 2017 paper by Ashish Vaswani and colleagues, dropped recurrence and convolutions in favor of attention mechanisms. The authors reported a score of 28.4 BLEU on the WMT 2014 English-to-German translation task, and 41.8 BLEU on English-to-French after 3.5 days of training on eight GPUs. Transformers underpin many large language models.

Generative adversarial networks, proposed by Ian Goodfellow and colleagues in 2014, train two models against each other. A generator tries to produce samples that look like the training data, while a discriminator tries to tell real samples from generated ones. The pair can be trained with backpropagation alone.

Recurrent networks process a sequence one element at a time and carry information forward, which fits language and audio.

Applications of deep learning

The Nature review lists speech recognition, visual object recognition, object detection, drug discovery and genomics among the areas where deep learning has improved the state of the art. In practice this includes computer vision tasks such as reading medical images or sorting photographs, and language tasks such as translation covered under natural language processing.

The same methods drive generative AI systems that produce text, images and audio. Voice interfaces and recommendation tools also use deep networks, although the exact models behind commercial products are rarely published in full.

The 2012 joint paper from several speech laboratories, co-written by Hinton, summarized a breakthrough in classifying phonetic units for automatic speech recognition.

Advantages and disadvantages of deep learning

The chief advantage is that the model learns useful features from raw data instead of relying on features designed by hand. This lets one general approach work across images, sound and text, and results often improve as models and datasets grow.

The costs follow from the same scale. The 2012 ImageNet network needed 1.3 million labeled training images and specialized hardware, and its authors added a new regularization method to reduce overfitting, a problem in which a model memorizes its training data instead of learning general patterns.

Interpretability is another limit. Because the representations are learned instead of written by people, it can be hard to explain why a model gave a particular answer, and a model can perform poorly on data unlike what it saw in training. These limits are an active research topic, and they matter most where errors are costly.

Deep learning vs. machine learning

Deep learning is a subset of machine learning, which is itself a branch of artificial intelligence. Many machine learning methods, such as linear models and decision trees, work well on small, structured datasets and are easier to inspect.

Deep learning usually becomes the better choice when the data is large and unstructured, such as pixels or audio waveforms, and when engineers want the model to find its own features. The two approaches are complementary, and practitioners often compare them on the same problem before choosing.

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 4, 2026Researched and drafted with AI assistanceReport an issue