Specialty Digest

Global English editionLearners’ Study Dashboard — in development
The Knowledge Library for All

Deep Reinforcement Learning

nounMachine Learningalso deep RL, DRL

In one line

Deep reinforcement learning combines reinforcement learning with deep neural networks so that an agent can learn good behavior directly from high-dimensional inputs such as images.

Deep reinforcement learning is a branch of artificial intelligence that combines reinforcement learning, in which an agent learns by trial and error from rewards, with deep learning, in which multilayer neural networks learn patterns from raw data. The neural network lets the agent handle complex inputs such as images, sound or sensor readings, instead of relying on small hand-built tables. The combination produced some of the most striking results in modern AI, from programs that master video games to systems that defeat champions at Go, and it now supports work in robotics and in refining language models.

Why combine reinforcement learning with deep learning?

Classic reinforcement learning stores a value for every state and action. That works for a tiny grid world but fails for a video game screen or a robot camera, where the number of possible states is astronomically large. A deep network solves this by generalizing. It learns to map a raw input, such as the pixels of a screen, to a value estimate or a choice of action, and similar inputs produce similar outputs. This means that experience in one situation helps the agent in many related ones. At the same time, reinforcement learning gives deep networks something supervised learning cannot: a way to learn from consequences when no one has labeled the correct action.

How deep reinforcement learning works

The basic loop is the same as in reinforcement learning. The agent observes a state, a neural network outputs an action or action values, the environment returns a reward and a new state, and the network’s parameters are updated to increase future reward. Training uses gradient descent, as in other deep learning, but the training targets are built from the agent’s own experience. This creates special problems. Successive observations are highly correlated, and the targets move as the network changes, which can make learning unstable. Much of the research in the field has been about taming these problems.

Main approaches

  • Deep Q-Networks (DQN): a network estimates the value of each action. DeepMind’s version, published in 2013 and in the journal Nature in 2015, stabilized training with two ideas, experience replay, which stores past experience and learns from random samples, and a separate target network that changes slowly.
  • Policy gradient methods: the network outputs the probabilities of actions directly and is adjusted to make rewarding actions more likely. Examples include REINFORCE and trust region policy optimization.
  • Actor-critic methods: one network chooses actions while another judges them. A3C, DDPG, SAC and proximal policy optimization (PPO) belong here, and PPO has become a widely used default.
  • Model-based methods: the agent learns a model of the world and plans using imagined experience, as in MuZero from DeepMind, which learned to play board games and Atari without being told the rules.

Milestones

In 2013, a small DeepMind team reported a network that learned to play Atari 2600 games from raw pixels and score alone. In 2016, AlphaGo, which combined deep networks, reinforcement learning from self-play and tree search, defeated Lee Sedol at Go, a game long considered too complex for computers. AlphaGo Zero in 2017 learned from self-play alone, with no human games, and AlphaZero extended the method to chess and shogi. In 2019, OpenAI Five defeated professional players at the video game Dota 2, and DeepMind’s AlphaStar reached a top level in StarCraft II. In 2022, researchers used deep reinforcement learning to control plasma in a fusion reactor simulator and in experiments, showing uses beyond games.

Applications of deep reinforcement learning

Robotics is a major area. Robots trained in simulation learn to walk, balance and pick up objects, and techniques that randomize the simulation help the policies work on real machines. Data center cooling, power grid management and chip design have all been explored as control problems. Recommendation and advertising systems use it to decide what to show a user when each choice influences later behavior. Autonomous driving research uses it for planning and decision making, though safety rules limit use on public roads. In language technology, reinforcement learning from human feedback uses algorithms such as PPO to tune large language models toward answers that people prefer.

Advantages and challenges

Deep reinforcement learning can learn directly from raw sensory data, discover original strategies and adapt to environments too complex to program by hand. Its challenges are substantial. It is sample inefficient, often needing millions or billions of interactions, which is practical only in simulation or with large computing budgets. Results can vary a great deal between random seeds and settings, making experiments hard to reproduce. Reward design is difficult, and agents may find shortcuts that satisfy the reward without achieving the intended goal. Trained agents can be brittle, failing when conditions change slightly, and they raise safety questions when deployed in the physical world. Researchers are addressing these problems with better exploration methods, offline learning from stored data, imitation learning from demonstrations and more careful safety constraints.

Deep reinforcement learning vs. traditional reinforcement learning

Traditional methods work with small, discrete problems and have strong theoretical guarantees, but they cannot cope with raw images or continuous control. Deep reinforcement learning scales to those problems but gives up most of the guarantees, and its success depends on engineering choices. Practitioners often start with simple methods on small problems and turn to deep networks only when the state space demands it.

Reward design and safety

Choosing the reward is often the hardest step in a deep reinforcement learning project. A reward that is too sparse, such as a single point for winning a long game, gives the agent almost nothing to learn from. A reward that is too generous or poorly specified invites reward hacking, where the agent maximizes the number without doing what the designers wanted. Researchers use techniques such as reward shaping, which adds small intermediate rewards, and inverse reinforcement learning, which infers a reward from examples of expert behavior. For high-stakes uses, teams add safety constraints, test agents in varied simulations and keep humans in the loop, because a policy that performs well on average can still fail badly in rare situations.

FAQs

Is deep reinforcement learning the same as deep learning? No. Deep learning is the use of multilayer neural networks. Deep reinforcement learning uses such networks inside a reinforcement learning agent.

What was the first major deep reinforcement learning success? DeepMind’s Deep Q-Network, which learned to play Atari games from pixels, is usually cited as the breakthrough.

Why is deep reinforcement learning hard to use in the real world? It needs very many trials, can behave unpredictably and depends on carefully designed rewards, so safe use outside simulation takes extra effort.

Key facts

  • Deep reinforcement learning uses neural networks to represent the policy, the value function or both.1
  • In 2013 and 2015, DeepMind's Deep Q-Network learned to play dozens of Atari games from raw pixels, reaching human-level performance on many.2
  • Common challenges include sample inefficiency, unstable training and reward design.3

Related concepts

Quick checkWhat did DeepMind's Deep Q-Network show in 2013 and 2015?Show answer

That a single deep neural network could learn to play many Atari games directly from screen pixels, reaching human-level scores on many of them.

Sources

  1. Mnih et al., Human-level control through deep reinforcement learning, https://www.nature.com/articles/nature14236
  2. OpenAI Spinning Up, Key Concepts in RL, https://spinningup.openai.com/en/latest/spinningup/rl_intro.html
  3. Sutton and Barto, Reinforcement Learning: An Introduction, http://incompleteideas.net/book/the-book-2nd.html

Editorially reviewed by Specialty Digest Editorial TeamLast reviewed November 17, 2026Researched and drafted with AI assistanceReport an issue