Deep reinforcement learning: Difference between revisions

Revision as of 06:52, 1 June 2024

Revision as of 17:58, 17 June 2024

Line 7:

=== Deep learning ===

[[File:Neural_network_example.svg|thumb|241x241px|Depiction of a basic artificial neural network]]

[[Deep learning]] is a form of [[machine learning]] that utilizes a neural network to transform a set of inputs into a set of outputs via an [[artificial neural network]]. Deep learning methods, often using [[supervised learning]] with labeled datasets, have been shown to solve tasks that involve handling complex, high-dimensional raw input data (such as images) with less manual [[feature engineering]] than prior methods, enabling significant progress in several fields including [[computer vision]] and [[natural language processing]]. In the past decade, deep RL has achieved remarkable results on a range of problems, from single and multiplayer games such as [[Go (game)|GOGo]], [[Atari Games]], and [[Dota 2|''Dota 2'']] to robotics.<ref>{{Cite web |last=Graesser |first=Laura |title=Foundations of Deep Reinforcement Learning: Theory and Practice in Python |url=https://openlibrary.telkomuniversity.ac.id/home/catalog/id/198650/slug/foundations-of-deep-reinforcement-learning-theory-and-practice-in-python.html |access-date=2023-07-01 |website=Open Library Telkom University}}</ref>

=== Reinforcement learning ===

Line 24:

Katsunari Shibata's group showed that various functions emerge in this framework,<ref name="Shibata3"/><ref name="Shibata4"/><ref name="Shibata2"/> including image recognition, color constancy, sensor motion (active recognition), hand-eye coordination and hand reaching movement, explanation of brain activities, knowledge transfer, memory,<ref name="Shibata5"/> selective attention, prediction, and exploration.<ref name="Shibata4"/><ref name="Shibata6"/>

Starting around 2012, the so -called [[Deep learning#Deep learning revolution | Deepdeep learning revolution]] led to an increased interest in using deep neural networks as function approximators across a variety of domains. This led to a renewed interest in researchers using deep neural networks to learn the policy, value, and/or Q functions present in existing reinforcement learning algorithms.

Beginning around 2013, [[DeepMind]] showed impressive learning results using deep RL to play [[Atari]] video games.<ref name="DQN1"/><ref name="DQN2"/> The computer player a neural network trained using a deep RL algorithm, a deep version of [[Q-learning]] they termed deep Q-networks (DQN), with the game score as the reward. They used a deep [[convolutional neural network]] to process 4 frames RGB pixels (84x84) as inputs. All 49 games were learned using the same network architecture and with minimal prior knowledge, outperforming competing methods on almost all the games and performing at a level comparable or superior to a professional human game tester.<ref name="DQN2" />

Deep reinforcement learning reached another milestone in 2015 when [[AlphaGo]],<ref name="AlphaGo"/> a computer program trained with deep RL to play [[Go (game)|Go]], became the first computer Go program to beat a human professional Go player without handicap on a full-sized 19×19 board.

In a subsequent project in 2017, [[AlphaZero]] improved performance on Go while also demonstrating they could use the same algorithm to learn to play [[chess]] and [[shogi]] at a level competitive or superior to existing computer programs for those games, and again improved in 2019 with [[MuZero]].<ref name="muzero"/> Separately, another milestone was achieved by researchers from [[Carnegie Mellon University]] in 2019 developing [[Pluribus (poker bot)|Pluribus]], a computer program to play [[poker]] that was the first to beat professionals at multiplayer games of no-limit [[Texas hold 'em]]. [[OpenAI Five]], a program for playing five-on-five [[Dota 2|''Dota 2'']] beat the previous world champions in a demonstration match in 2019.

Deep reinforcement learning has also been applied to many domains beyond games. In robotics, it has been used to let robots perform simple household tasks<ref name="levine2016"/> and solve a Rubik's cube with a robot hand.<ref name="openaihand"/><ref name="openaihandarxiv"/> Deep RL has also found sustainability applications, used to reduce energy consumption at data centers.<ref name="deepmindcooling"/> Deep RL for [[autonomous driving]] is an active area of research in academia and industry.<ref name="neurips2021ml4ad"/> [[Loon_LLC|Loon]] explored deep RL for autonomously navigating their high-altitude balloons.<ref name="loonrl"/>