Dota 2 with Large Scale Deep Reinforcement Learning

OpenAI, :, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Dębiak, Przemysław, Christy Dennison et al.

arXiv (Cornell University) 2019 preprint cited by 969

Source Open-access PDF More in Software & Coding

importance 6/10 solved Artificial Intelligence in Games reinforcement learning stage: research

Problem

No AI system could defeat world champions at Dota 2, an esports game with long time horizons, imperfect information and complex continuous state-action spaces.

Why it matters

World-champion esports performance is a major AI milestone reflecting real-world decision complexity.

What the AI did

OpenAI Five applied self-play reinforcement learning at massive scale, training for 10 months on batches of roughly 2 million frames every 2 seconds via a distributed training system.

Results

On April 13th, 2019, became the first AI system to defeat the Dota 2 world champions (Team OG).

Impact

Demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.

Record

Venue	arXiv (Cornell University)
Published	2019
Work type	preprint
Authors	OpenAI, :, Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Dębiak, Przemysław, Christy Dennison et al.
Citations	969 (OpenAlex, at import time)
References	37
Model or method	OpenAI Five (self-play reinforcement learning)
Organization	OpenAI
Primary source	https://doi.org/10.48550/arxiv.1912.06680
Open access	green
Keywords	reinforcement learning, self-play, Dota 2, esports, OpenAI Five

Importance rating

6 / 10

significant — thousands to millions affected

First AI to defeat esports world champions; abstract frames Dota 2 challenges as increasingly central to capable AI systems, a genuinely new capability milestone.

Basis: inferred. Judged from the source text, which gives no explicit scale. Stated scale: not stated in source. Method and limits.

More in Software & Coding

#	Paper	Year	Citations	Importance
1	ImageNet classification with deep convolutional neural networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton	2017	106,345	8
2	Deep Residual Learning for Image Recognition Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun	2016	221,134	8
3	Faster sorting algorithms discovered using deep reinforcement learning Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco S...	2023	175	8
4	A deep learning‐based approach for software vulnerability detection using code metrics Fazli Subhan, Xiaoxue Wu, Lili Bo, Xiaobing Sun, Muhammad Rahman	2022	42	7
5	Deep Learning-Based Attack Detection and Classification in Android Devices Alfonso Gómez, Antonio Muñoz	2023	41	7