Competition-level code generation with AlphaCode

Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling et al.

Science 2022 article cited by 1,010

Source Open-access PDF More in Software & Coding

importance 5/10 partially solved Software Engineering Research software development stage: research

Problem

Competitive programming problems require problem-solving skills beyond existing code-generation models; the paper builds a system that generates programs solving such problems.

Why it matters

Systems that generate programs could make programming more productive and accessible.

What the AI did

AlphaCode generates millions of diverse candidate programs using specially trained transformer-based networks, then filters and clusters them down to a maximum of 10 submissions.

Results

Average ranking in the top 54.3% in simulated evaluations on recent Codeforces competitions; first AI system to perform competitively in programming competitions.

Impact

Marks the first time an AI system performed competitively in programming competitions, though problems are not consistently solved.

Record

Venue	Science
Published	2022
Work type	article
Authors	Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling et al.
Citations	1,010 (OpenAlex, at import time)
References	39
Model or method	AlphaCode (transformer-based networks)
Organization	not stated in source
Primary source	https://doi.org/10.1126/science.abq1158
Open access	green
Keywords	AlphaCode, code generation, competitive programming, transformers, Codeforces

Importance rating

5 / 10

real but narrower advance

Abstract states program generation could make programming more productive and accessible and marks the first AI to perform competitively in competitions; a genuinely new but still narrow capability.

Basis: stated. The source itself states the scale. Stated scale: not stated in source. Method and limits.

More in Software & Coding

#	Paper	Year	Citations	Importance
1	ImageNet classification with deep convolutional neural networks Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton	2017	106,345	8
2	Deep Residual Learning for Image Recognition Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun	2016	221,134	8
3	Faster sorting algorithms discovered using deep reinforcement learning Daniel J. Mankowitz, Andrea Michi, Anton Zhernov, Marco Gelmi, Marco S...	2023	175	8
4	A deep learning‐based approach for software vulnerability detection using code metrics Fazli Subhan, Xiaoxue Wu, Lili Bo, Xiaobing Sun, Muhammad Rahman	2022	42	7
5	Deep Learning-Based Attack Detection and Classification in Android Devices Alfonso Gómez, Antonio Muñoz	2023	41	7