![]() To play the game, we introduce AlphaDev, a learning agent that is trained to search for correct and efficient algorithms. A single incorrect instruction in the AssemblyGame can potentially invalidate the entire algorithm, making exploration in this space of games incredibly challenging. The hardness of the AssemblyGame arises not only from the size of the search space, which is similar to extremely challenging games such as chess (10 120 games) 31 and Go (10 700 games) 32, but also from the nature of the reward function. This is challenging as the player needs to consider the combinatorial space of assembly instructions to yield an algorithm that is both provably correct and fast. In this game, the player selects a series of low-level CPU instructions, which we refer to as assembly instructions 30, to combine to yield a new and efficient sorting algorithm. We formulate the problem of discovering new, efficient sorting algorithms as a single-player game that we refer to as AssemblyGame. Fixed sort algorithms sort sequences of a fixed length (for example, sort 3 can only sort sequences of length 3), whereas variable sort algorithms can sort a sequence of varying size (for example, variable sort 5 can sort sequences ranging from one to five elements). In this work, we focus on two types of small sort algorithm: (1) the fixed sort and (2) the variable sort. A key component of practical solutions is a small sort over a short sequence of elements this algorithm is called repeatedly when sorting large arrays that use divide-and-conquer approaches 29. Decades of computer science research have focused on discovering and optimizing sorting algorithms 26, 27, 28. This is taught in elementary computer science classes around the world 21, 22 and is used ubiquitously by a vast range of applications 23, 24, 25. One of the fundamental questions in computer science is how to sort a sequence 17, 18, 19, 20. Using deep reinforcement learning (DRL), we can take this a step further by generating correct and performant algorithms by optimizing for actual measured latency at the CPU instruction level, by more efficiently searching and considering the space of correct and fast programs compared to previous work. These include enumerative search techniques 4, 5, 6, 7 and stochastic search 5, 6, 8, 9, 10 as well as the more recent trend of using deep learning in program synthesis for generating correct programs 11, 12, 13, 14, 15, 16. The work in classical program synthesis literature, spanning many decades, aims to generate correct programs and/or optimize programs using proxies for latency. However, many algorithms have reached a stage whereby human experts have not been able to optimize them further, leading to an ever-growing computational bottleneck. Human intuition and know-how have been crucial in improving algorithms. We also present results in extra domains, showcasing the generality of the approach. This change to this part of the sort library represents the replacement of a component with an algorithm that has been automatically discovered using reinforcement learning. These algorithms have been integrated into the LLVM standard C++ sort library 3. AlphaDev discovered small sorting algorithms from scratch that outperformed previously known human benchmarks. ![]() ![]() We then trained a new deep reinforcement learning agent, AlphaDev, to play this game. ![]() To realize this, we formulated the task of finding a better sorting routine as a single-player game. Here we show how artificial intelligence can go beyond the current state of the art by discovering hitherto unknown routines. Whereas remarkable progress has been achieved in the past 2, making further improvements on the efficiency of these routines has proved challenging for both human scientists and computational approaches. As demand for computation grows, it has become critical for these algorithms to be as performant as possible. Fundamental algorithms such as sorting or hashing are used trillions of times on any given day 1. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |