{"id":24910,"date":"2020-08-25T06:00:00","date_gmt":"2020-08-25T13:00:00","guid":{"rendered":"https:\/\/insidebigdata.com\/?p=24910"},"modified":"2020-08-26T09:52:08","modified_gmt":"2020-08-26T16:52:08","slug":"book-review-deep-reinforcement-learning-hands-on","status":"publish","type":"post","link":"https:\/\/insidebigdata.com\/2020\/08\/25\/book-review-deep-reinforcement-learning-hands-on\/","title":{"rendered":"Book Review: Deep Reinforcement Learning Hands-On"},"content":{"rendered":"\n
\"\"<\/figure><\/div>\n\n\n\n

Reinforcement learning (RL) is a hugely popular area of deep learning, and many data scientists are exploring this AI technology to broaden their skillet to include a number of important problem domains like chatbots, robotics, discrete optimization, web automation and much more. As a result of this wide-spread interest in RL, there are many available educational resources specifically tailored to this class of deep learning – boot camps, training certificates, educational specializations, etc. But if you’re a data scientist who has been programming in Python (with object oriented features) for a while, and has some experience with other forms of deep learning using a framework like TensorFlow, then maybe this new book, “Deep Reinforcement Learning Hands-On,” by Maxim Lapan from Packt, might be a great way to kick-start yourself into becoming productive with RL. <\/p>\n\n\n\n

Reinforcement Learning<\/strong><\/p>\n\n\n\n

RL development is being driven by a number of large companies and research groups, including Google, Microsoft, and Facebook. RL requires considerable investment in research as the field is growing to enable data scientists to be able to take prescribed methods and apply them to a problem domain. This is similar to how the states of computer vision and NLP were several years ago. A good portion of the AI research papers appearing on the arXiv.org pre-print server deal with RL. <\/p>\n\n\n\n

As a journalist, I’m seeing that the field of RL is attracting lots of attention, both from researchers and practitioners. Fortunately, this hands-on book helps readers to understand RL methods using real-life problems, and make the exciting RL field accessible to a much wider audience than just research groups or large AI companies.<\/p>\n\n\n\n

The emphasis of the book is to enable readers to feel that modern RL is an approachable, practical, and useful aspect of machine learning. The book will help readers understand RL methods using real-life, relatable problems. Information about recent research is widely available via arXiv.org and research blogs, but it is too specialized and grounded with mathematics for many data scientists to easily consume. If readers are new to this field, these papers might not be the best source of information, at least not early on. This book gets around such complexities by addressing the lack of practical and structured information about RL methods and approaches. <\/p>\n\n\n\n

New in the Second Edition<\/strong><\/p>\n\n\n\n

The second edition is devoted to the very latest RL tools and techniques, focusing on new innovations in this emerging field. It includes six new chapters that give readers the hands-on ability to code intelligent learning agents to perform a range of practical tasks. New topics include robotics and multiple agent learning. The chapter on trust regions has been updated with the new SAC algorithm. The author has also developed a library for easily interfacing the OpenAI environment which was used all through the first edition but is now properly documented.<\/p>\n\n\n\n

After a clear definition of RL and associated nomenclature, the Gym library and its classes are explained. Gym is a rich library used widely by RL practitioners and the book’s code depends on it. A straight-forward and impressive RL application using Gym is given and explained carefully.<\/p>\n\n\n\n

Fun topics include – using the TextWorld environment from Microsoft Research to solve text-based interactive fiction games, solving discrete optimization problems (showcased using Rubik\u2019s Cube), and learning to apply RL methods to the robotics domain.<\/p>\n\n\n\n

The book then dives into an important aspect of RL methods: advanced exploration. Several modern exploration techniques are described, implemented, and compared. Readers will also discover basic methods applied to a multi-agent environment.<\/p>\n\n\n\n

All examples have been updated for PyTorch 1.3. A good introduction to PyTorch is given in Chapter 3. PyTorch Ignite has also been introduced to make the RL coding more concise.<\/p>\n\n\n\n

To demonstrate the breadth of coverage of the subject, here are the chapters included in the book:<\/p>\n\n\n\n