Dynamic Programming and Optimal Control, Volume I, 4th Edition,動的計画法と最適制御, 第4版, 第1巻,9781886529434,978-1-886529-43-4
Desciption
The leading and most up-to-date textbook on the far-ranging algorithmic methododogy of Dynamic Programming, which can be used for optimal control, Markovian decision problems, planning and sequential decision making under uncertainty, and discrete/combinatorial optimization. The treatment focuses on basic unifying themes, and conceptual foundations. It illustrates the versatility, power, and generality of the method with many examples and applications from engineering, operations research, and other fields. It also addresses extensively the practical application of the methodology, possibly through the use of approximations, and provides an extensive treatment of the far-reaching methodology of Neuro-Dynamic Programming/Reinforcement Learning.
The first volume is oriented towards modeling, conceptualization, and finite-horizon problems, but also includes a substantive introduction to infinite horizon problems that is suitable for classroom use. The second volume is oriented towards mathematical analysis and computation, treats infinite horizon problems extensively, and provides an up-to-date account of approximate large-scale dynamic programming and reinforcement learning. The text contains many illustrations, worked-out examples, and exercises.
This extensive work, aside from its focus on the mainstream dynamic programming and optimal control topics, relates to our Abstract Dynamic Programming (Athena Scientific, 2013), a synthesis of classical research on the foundations of dynamic programming with modern approximate dynamic programming theory, and the new class of semicontractive models, Stochastic Optimal Control: The Discrete-Time Case (Athena Scientific, 1996), which deals with the mathematical foundations of the subject, Neuro-Dynamic Programming (Athena Scientific, 1996), which develops the fundamental theory for approximation methods in dynamic programming, and Introduction to Probability (2nd Edition, Athena Scientific, 2008), which provides the prerequisite probabilistic background.
- New features of the 4th edition of Vol. I (see the Preface for details):
・provides textbook accounts of recent original research on approximate DP, limited lookahead policies, rollout algorithms, model predictive control, Monte-Carlo tree search and the recent uses of deep neural networks in computer game programs such as Go.
・includes a substantial number of new exercises, detailed solutions of many of which are posted on the internet