Abstract Dynamic Programming, 2nd Edition, 2018
by Dimitri P.
The 2nd edition of the research monograph "Abstract Dynamic Programming," has now appeared and is available in hardcover from the publishing company, Athena Scientific, or from Amazon.com.
The 2nd edition aims primarily to amplify the presentation of the semicontractive models of Chapter 3 and Chapter 4 of the first (2013) edition, and to supplement it with a broad spectrum of research results that I obtained and published in journals and reports since the first edition was written (see below). As a result, the size of this material more than doubled, and the size of the book increased by nearly 40%.
The book can be downloaded and used freely for noncommercial purposes. The version below corrects a few errata from the book's first printing, and is identical to the book's second printing (to appear in 2021).
Abstract Dynamic Programming, 2ND EDITION, Complete
In addition to the changes in Chapters 3, and 4, I have also eliminated from the second edition the material of the first edition that deals with restricted policies and Borel space models (Chapter 5 and Appendix C). These models are motivated in part by the complex measurability questions that arise in mathematically rigorous theories of stochastic optimal control involving continuous probability spaces. The restricted policies framework aims primarily to extend abstract DP ideas to Borel space models. Since this material is fully covered in Chapter 6 of the 1978 monograph by Bertsekas and Shreve, and followup research on the subject has been limited, I decided to omit Chapter 5 and Appendix C of the first edition from the second edition and just post them below.
Chapter 5 of 1st Edition
Appendix C of 1st Edition
Supporting series of five videolectures on Semicontractive Dynamic Programming
The following papers and reports have a strong connection to the book, and among others amplify on the analysis and the range of applications of the semicontractive models of Chapters 3 and 4:
D. P. Bertsekas, "Regular Policies in Abstract Dynamic Programming", Lab. for Information and Decision
Systems Report LIDS-P-3173, MIT, May 2015 (revised August, 2016); to appear in SIAM J. on Control and Optimization (Related Lecture Slides); (Related Video Lectures).
D. P. Bertsekas, "Value and Policy Iteration in Deterministic Optimal Control and Adaptive Dynamic Programming", Lab. for Information and Decision
Systems Report LIDS-P-3174, MIT, May 2015 (revised Sept. 2015); IEEE Transactions on Neural Networks and Learning Systems, Vol. 28, 2017, pp. 500-509.
D. P. Bertsekas and H. Yu, "Stochastic Shortest Path Problems Under Weak Conditions", Lab. for Information and Decision
Systems Report LIDS-P-2909, MIT, January 2016.
D. P. Bertsekas, "Robust Shortest Path Planning and Semicontractive Dynamic Programming", Lab. for Information and Decision
Systems Report LIDS-P-2915, MIT, Jan. 2016; revised in June 2016; to appear in Naval Research Logistics.
D. P. Bertsekas, "Affine Monotonic and Risk-Sensitive Models in Dynamic Programming", Lab. for Information and Decision
Systems Report LIDS-3204, MIT, June 2016; to appear in IEEE Transactions on Aut. Control.
D. P. Bertsekas, "Stable Optimal Control and Semicontractive Dynamic Programming", Lab. for Information and Decision
Systems Report LIDS-P-3506, MIT, May 2017; to appear in SIAM J. on Control and Optimization (Related Lecture Slides).
D. P. Bertsekas, "Proper Policies in Infinite-State Stochastic Shortest Path Problems," arXiv preprint arXiv:1711.10129. (Related Lecture Slides).
Bertsekas, D., "Multiagent Value Iteration Algorithms in Dynamic Programming and Reinforcement Learning," ASU Report, April 2020, arXiv preprint, arXiv:2005.01627.
An updated version of Chapter 4 of the author's Dynamic Programming book, Vol. II, which incorporates recent research on a variety of undiscounted problems and relates to abstract DP topics;
(Related Lecture Slides).