Stochastic Optimal Control: The Discrete-Time Case
This book was originally published by Academic Press in 1978, and republished by Athena Scientific in 1996 in
paperback form. It can be purchased from Athena Scientific or
it can be freely downloaded in scanned form (330 pages, about 20
Megs).
The book is a comprehensive and theoretically sound treatment of the mathematical foundations of
stochastic optimal control of discrete-time systems, including the treatment of the intricate measure-theoretic
issues.
Review :
"Bertsekas and Shreve have written a fine book. The exposition is extremely clear
and a helpful introductory chapter provides orientation and a guide to the rather intimidating mass of literature
on the subject. Apart from anything else, the book serves as an excellent introduction to the arcane world of
analytic sets and other lesser known byways of measure theory."
Mark H. A. Davis, Imperial College, in IEEE
Trans. on Automatic Control "
Stochastic Optimal Control: The Discrete-Time Case
Introduction
- Structure of Sequential Decision Problems
- Discrete-Time Optimal Control Problems - Measurability Questions
- The Present Work Related to the Literature
Monotone Mappings Underlying Dynamic Programming
Models
- Notation and Assumptions
- Main Results
- Application to Specific Models
- Deterministic Optimal Control
- Stochastic Optimal Control - Countable Disturbance Space
- Stochastic Optimal Control - Outer Integral Formulation
- Stochastic Optimal Control - Multiplicable Cost Functional
- Minimax Control
Finite Horizon Models
- General Results and Assumptions
- Main Results
- Application to Specific Models
Infinite Horizon Models under a Contraction Assumption
- General Results and Assumptions
- Convergence and Existence Results
- Computational Methods
- Successive Approximation
- Policy Iteration
- Mathematical Programming
- Application to Specific Models
Infinite Horizon Models under Monotonicity Assumptions
- General Results and Assumptions
- The Optimality Equation
- Characterization of Optimal Policies
- Convergence of the Dynamic Programming Algorithm - Existence of Stationary Policies
- Application to Specific Models
A Generalized Abstract Dynamic Programming Model
- General Results and Assumptions
- Analysis of Finite Horizon Models
- Analysis of Infinite Horizon Models under a Contraction Assumption
Borel Spaces and their Probability Measures
- Notation
- Metrizable Spaces
- Borel Spaces
- Probability Measures on Borel Spaces
- Characterization of Probability Measures
- The Weak Topology
- Stochastic Kernels
- Integration
- Semicontinuous Functions and Borel-Measurable Selection
- Analytic Sets
- Equivalent Definitions of Analytic Sets
- Measurability Properties of Analytic Sets
- An Analytic Set of Probability Measures
- Lower Semianalytic Functions and Universally Measurable Selection
The Finite Horizon Borel Model
- The Model
- The Dynamic Programming Algorithm - Existence of Optimal and epsilon-Optimal Policies
- The Semincontinuous Models
The Infinite Horizon Borel Models
- The Stochastic Model
- The Deterministic Model
- Relations Between the Models
- The Optimality Equation - Characterization of Optimal Policies
- Convergence of the Dynamic Programming Algorithm - Existence of Stationary Optimal Policies
- Existence of epsilon-Optimal Policies
The Imperfect State Information Model
- Reduction of the Nonstationary Model - State Augmentation
- Reduction of the Imperfect State Information Model - Sufficient Statistics
- Existence of Sufficient Statistics for Control
- Filtering and the Conditional Distribution of the States
- The Identity Mappings
Miscellaneous
- Limit-Measurable Policies
- Analytically Measurable Policies
- Models with Multiplicative Cost
Appendix A: The Outer Integral
Appendix B: Additional Measurability Properties of Borel Spaces
Appendix C: The Hausdorff Metric and the Exponential Topology
References
Index