Approximations for constrained Markov decision problems
Type :
Book chapter
Publication Status :
Published
Access :
restrictedAccess
Abstract
This chapter studies the finite-state approximation of a discrete-time constrained Markov decision process with compact state space, under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted problem, we prove the convergence of the optimal value function of the finite-state model to the optimal value function of the original model. Under further continuity conditions on the transition probability of the original discounted model, we also establish a method to compute approximately optimal policies. For the average cost criterion, instead of using the finite-state linear programming approximation method, we use a direct method to establish analogous results under drift and minorization conditions which guarantee the geometric ergodicity of Markov chains induced by stationary policies.
Source :
Finite Approximations in Discrete-Time Stochastic Control, Part of the Systems & Control: Foundations & Applications book series (SCFA)
Date :
2018
Publisher :
Springer
Collections
Share this page