Browsing by Author "Yuksel, S."

Now showing 1 - 2 of 2

Metadata only
Asymptotic optimality of finite model approximations for partially observed markov decision processes with discounted cost
(IEEE, 2020-01) Saldı, Naci; Yuksel, S.; Linder, T.; Natural and Mathematical Sciences; SALDI, Naci
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully observed one on the belief space, the finite models are obtained through the uniform quantization of the state and action spaces of the belief space Markov decision process (MDP). Under mild assumptions on the components of the original model, it is established that the policies obtained from these finite models are nearly optimal for the belief space MDP, and so, for the original partially observed problem. The assumptions essentially require that the belief space MDP satisfies a mild weak continuity condition. We provide an example and introduce explicit approximation procedures for the quantization of the set of probability measures on the state space of POMDP (i.e., belief space).
Metadata only
On weak feller continuity properties of non-linear filters
(IEEE, 2019) Kara, A. D.; Saldı, Naci; Yuksel, S.; Natural and Mathematical Sciences; SALDI, Naci
Weak Feller property of controlled and control free Markov chains leads to many desirable properties. In control-free setups this leads to the existence of invariant probability measures for compact spaces and applicability of numerical approximation methods. For controlled setups, this leads to existence and approximation results for optimal control policies. We know from stochastic control theory that partially observed systems can be converted to fully observed systems by replacing the original state space with a probability measure-valued state space, with the corresponding kernel acting on probability measures known as the non-linear filter process. In this paper, we prove the weak Feller property of the non-linear filter process (i) first under weak continuity of the transition probability of controlled Markov chain and total variation continuity of its observation channel, and then, (ii) under total variation continuity of the transition probability of controlled Markov chain. The former result (i) has first appeared in Feinberg et. al. (2016). Here, we present a concise and easy to follow alternative proof for this existing result. The latter result (ii) establishes weak Feller property of non-linear filter process under conditions which have not been previously reported in the literature.