Convergence of Markovian Stochastic Approximation with discontinuous dynamics

الأصل · EN

This paper is devoted to the convergence analysis of stochastic approximation algorithms of the form θ_n+1 = θ_n + γ_n+1 H_θ_n(X_n+1) where {θ_nn, n ≥ 0} is a Rᵈ-valued sequence, {γ, n ≥ 0} is a deterministic step-size sequence and {X_n, n ≥ 0} is a controlled Markov chain. We study the convergence under weak assumptions on smoothness-in-θ of the function θ H_θ(x). It is usually assumed that this function is continuous for any x; in this work, we relax this condition. Our results are illustrated by considering stochastic approximation algorithms for (adaptive) quantile estimation and a penalized version of the vector quantization.