Ergodic Theory for Controlled Markov Chains with Stationary Inputs - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue The Annals of Applied Probability Année : 2018

Ergodic Theory for Controlled Markov Chains with Stationary Inputs

Résumé

Consider a stochastic process X on a finite state space X = {1,. .. , d}. It is conditionally Markov, given a real-valued 'input process' ζ. This is assumed to be small, which is modeled through the scaling, ζ t = εζ 1 t , 0 ≤ ε ≤ 1 , where ζ 1 is a bounded stationary process. The following conclusions are obtained, subject to smoothness assumptions on the controlled transition matrix and a mixing condition on ζ: (i) A stationary version of the process is constructed, that is coupled with a stationary version of the Markov chain X • obtained with ζ ≡ 0. The triple (X, X • , ζ) is a jointly stationary process satisfying P{X(t) = X • (t)} = O(ε) Moreover, a second-order Taylor-series approximation is obtained: P{X(t) = i} = P{X • (t) = i} + ε 2 π (2) (i) + o(ε 2), 1 ≤ i ≤ d, with an explicit formula for the vector π (2) ∈ R d. (ii) For any m ≥ 1 and any function f : {1,. .. , d} × R → R m , the stationary stochastic process Y (t) = f (X(t), ζ(t)) has a power spectral density S f that admits a second order Taylor series expansion: A function S (2) f : [−π, π] → C m×m is constructed such that S f (θ) = S • f (θ) + ε 2 S (2) f (θ) + o(ε 2), θ ∈ [−π, π] in which the first term is the power spectral density obtained with ε = 0. An explicit formula for the function S (2) f is obtained, based in part on the bounds in (i). The results are illustrated with two general examples: mean field games, and a version of the timing channel of Anantharam and Verdu.
Fichier principal
Vignette du fichier
AAP1606-015R2A0.pdf (2.54 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01672476 , version 1 (25-12-2017)

Identifiants

  • HAL Id : hal-01672476 , version 1

Citer

Yue Chen, Ana Bušić, Sean Meyn. Ergodic Theory for Controlled Markov Chains with Stationary Inputs. The Annals of Applied Probability, 2018, 28 (1), pp.79-111. ⟨hal-01672476⟩
235 Consultations
137 Téléchargements

Partager

Gmail Facebook X LinkedIn More