Return times

Extending what we learned

Let \(X\) be Markov chain on a finite state space \(S\) with an irreducible and aperiodic transition matrix \(P\) so that there is an unique stationary distribution \(\pi\) on \(S\).

Let \[S_n = \sum_{k=0} ^{n-1}\mathbf{1}[X_{k+1} = t, X_k=s]\]

• Prove that \(S_n/n \to \pi_s p_{st}\) using a return time argument.

• Demonstrate your return time argument by simulations.

Endnotes

Version: 11 November 2024