{r setup, include=FALSE} knitr::opts_chunk$set(echo = TRUE)  {exercise, name="Exponential random random variables"} Let$E_1, \ldots, E_n$be independent exponential random variables with rates$\lambda_1, \ldots, \lambda_n$Show that $$\mathbb{P} ( E_1 = \min (E_1, \ldots, E_n)) = \frac{\lambda_1}{ \lambda_1 + \cdots + \lambda_n}.$$  {exercise, name="Q matrices"} Consider the continuous-time Markov chain$X$with three state$\{1,2,3\}$with$Q$matrix given by  {r} Q <- matrix(c(-6,3,3, 2,-3,1, 2,7,-9), nrow =3) Q = t(Q) Q  * Find the stationary distribution$\pi$. * Start the Markov chain at state$i$, and write code so that you obtain the state of the chain at time$t$. * Starting at state$1$, see what state the chain is in after a large time$t$; repeat for a large number of times. * On average, how often is$X(t) =3$for large$t$? * Discuss this experiment in relation to the theory we [discussed](https://tsoo-math.github.io/ucl/continuous-timeMC.html). {exercise, name = "Stationary measures"} Let$P$be a transition matrix semigroup for an irreducible continuous-time Markov chain on a finite number of states$A$with the stationary measure$\pi$. Let$Q$be the generator. Let$M$be transition matrix for the corresponding jump chain with the corresponding stationary measure$\hat{\pi}\$. Show that $$\hat{\pi}_i = \frac{q_{ii} \pi_i}{\sum_j q_{jj} \pi_j }.$$ 

* Version: r format(Sys.time(), '%d %B %Y') * [Rmd Source](https://tsoo-math.github.io/ucl/QHW5.Rmd)