Chapter 4: E4.8 Solution

Working problems is a crucial part of learning mathematics. No one can learn topology merely by poring over the definitions, theorems, and examples that are worked out in the text. One must work part of it out for oneself. To provide that opportunity is the purpose of the exercises.

James R. Munkres

$^{*}$ Blackwell’s test of imagination

This exercise assumes that you are familiar with continuous-parameter Markov chains with two states.

For each $n\in\mathbb{N}$ , let $X^{(n)}=\{X^{(n)}(t):t\ge O\}$ be a Markov chain with state-space the two-point set $\{0,1\}$ with Q-matrix $\begin{align*} Q^{(n)}=\begin{pmatrix}-a_{n} & a_{n}\\ b_{n} & -b_{n} \end{pmatrix}, & a_{n},b_{n}>0,\\ \end{align*}$ and transition function $P^{(n)}(t)=\exp(tQ^{(n)})$ . Show that, for every $t$ , $\begin{align*} p_{00}^{(n)}(t)\ge b_{n}/(a_{n}+b_{n}), & p_{01}^{(n)}(t)\le a_{n}/(a_{n}+b_{n}). \end{align*}$ The processes $(X^{(n)}:n\in\mathbb{N})$ are independent and $X^{(n)}(0)=0$ for every $n$ . Each $X^{(n)}$ has right-continuous paths.

Suppose that $\sum a_{n}=\infty$ and $\sum a_{n}/b_{n}<\infty$ .

Prove that if $t$ is a fixed time then $\mathbb{P}\{X^{(n)}(t)=1\text{ for infinitely many }n\}=0.\tag{∗}$ Use Weierstrass’s M-test to show that $\sum_{n}\log p_{00}^{(n)}(t)$ is uniformly convergent on $[0,1]$ , and deduce that $\mathbb{P}\{X^{(n)}(t)=0\text{ for ALL }n\}\to1\text{ \, as }t\downarrow0.$ Prove that $\mathbb{P}\{X^{(n)}(s)=0,\forall s\le t,\forall n\}=0\text{ for every }t>0$ and discuss with your tutor why it is almost surely true that $\begin{align*} & \text{within every non-empty time interval, infinitely many of the }X^{(n)}\tag{∗∗}\\ & \text{chains jump.} \end{align*}$ Now imagine the whole behaviour.

Notes. Almost surely, the process $X=(X^{(n)})$ spends almost all its time in the countable subset of $\{0,1\}^{\mathbb{N}}$ consisting of sequences with only finitely many 1’s. This follows from (*) and Fubini’s Theorem 8.2. However, it is a.s. true that $X$ visits uncountable points of $\{0,1\}^{\mathbb{N}}$ during every non-empty time interval. This follows from (**) and the Baire category theorem A1.12. By using much deeper techniques, one can show that for certain choices of $(a_{n})$ and $(b_{n})$ , $X$ will almost certainly visit every point of $\{0,1\}^{\mathbb{N}}$ uncountably often within a finite time.

By either solving the forward equation or using eigenvalues together with $p_{00}^{(n)}(0)=1$ and $\tfrac{d}{dt}p_{00}^{(n)}(0)=-a_{n}$ , we obtain $\begin{align*} p_{00}^{(n)}(t) & =\tfrac{b_{n}}{a_{n}+b_{n}}+\tfrac{a_{n}}{a_{n}+b_{n}}e^{-(a_{n}+b_{n})t},\\ p_{01}^{(n)}(t) & =\tfrac{a_{n}}{a_{n}+b_{n}}-\tfrac{a_{n}}{a_{n}+b_{n}}e^{-(a_{n}+b_{n})t}, \end{align*}$ and the inequalities follow. Then, by BC1 and the second assumption, we have $(\ast)$ .

Further, $\begin{align*} |\log p_{00}^{(n)}(t)| & =-\log p_{00}^{(n)}(t)\\ & \le\tfrac{1}{p_{00}^{(n)}(t)}-1\\ & \le\tfrac{a_{n}}{b_{n}}, \end{align*}$ and, by Weierstrass’s M-test, $\sum_{n}\log p_{00}^{(n)}(t)$ converges uniformly to a continuous function. Therefore, $\begin{align*} \mathbb{P}\{X^{(n)}(t)=0\text{ for ALL }n\} & =\exp\sum_{n}\log p_{00}^{(n)}(t)\\ & \to1 \end{align*}$ as $t\downarrow0$ .

Note, that so far we have used the second assumption only, as for the statements above all we need is that the rate of going back to $0$ grows fast enough as $n\to\infty$ . However, now we need to prove a statement that is true if the rate of staying at $0$ is not too small by itself as $n\to\infty$ , so we will use the first assumption. We have,

Similarly, since for some $N\in\mathbb{N}$ , for all $n\ge N$ , $a_{n}\le b_{n}$ , $\begin{align*} \mathbb{P}\{X^{(n)}(s)=X^{(n)}(t_{1}),\forall t_{1}\le s\le t_{2},\forall n\ge M\} & =\\ \prod_{n\ge M}\mathbb{P}\{X^{(n)}(s)=X^{(n)}(t_{1}),\forall t_{1}\le s\le t_{2}\} & \le\\ \prod_{n\ge\max\{M,N\}}\exp(-a_{n}t) & =\\ 0. \end{align*}$ Hence, statement $(\ast\ast)$ .

Subpages

Chapter 4: E4.8 Solution

Chapter 4: E4.8 Solution