Chapter 4: Independence

4.1. Definitions of independence. 4.2. The $\pi$ -system Lemma; and the more familiar definitions. 4.3. Second Borel-Cantelli Lemma (BC2). 4.4. Example. 4.5. A fundamental question for modelling. 4.6. A coin-tossing model with applications. 4.7. Notation: lID RV s. 4.8. Stochastic processes; Markov chains. 4.9. Monkey typing Shakespeare. 4.10. Definition. Tail $\sigma$ -algebras. 4.11. Theorem. Kolmogorov’s 0-1 law. 4.12. Exercise/Warning.

Independence

$\mathcal{F}_{1},\mathcal{F}_{2},\ldots\subset\mathcal{F}$ are independent sub- $\sigma$ -algebras if for any $n\in\mathbb{N}$ , $1\le i_{1}<\ldots<i_{n}$ , and $F_{i_{k}}\in\mathcal{F}_{i_{k}}$ , $k\in\{1,\ldots,n\}$ , $\mathbb{P}(\cap_{k=1}^{n}F_{i_{k}})=\prod_{k=1}^{n}\mathbb{P}(F_{i_{k}}).$

$X_{1},X_{2},\ldots$ are independent random variables if $\sigma(X_{1}),\sigma(X_{2}),\ldots$ are independent.

$E_{1},E_{2},\ldots$ are independent events if $\sigma(E_{1}),\sigma(E_{2}),\ldots$ are independent (alternatively, $I_{E_{1}},I_{E_{2}},\ldots$ are independent).

$\pi$ -systems $\{\mathcal{I}_{k}\}_{k=1}^{n}$ are independent iff $\{\sigma(\mathcal{I}_{k})\}_{k=1}^{n}$ are independent.

Note, that for $n>2$ , it is not enough to require that for $I_{k}\in\mathcal{I}_{k}$ , $k\in\{1,\ldots,n\}$ , $\mathbb{P}(\cap_{k=1}^{n}I_{k})=\prod_{k=1}^{n}\mathbb{P}(I_{k})$ unless each $\Omega\in\cap_{k=1}^{n}\mathcal{I}_{k}$ .
However, random variables $X_{1},\ldots,X_{n}$ are independent iff $\mathbb{P}(X_{k}\le x_{k}|k\in\{1,\ldots,n\})=\prod_{k=1}^{n}\mathbb{P}(X_{k}\le x_{k}).$

Sequences of independent random variables

Given a sequence of distribution functions $\{F_{n}\}_{n\in\mathbb{N}}$ , we can construct a sequence of independent random variables $Y_{n}(\omega)$ where $\omega\in([0,1],\mathcal{B}[0,1],\mbox{Leb})$ such that $Y_{n}\sim F_{n}$ for all $n\in\mathbb{N}$ .

Stochastic processes and Markov chains

A stochastic process is a collection of random variables $Y=(Y_{t})_{t\in T}$ on $(\Omega,\mathcal{F},\mathbb{P})$ . The map $t\to Y_{t}(\omega)$ is called the sample path of $Y$ corresponding to the sample point $\omega$ .

A (time-homogeneous) Markov chain $Z=(Z_{n}:n\ge0)$ on a finite or countable set $E$ with initial distribution $\mu=(\mu_{i})_{i\in E}$ and 1-step transition matrix $P=(p_{ij})_{i,j\in E}$ is a stochastic process such that $\mathbb{P}(Z_{0}=i_{0},\ldots,Z_{n}=i_{n})=\mu_{i_{0}}\prod_{k=1}^{n}p_{i_{k-1}i_{k}}$ .

Tail -algebras

The tail $\sigma$ -algebra of a sequence $X_{1},X_{2},\ldots$ of random variables is $\begin{eqnarray*} \mathcal{T} & = & \cap_{n\in\mathbb{N}}\mathcal{T}_{n}\\ & = & \cap_{n\in\mathbb{N}}\sigma(X_{n+1},X_{n+2},\ldots). \end{eqnarray*}$

Given a $\sigma$ -algebra $\mathcal{Y}$ and the decreasing sequence $\mathcal{T}_{n}$ , in general, $\cap_{n\in\mathbb{N}}\sigma(\mathcal{Y},\mathcal{T}_{n})\neq\sigma(\mathcal{Y},\cap_{n\in\mathbb{N}}\mathcal{T}_{n})$ .
- For example, take i.i.d. $Y_{n}=\begin{cases} 1 & \mbox{w.p.}\tfrac{1}{2}\\ -1 & \mbox{w.p.}\tfrac{1}{2} \end{cases}$ for $n\ge0$ , $\mathcal{Y}=\sigma(Y_{1},Y_{2},\ldots)$ , and $X_{n}=Y_{0}\ldots Y_{n}$ . Then, $Y_{0}\in m(\cap_{n\in\mathbb{N}}\sigma(\mathcal{Y},\mathcal{T}_{n}))$ , and $Y_{0}$ is independent of $\sigma(\mathcal{Y},\cap_{n\in\mathbb{N}}\mathcal{T}_{n})$ .

(Kolmogorov’s 0-1 Law) The tail $\sigma$ -algebra $\mathcal{T}$ of a sequence of independent random variables is $\mathbb{P}$ -trivial, i.e. if $T\in\mathcal{T}$ , then $\mathbb{P}(T)\in\{0,1\}$ , and if $\xi$ is $\mathcal{T}$ -measurable, then for some $x\in[-\infty,\infty]$ , $\mathbb{P}(\xi=x)=1$ .

Second Borel-Cantelli Lemma (BC2)

If $E_{1},E_{2},\ldots$ are independent, and $\sum_{n\in\mathbb{N}}\mathbb{P}(E_{n})=\infty$ , then $\mathbb{P}(\limsup E_{n})=\mathbb{P}(E_{n},\mbox{ i.o.})=1$ .

Kolmogorov’s Law of the Iterated Logarithm

If $X_{1},X_{2},\ldots$ are i.i.d., $\mathbb{E}X_{n}=0$ , $\mbox{Var}X_{n}=1$ , then $\begin{eqnarray*} \limsup\tfrac{S_{n}}{\sqrt{2n\log\log n}} & = & +1\mbox{ a.s., and}\\ \liminf\tfrac{S_{n}}{\sqrt{2n\log\log n}} & = & -1\mbox{ a.s.} \end{eqnarray*}$

Strassen’s Law of the Iterated Logarithm

If $X_{1},X_{2},\ldots$ are i.i.d., $\mathbb{E}X_{n}=0$ , $\mbox{Var}X_{n}=1$ , $S(t,\omega):[0,+\infty)\to\mathbb{R}$ is the linear interpolation of $S_{n}(\omega)$ , and $Z_{m}(t,\omega)=\tfrac{S_{mt}(\omega)}{\sqrt{2m\log\log m}}:[0,1]\to\mathbb{R}$ , then we define $K(\omega)=\{f(t,\omega):[0,1]\to\mathbb{R}|\exists\{m_{k}\}:Z_{m_{k}}(t,\omega)\rightrightarrows f(t,\omega)\}.$ Let $K=\{f(t):[0,1]\to\mathbb{R}|\exists h(t):\int_{0}^{1}h(s)^{2}ds\le1,f(t)=\int_{0}^{t}h(s)ds\}.$ Then, $\mathbb{P}(K(\omega)=K)=1.$

Subpages

Chapter 4: Independence

Chapter 4: Independence

Independence

Sequences of independent random variables

Stochastic processes and Markov chains

Tail \sigma -algebras

Second Borel-Cantelli Lemma (BC2)

Kolmogorov’s Law of the Iterated Logarithm

Strassen’s Law of the Iterated Logarithm

Tail -algebras