5.1 Introduction
-
The law of large numbers says that the sample average converges in proability to the expectation $ \mu = \mathbb{E}(X_i)$
-
The central limit theorem says that $ \sqrt{n} (\overline{X - \mu})$ converges in distribution to a Normal distribution.
5.2 Types of Convergence
- $X_n$ converges to $X$ in probability, written $X_n \xrightarrow{P}{} X$ if for every $\epsilon > 0$
$$ \mathbb{P}(|X_n - X| > \epsilon) \rightarrow 0$$
- $X_n$ converges to $X$ in distribution, written $X_n \rightsquigarrow X$ if
$$ \lim_{n\rightarrow \infty} F_n(t) = F(t) $$
- $X_n$ converges to $X$ in quadratic mean (also called convergence in $L_2$), written $X_n \xrightarrow{qm}{} X$, if
$$ \mathbb{E}(X_n - X)^2 \rightarrow 0 \ \text{as} \ n \rightarrow \infty $$
5.3 The Law of Large Numbers
- The weak Law of Large Numbers
$$ If X_1 , … , X_n \ \text{ are IID, then } \overline{X}_n \xrightarrow{P}{} \mu$$
5.4 The Central Limit Theorem
- Let $X_1, … , X_n$ be IID with mean $\mu$ and variance $\sigma^2$. Let $\overline{X_n} = n^{-1} \sum_{i=1}^n X_i$ Then,
$$Z_n \equiv \frac{\overline{X_n}-\mu}{\sqrt{\mathbb{V(\overline{X_n})}}} = \frac{\sqrt{n}({\overline{X_n}-\mu)}}{\sigma} \rightsquigarrow Z$$