3.1 Expectation of a Random Variable
- The expected value, or mean, or first moment of $X$ is defined to be
$$ \mathbb{E}(X) = \int xdF(x) = \begin{cases} \sum_x xf(x) & (\text{if X is discrete})\ \int xf(x)dx & ( \text{if X is continuous}) \end{cases}$$
3.3 Variance and Covariance
- The variance measures the spread of a distribution
$$ \sigma^2 = \mathbb{E}(X-\mu)^2 = \int (x-\mu)^2 dF(x) $$
- The covariance and correlation between $X$ and $Y$ measure how strong the linear relationship is between $X$ and $Y$
$$\text{Cov}(X,Y) = \mathbb{E}((X-\mu_X)(Y- \mu_Y ))$$
$$ \text{Correlation} \ \rho = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y} $$
3.5 Conditional Expectation
- The conditional expectation of $X$ given $Y=y$ is
$$ \mathbb{E}(X | Y=y) = \begin{cases} \sum x f_{X|Y} (x|y) dx & (\text{discrete case}) \ \int x f_{X|Y}(x|y)dx & (\text{continuous case}) \end{cases}$$
3.6 Moment Generating Functions
- The moment generating function or Laplace transform , of $X$ is defined by
$$ \psi_X(t) = \mathbb{E}(e^{tX}) = \int e^{tx}dF(x) $$