Statistics #2 | Random Variables

    2. Statistics  Comments

2.1 Introduction

  • How do we link sample spaces and events to data? Random Variable!

  • A random variable is a mapping $ X : \Omega \rightarrow \mathbb{R} $ that assigns a real number $X(\omega)$ to each outcome $\omega$.


2.2 Distribuition Functions and Probability Functions

  • Cumulative Distribution Function (CDF) : is the function $F_x : \mathbb{R} \rightarrow [0,1] $ defined by

$$F_X(x) = \mathbb{P}(X \leq x)$$

  • Probability Mass Function : A Random Variable $X$ is discrete if it takes countably many values ${x_1, x_2, …}$. We define probability mass function for $X$ by

$$f_X(x) = \mathbb{P}(X=x).$$

  • Probability Density Function : A Random Variabe $X$ is continuous if there exists a function $f_X$ such that $f_X(x) \geq$ for all $x$, $\int f_X(x)dx = 1$ and for every $a \leq b$, the probability Density function is

$$ \mathbb{P}(a<X<b) = \int_{a}^b f_X(x)dx $$


2.3 Some Important Discrete Random Variables

  • The Point Mass Distribution

$$ F(x) = \begin{cases} 0 & (x<a)\ 1 & (x\geq a) \end{cases} $$

  • The Discrete Uniform Distribution

$$ f(x) = \begin{cases} 1/k & ( \text{for} \ x=1, … ,k ) \ 0 & ( \text{otherwise} ) \end{cases} $$

  • The Bernoulli Distribution : Let $X$ represent a binary coin flip. Then $\mathbb{P}(X=1) = p $ and $\mathbb{P}(X=0) = 1- p$ for some $p \in [0,1] $. We saye that $X$ has a Bernoulli Distribution written $X \sim Bernoulli(p)$.

$$ f(x) = p^x(1-p)^{1-x} \ \text{for} \ x \in {0,1}$$

  • The Binomial Distribution

$$ f(x) = \begin{cases} {n \choose x} p^x (1-p)^{n-x} & ( \text{for} \ x=0, … ,n ) \ 0 & ( \text{otherwise} ) \end{cases} $$

  • The Geometric Distribution

$$ \mathbb{P}(X=k) = p(1-p)^{k-1} , \ k \geq 1 $$

  • The Poisson Distribution : The Poisson is often used as a model for counts of rare events like radioactive decay and traffic accidents.

$$ f(x) = e^{- \lambda} \frac{\lambda^x}{x!}, \ x \geq 0 $$


2.4 Some Important Continuous Random Variables

The Uniform Distribution:

$$ f(x) = \begin{cases} \frac{1}{b-a} & ( \text{for} \ x \in [a,b]) \ 0 & ( \text{otherwise} ) \end{cases} $$

Gaussian Distribution : $X$ has a Normal distribution with parameters $ \mu $ and $\sigma$, denoted by $X \sim N(\mu, \sigma^2)$

Exponential Distribution : $X$ has an Exponential distribution with parameter $\beta$, denoted by $X \sim \text{Exp}(\beta)$, if

$$f(x) = \frac{1}{\beta} e^{-x/\beta}$$


2.5 Bivariate Distributions

  • Joint Mass Function : Given a pair of discrete random variables $X$ and $Y$ define the joint mass function by $f(x,y) = \mathbb{P}(X = x, Y = y)$

2.6 Marginal Distributions

  • If $(X, Y)$ have joint distribution with mass function $f_{X,Y}$ then the marginal mass function for $X$ and $Y$ is defined by

$$ f_X(x) = \mathbb{P}(X=x) = \sum_y \mathbb{P}(X=x, Y=y) = \sum_y f(x,y) $$

$$ f_Y(y) = \mathbb{P}(Y=y) = \sum_x \mathbb{P}(X=x, Y=y) = \sum_x f(x,y) $$

  • For continuous random variables, the marginal densities are

$$ f_X(x) = \int f(x,y) dy, \ \text{and} f_Y(y) = \int f(x,y)dx $$


2.7 Independent Random Variables

  • Two random variables $X$ and $Y$ are independent if for every $A$ and $B$,

$$ \mathbb{P}(X \in A, Y \in B) = \mathbb{P}(X \in A) \mathbb{P}(Y \in B)$$


2.8 Conditional Distributions

  • The conditional probability mass function is

$$ f_{X|Y}(x|y) = \mathbb{P}(X=x | Y=y) = \frac{\mathbb{P}(X=x, Y=y)}{\mathbb{P}(Y=y)} = \frac{f_{X,Y}(x,y)}{f_Y(y)}$$


2.9 Multivariate Distributions and IID Samples

  • If $X_1, … , X_n $ are independent and each has the same marginal distribution with CDF $F$ we say that $X_1, … , X_n$ are IID(independent and identically distributed) and we write

$$ X_1 , … , X_n \sim F.$$


2.10 Two Important Multivariate Distributions

  • Multinomial : The multivariate version of a Binomial is called a Multinomial.

  • Multicariate Normal : The univariate normal has two parameters, $\mu$ and $\sigma$. In the multivariate version, $\mu$ is a vector and $\sigma$ is replaced by a matrix $\Sigma$


2.11 Transformations of Random variables

  • Suppose that $X$ is a random variable with PDF $f_X$ and CDF $F_X$. Let $Y$ = r(X) be a function of $X$, for example, $Y = X^2$ or $Y = e^X$. We call $Y=r(X)$ a Transformation of $X$.

2.12 Transformations of Several Random variables

  • Three Steps for Transformation

    1. For each $z$, find the set $A_z = { (x,y) : r(x,y) \leq z }$

    2. Find the CDF

      $$F_Z(z) = \mathbb{P}(Z \leq z) = \mathbb{P}(r(X,Y) \leq z) = \mathbb{P}({ (x,y); r(x,y) \leq z })$$

    3. Them $f_Z(z) = F’_{Z}(z)$