Random Variables and Distributions

Recall that. for any random "experiment", the set of all possible outcomes is denoted by $S$.

A random variable is a function $X: S \to \R$, i.e. it is a rule that associates a (real) number to every outcome of the experiment.

$S$ is the domain of the random variable $X;X(S) \subseteq R$ is its range.

A probability distribution function (p.d.f.) is a function $f : \R → \R$ which specifies the probabilities of the values in $X(S)$.

When $S$ is discrete, we say that $X$ is a $discrete r.v.$ and the p.d.f. is called a probability mass function (p.m.f.).

The p.m.f. of $X$ is $f(x) = P ({s ∈ S : X(s) = x}) := P(X = x)$.

The cumulative distribution function (c.d.f.) of $X$ is $F(x) = P(X ≤ x)$.

Expectation of a Discrete Random Variable

The expectation of a discrete random variable $X$ is defined as:

$E|X| = \sum x\cdot P (X=x) = \sum xf(x)$.

This can be thought of as the sum of $x$ multiplied by the probability of $x$.

Mean and Variance

The expectation can be interpreted as the average or the mean of $X$, denoted as $\mu$.

$E[(Z-E[Z])^2]$ can be thought of the distance from the mean or the variance denoted as $\mathrm{Var}[X]$.

$\mathrm{Var}[X] = E[(X-\mu x)^2] = E[X^2]-\mu^2_x = E[X^2] - E^2[X] = \sigma^2_x$

Standard Deviation

$\mathrm{SD}[X] = \sqrt{\mathrm{Var}[X]} = \sigma_x$

The mean gives some idea as to where the bulk of the distribution is, whereas the variance and standard deviation provide information about the spread; distributions with higher variance/SD are more spread about the average.

Binomial Distribution

Binomial coefficient: $\binom{n}{k} = \frac{n!}{(n-r)r!}$.

A Bernouli trial is a random experiment with two possible outcomes, "success" and "failure". Let $p$ denote teh probability of success.

A binomial experiment consists of $n$ repeated independent Bernoulli trials, each with the same probability of success, $p$.

Binomial Distribution can be defined as:

$P(X=x) = \sum p^x(1-p)^{n-x} = \binom{x}{k}p^x(1-p)^{n-x}$

Expectation, $E[X] = np$ and Variance, $Var[X] = np(1-p)$

Geometric Distribution

P$(X=x) = (1-p)^{x-1} \cdot p$

$E[X] = \frac{1}{p}$ and $\mathrm{Var}[X] = \frac{1-p}{p^2}$

Negative Binomial Distribution

$P(X=x) = \binom{x-1}{k-1}p^{k-1}(1-p)^{x-k}\cdot p$
$= \binom{x-1}{k-1}(1-p)^{x-k}\cdot p$

$E[X] = \frac{k}{p}$ and $\mathrm{Var}[X] = \frac{k(1-p)}{p^2}$

Poisson Distribution

$P(X=x) = \frac{\lambda ^x(e^{-\lambda})}{x!}$

$E[X] = \lambda$ and $\mathrm{Var}[X] = \lambda$

where $\lambda$ is the rate of a Poisson Process.