September 8

Discrete Probability Distribution

The probability of a specific outcome $x$ for a random variable $X$ is given by the probability mass function (PMF), denoted as $P(X=x)$ or $f(x)$. For a function to be a valid PMF, it must satisfy two essential properties:

Non-negativity: The probability of any outcome must be between 0 and 1, inclusive.
\[0 \le P(X=x) \le 1\]
Total Probability: The sum of all probabilities for every possible outcome must equal 1. This means you’ve accounted for every single possibility.
\[\sum_{x}{P(X=x)} = 1\]

Measures of Central Tendency and Dispersion

Mean (Expected Value)

The mean or expected value, denoted by $\mu$ or $E(X)$, is the long-run average value of the random variable. It’s a weighted average of all possible values, where each value is weighted by its probability.

\[\mu = E(X) = \mu_X = \sum_{x}{x \cdot P(X=x)}\]

Example: Consider a fair six-sided die roll. Let $X$ be the outcome of the roll. The possible outcomes are $x = {1, 2, 3, 4, 5, 6}$, and the probability of each is $P(X=x) = 1/6$.
- The mean is:
  \[E(X) = 1(\frac{1}{6}) + 2(\frac{1}{6}) + 3(\frac{1}{6}) + 4(\frac{1}{6}) + 5(\frac{1}{6}) + 6(\frac{1}{6}) = \frac{21}{6} = 3.5\]
- This shows that even though you can’t roll a 3.5, the average outcome over many rolls is 3.5.

Variance

The variance, denoted by $Var(X)$ or $\sigma^2$, measures the spread or dispersion of the distribution around its mean. A low variance indicates that the values are clustered closely around the mean, while a high variance suggests they are more spread out.

\[Var(X) = \sigma_X^2 = E[(X-\mu_X)^2] = \sum_{x}(x-\mu)^2 P(X=x)\]

Example: Using the six-sided die roll, the variance is calculated as:
\[Var(X) = (1-3.5)^2(\frac{1}{6}) + (2-3.5)^2(\frac{1}{6}) + \dots + (6-3.5)^2(\frac{1}{6}) \approx 2.917\]
- The standard deviation ($\sigma = \sqrt{Var(X)}$) is the square root of the variance, providing a measure of spread in the same units as the random variable. For this example, $\sigma \approx \sqrt{2.917} \approx 1.708$.