December 1

Bivariate Distribution

\[\rho = \frac{S_{xy}}{\sqrt{S_{xx}}\sqrt{S_{yy}}}\]

The bivariate distribution describes the joint probability distribution of two random variables $X$ and $Y$. It provides insights into the relationship between the variables, including their correlation and regression.

Regression Line

The regression line predicts the value of $Y$ based on $X$ using the equation:

\[\hat{Y} = a + bX = \hat{B}_0 + \hat{B}_1 X\]

Where:

$\hat{Y}$: Predicted value of $Y$
$\hat{B}_0$: Intercept of the regression line
$\hat{B}_1$: Slope of the regression line, calculated as: $\hat{B}_1 = \frac{S_{xy}}{S_{xx}}$
$a$: Alternative notation for the intercept
$b$: Alternative notation for the slope

The slope $\hat{B}_1$ represents the change in $Y$ for a one-unit change in $X$, while the intercept $\hat{B}_0$ represents the value of $Y$ when $X = 0$.

Joint Distribution

The joint distribution describes the probability distribution of two random variables $X$ and $Y$ occurring simultaneously. It is represented as $f(x, y)$ for continuous random variables or $P(X=x, Y=y)$ for discrete random variables.

Properties of Joint Distribution

Non-Negativity: $f(x, y) \geq 0 \quad \text{or} \quad P(X=x, Y=y) \geq 0$ The probability or density function must always be non-negative.
Normalization:
- For discrete random variables: $\sum_{x} \sum_{y} P(X=x, Y=y) = 1$
- For continuous random variables: $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1$
Marginal Distribution: The marginal distribution of $X$ or $Y$ can be obtained by summing (discrete) or integrating (continuous) over the other variable:
- Discrete: $P(X=x) = \sum_{y} P(X=x, Y=y)$ $P(Y=y) = \sum_{x} P(X=x, Y=y)$
- Continuous: $f_X(x) = \int_{-\infty}^{\infty} f(x, y) \, dy$ $f_Y(y) = \int_{-\infty}^{\infty} f(x, y) \, dx$
Independence: $X$ and $Y$ are independent if and only if: $f(x, y) = f_X(x) f_Y(y) \quad \text{or} \quad P(X=x, Y=y) = P(X=x) P(Y=y)$

Example: Discrete Joint Distribution

Consider a scenario where we randomly select 5 items from a group of 120 items, consisting of 90 of type $X$ and 30 of type $Y$. The joint probability mass function is given by: $P(X=x, Y=y) = \frac{\binom{90}{x} \binom{30}{y}}{\binom{120}{5}}$

Example: Continuous Joint Distribution

For continuous random variables $X$ and $Y$, suppose their joint probability density function is: $f(x, y) = \begin{cases} c(x + y), & 0 \leq x \leq 1, \ 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$ To find $c$, use the normalization property: $\int_{0}^{1} \int_{0}^{1} c(x + y) \, dx \, dy = 1$

Example: Discrete

The joint probability mass function for selecting $x$ items of type $X$ and $y$ items of type $Y$ is given by: $P(X=x, Y=y) = \frac{\binom{90}{x} \binom{30}{y}}{\binom{120}{5}}$

Notes:

$\binom{n}{k}$ represents the binomial coefficient, which calculates the number of ways to choose $k$ items from $n$ items.
The denominator $\binom{120}{5}$ represents the total number of ways to choose 5 items from the group of 120 items.
The numerator $\binom{90}{x} \binom{30}{y}$ represents the number of ways to choose $x$ items from the 90 items of type $X$ and $y$ items from the 30 items of type $Y$.

Marginal pmf of $x$ and $y$

The marginal probability mass function (pmf) of $X$ and $Y$ can be derived from the joint pmf by summing over the other variable:

Marginal pmf of $X$: $P(X=x) = \sum_{y} P(X=x, Y=y)$
Marginal pmf of $Y$: $P(Y=y) = \sum_{x} P(X=x, Y=y)$

These equations allow us to compute the probabilities of $X$ and $Y$ independently by aggregating over all possible values of the other variable.

Joint pdf of r.v. $X$ and $Y$

The joint probability density function (pdf) of continuous random variables $X$ and $Y$ describes the likelihood of $X$ and $Y$ taking on specific values simultaneously. It is denoted as $f(x, y)$.

Suppose the joint pdf is given by: $f(x, y) = \begin{cases} c x, & 0 \leq x \leq 1, \ 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$

To find $c$, use the normalization property: $\int_{0}^{1} \int_{0}^{1} c x \, dx \, dy = 1$

Properties of Joint pdf:

Non-Negativity: $f(x, y) \geq 0$
Normalization: $\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(x, y) \, dx \, dy = 1$
Marginal pdfs:
- Marginal pdf of $X$: $f_X(x) = \int_{-\infty}^{\infty} f(x, y) \, dy$
- Marginal pdf of $Y$: $f_Y(y) = \int_{-\infty}^{\infty} f(x, y) \, dx$
Independence: $X$ and $Y$ are independent if and only if: $f(x, y) = f_X(x) f_Y(y)$

Example: Joint pdf of $X$ and $Y$

Suppose the joint pdf of $X$ and $Y$ is given by: $f(x, y) = \begin{cases} 2x, & 0 \leq x \leq 1, \ 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$

Marginal pdf of $X$:

To find $f_X(x)$, integrate $f(x, y)$ over $y$: $f_X(x) = \int_{0}^{1} f(x, y) \, dy = \int_{0}^{1} 2x \, dy = 2x \cdot [y]_{0}^{1} = 2x.$ Thus, the marginal pdf of $X$ is: $f_X(x) = \begin{cases} 2x, & 0 \leq x \leq 1 \\ 0, & \text{otherwise} \end{cases}$

Marginal pdf of $Y$:

To find $f_Y(y)$, integrate $f(x, y)$ over $x$: $f_Y(y) = \int_{0}^{1} f(x, y) \, dx = \int_{0}^{1} 2x \, dx = \left[x^2\right]_{0}^{1} = 1.$ Thus, the marginal pdf of $Y$ is: $f_Y(y) = \begin{cases} 1, & 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$

Expected Value of $X$:

The expected value of $X$ is calculated as: $E(X) = \int_{0}^{1} x f_X(x) \, dx = \int_{0}^{1} x \cdot 2x \, dx = \int_{0}^{1} 2x^2 \, dx = \left[\frac{2x^3}{3}\right]_{0}^{1} = \frac{2}{3}.$

Variance of $X$:

The variance of $X$ is calculated as: $\text{Var}(X) = E(X^2) - [E(X)]^2.$ First, calculate $E(X^2)$: $E(X^2) = \int_{0}^{1} x^2 f_X(x) \, dx = \int_{0}^{1} x^2 \cdot 2x \, dx = \int_{0}^{1} 2x^3 \, dx = \left[\frac{2x^4}{4}\right]_{0}^{1} = \frac{1}{2}.$ Now, substitute into the variance formula: $\text{Var}(X) = \frac{1}{2} - \left(\frac{2}{3}\right)^2 = \frac{1}{2} - \frac{4}{9} = \frac{9}{18} - \frac{8}{18} = \frac{1}{18}.$

Expected Value of $Y$:

The expected value of $Y$ is calculated as: $E(Y) = \int_{0}^{1} y f_Y(y) \, dy = \int_{0}^{1} y \cdot 1 \, dy = \left[\frac{y^2}{2}\right]_{0}^{1} = \frac{1}{2}.$

Variance of $Y$:

The variance of $Y$ is calculated as: $\text{Var}(Y) = E(Y^2) - [E(Y)]^2.$ First, calculate $E(Y^2)$: $E(Y^2) = \int_{0}^{1} y^2 f_Y(y) \, dy = \int_{0}^{1} y^2 \cdot 1 \, dy = \left[\frac{y^3}{3}\right]_{0}^{1} = \frac{1}{3}.$ Now, substitute into the variance formula: $\text{Var}(Y) = \frac{1}{3} - \left(\frac{1}{2}\right)^2 = \frac{1}{3} - \frac{1}{4} = \frac{4}{12} - \frac{3}{12} = \frac{1}{12}.$

Summary:

$f_X(x) = 2x$ for $0 \leq x \leq 1$.
$f_Y(y) = 1$ for $0 \leq y \leq 1$.
$E(X) = \frac{2}{3}$.
$\text{Var}(X) = \frac{1}{18}$.
$E(Y) = \frac{1}{2}$.
$\text{Var}(Y) = \frac{1}{12}$.

Conditional Density Function

The conditional density function describes the probability distribution of a random variable $X$ given that another random variable $Y$ has a specific value. It is denoted as $f(x \mid y)$ and is defined as: $f(x \mid y) = \frac{f(x, y)}{f_Y(y)}, \quad \text{for } f_Y(y) > 0.$

Properties of Conditional Density Functions

Non-Negativity: $f(x \mid y) \geq 0.$
Normalization: The conditional density integrates to 1 over all possible values of $X$: $\int_{-\infty}^{\infty} f(x \mid y) \, dx = 1.$
Relationship with Joint and Marginal Densities: The joint density can be expressed in terms of the conditional and marginal densities: $f(x, y) = f(x \mid y) f_Y(y).$

Example: Conditional Density Function

Suppose the joint pdf of $X$ and $Y$ is given by: $f(x, y) = \begin{cases} 2x, & 0 \leq x \leq 1, \ 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$

From the marginal pdf of $Y$, we know: $f_Y(y) = \begin{cases} 1, & 0 \leq y \leq 1 \\ 0, & \text{otherwise} \end{cases}$

The conditional density of $X$ given $Y = y$ is: $f(x \mid y) = \frac{f(x, y)}{f_Y(y)} = \frac{2x}{1} = 2x, \quad \text{for } 0 \leq x \leq 1, \ 0 \leq y \leq 1.$

Expected Value of $X$ Given $Y$

The expected value of $X$ given $Y = y$ is calculated as: $E(X \mid Y = y) = \int_{0}^{1} x f(x \mid y) \, dx = \int_{0}^{1} x \cdot 2x \, dx = \int_{0}^{1} 2x^2 \, dx = \left[\frac{2x^3}{3}\right]_{0}^{1} = \frac{2}{3}.$