2.1 The Normal Errors Model

In Section 1.2.3, we made no assumption about the distribution of the errors $\varepsilon$ in model (1.1) other than the mean and the variance.

We now make an additional assumption on the distribution of $\varepsilon$.

We first introduce some notation. We will use the "$\sim$" symbol to represent "distributed as".

Next to the "$\sim$" symbol will will denote the family of distributions. For example, we will use "N" to denote the normal family of distributions. So $$ X\sim N $$ is read as "The random variable $X$ is distributed as a normal random variable." Or more simply, "The random variable $X$ has a normal distribution" or "The random variable $X$ is normally distributed".

The particular distribution will be defined by some parameters. For the normal family of distributions, a particular normal distribution is defined by the mean $\mu$ and by the variance $\sigma^2$. We will place these parameters in parentheses. So $$ X\sim N\left(\mu, \sigma^2\right) $$ is read as "The random variable $X$ has a normal distribution with mean $\mu$ and variance $\sigma^2$".

To denote that a series of random variables are independent, we will use the notation iid. This notation stands for independent and identically distributed.

It is common to see this in front of or over the "$\sim$". So if there are a series of random variables, $X_i$, that were independent and each normally distributed with the same mean $\mu$ and same variance $\sigma^2$, then we could denote them as $$ X_i iid \sim N\left(\mu,\sigma^2\right) $$ or $$ X_i\overset{iid}{\sim}N\left(\mu,\sigma^2\right) $$.

In some contexts, it is implied that $X$ represents a series of random variables, thus the subscript $i$ is left off.

We now make an additional assumption on the random error term $\varepsilon$. We will assume that $\varepsilon$ is normally distributed with mean 0 and variance $\sigma^2$. Furthermore, any pair of random error terms, ($\varepsilon_i$ and $\varepsilon_j$), are jointly normal .

We still have the assumptions about uncorrelated errors from Section 1.2.3. Since any pair ($\varepsilon_i$ and $\varepsilon_j$) is jointly normal and uncorrelated, then this implies that the pair is independent.

Therefore, we have $$ \varepsilon \overset{iid}{\sim}N\left(0,\sigma^2\right) $$

With the additional assumption on $\varepsilon$, we now have the regression model as \begin{align*} Y_i=&\beta_0+\beta_1X_i+\varepsilon_i\\ \varepsilon_i\overset{iid}{\sim}& N\left(0,\sigma^2\right)\qquad\qquad\qquad\qquad(2.1) \end{align*} We call model (2.1) the normal errors model.

With the normal errors model, we can use the assumption of normally distributed errors to learn about the sampling distributions of the least squares estimators (1.4)

To learn about these sampling distributions, we will use the following two theorems from math stats (presented here without proof):

Theorem 2.1 Sum of Independent Normal Random Variables:
If $$ X_i\sim N\left(\mu_i,\sigma_i^2\right) $$ are independent, then the linear combination $\sum_i a_iX_i$ is also normally distributed where $a_i$ are constants. In particular $$ \sum_i a_iX_i \sim N\left(\sum_i a_i\mu_i, \sum_i a_i^2\sigma_i^2\right) $$

Theorem 2.2 Adding a Constant to a Normal Random Variable:
If $$ X\sim N\left(\mu,\sigma^2\right) $$ then for any real constant $c$, $$ X+c\sim N\left(\mu+c,\sigma^2\right) $$

For model (2.1), we see that $Y$ is just a constant added to $\varepsilon$ (recall that $X$ is treated as a constant). Therefore, from Theorem 2.2, \begin{align*} Y_i{\sim} N\left(\beta_0+\beta_1X_i,\sigma^2\right)\qquad\qquad\qquad(2.2) \end{align*} Note that the $Y$s are independent but not identically distributed. This is because the mean of the $Y$s changes based on the line $\beta_0+\beta_1X_i$.