Equation Bias

1. Core Concept Overview & Intuition

1.1 Introduction

In earlier work on the classical linear regression model, a central assumption is that the stochastic disturbance term is independent of (or at least uncorrelated with) the regressors. When this assumption holds, ordinary least squares (OLS) produces estimators that are unbiased in finite samples and consistent as the sample grows. In most economic settings, however, this independence assumption fails. The failure is especially severe in systems of simultaneous equations, where the regressors are themselves endogenous variables determined within the system.

When a regressor is correlated with the disturbance, OLS applied equation by equation yields estimates that are both biased in small samples and inconsistent asymptotically. The discrepancy between the probability limit of the OLS estimator and the true parameter is called equation bias (also termed simultaneous equation bias). This unit develops the algebra that makes the bias explicit, distinguishes finite-sample bias from asymptotic bias, and shows why consistency fails.

1.2 Learning Outcomes

By the end of this unit the learner will be able to:

1. estimate the bias in the estimation of a simultaneous system;
2. demonstrate the asymptotic unbiasedness and the consistency (or inconsistency) of estimators.

1.3 Simultaneous Equation Relationships

A simultaneous equation relationship is a system in which two or more endogenous variables are jointly determined, so that each equation contains variables that are themselves explained elsewhere in the system. The simplest illustrative system is a Keynesian consumption–income model:

$$
C_t = \beta_1 + \beta_2 Y_t + u_t \tag{2.1a}
$$

$$
Y_t = C_t + I_t \tag{2.1b}
$$

Here $C_t$ denotes consumption, $Y_t$ denotes income, $I_t$ denotes investment, and $u_t$ is the stochastic disturbance. The maintained assumptions on the disturbance and on investment are

$$
E(u_t) = 0, \qquad E(u_t^2) = \sigma^2, \qquad E(u_t u_{t'}) = 0, \qquad E(u_t I_t) = 0.
$$

The crucial feature is that $Y_t$ appears on the right-hand side of (2.1a) but is itself determined by (2.1b). Because $Y_t$ depends on $C_t$, and $C_t$ depends on $u_t$, income and the disturbance are correlated. This endogeneity is the source of equation bias.

1.4 Intuition for the Bias

The OLS estimator of $\beta_2$ in (2.1a) is computed by regressing consumption on income. But income moves for two reasons: genuine variation in investment, and variation transmitted from the disturbance $u_t$ through the identity (2.1b). The second channel means that part of the movement in the regressor is mechanically tied to the error term. OLS therefore attributes to income some of the variation that actually belongs to the disturbance, and the estimated slope is systematically displaced from the true parameter. As the sample grows, this displacement does not vanish, because the population covariance between income and the disturbance is itself nonzero.

2. Mathematical Formalization & Core Definitions

2.1 Deviation Form

To simplify the algebra, variables are written in deviation form, that is, as deviations from their expected values:

$$
c_t = C_t - E(C_t), \qquad y_t = Y_t - E(Y_t).
$$

In deviation form, the intercept drops out and equation (2.1a) becomes

$$
y_t = \beta_2 c_t + u_t. \tag{2.1}
$$

The disturbance retains the properties

$$
E(u_t) = 0, \qquad E(u_t^2) = \sigma^2, \qquad E(u_t u_{t'}) = 0,
$$

but now the key endogeneity condition holds:

$$
E(y_t u_t) \neq 0.
$$

This follows directly from the identity (2.1b) written in deviation form. Let $i_t = I_t - E(I_t)$ denote investment in deviation form. Then

$$
y_t = c_t + i_t,
$$

and therefore

$$
E(y_t u_t) = E(c_t u_t) \neq 0,
$$

because both the consumption deviation and the income deviation contain the disturbance $u_t$.

2.2 The Normal Equations

The normal equation for the slope coefficient is obtained by multiplying equation (2.1) by $y_t$ and summing over $t$:

$$
\sum_t y_t^2 = \beta_2 \sum_t y_t c_t + \sum_t y_t u_t. \tag{2.2}
$$

In the classical model the final term $\sum_t y_t u_t$ would vanish in expectation. Here, because $y$ and $u$ are correlated, it does not disappear.

2.3 The OLS Estimator and Finite-Sample Bias

Dividing equation (2.2) throughout by $\sum_t y_t^2$, and noting that $\sum_t y_t c_t / \sum_t y_t^2$ is precisely the OLS estimator $\hat{\beta}_2$ applied to (2.1), we obtain

$$
\hat{\beta}_2 = \frac{\sum_t y_t c_t}{\sum_t y_t^2} = \beta_2 + \frac{\sum_t y_t u_t}{\sum_t y_t^2}. \tag{2.3}
$$

Taking expectations of (2.3) gives

$$
E(\hat{\beta}_2) = \beta_2 + E!\left( \frac{\sum_t y_t u_t}{\sum_t y_t^2} \right) \neq \beta_2,
$$

so that the finite (small) sample bias is

$$
\text{Bias} = E(\hat{\beta}_2) - \beta_2 = E!\left( \frac{\sum_t y_t u_t}{\sum_t y_t^2} \right) \neq 0.
$$

The estimator is therefore biased in finite samples.

2.4 The Probability Limit and Asymptotic Bias

To study asymptotic behaviour, take probability limits of (2.3). Dividing numerator and denominator by $n$ and applying the probability limit as $n \to \infty$,

$$
\operatorname{plim}_{n \to \infty} \hat{\beta}_2 = \beta_2 + \frac{\operatorname{n \to \infty} \frac{1}{n} \sum_t y_t u_t}{\operatorname*{plim}} \frac{1}{n} \sum_t y_t^2}. \tag{2.4
$$

As $n$ tends to infinity, the sample covariance of $Y$ and $u$,

$$
s_{yu} = \frac{1}{n} \sum_t y_t u_t,
$$

tends to the true population covariance of $Y$ and $u$, and the sample variance of $Y$,

$$
s_y^2 = \frac{1}{n} \sum_t y_t^2,
$$

tends to the true population variance $\operatorname{Var}(Y)$. Equation (2.4) therefore becomes

$$
\operatorname*{plim}_{n \to \infty} \hat{\beta}_2 = \beta_2 + \frac{\operatorname{Cov}(Y, u)}{\operatorname{Var}(Y)}. \tag{2.5}
$$

Hence the asymptotic bias is

$$
\text{Asymptotic Bias} = \operatorname*{plim}_{n \to \infty} \hat{\beta}_2 - \beta_2 = \frac{\operatorname{Cov}(Y, u)}{\operatorname{Var}(Y)} = \frac{\sigma^2}{(1 - \beta_2)\operatorname{Var}(Y)} \neq 0,
$$

using the result that $\operatorname{Cov}(Y, u) = \sigma^2 / (1 - \beta_2)$.

2.5 Consistency versus Asymptotic Unbiasedness

> Definition (Consistency). An estimator $\hat{\theta}n$ of a parameter $\theta$ is consistent if its probability limit equals the true parameter, that is, if $\operatorname*{plim}_n = \theta$.} \hat{\theta

> Definition (Asymptotic Unbiasedness). An estimator $\hat{\theta}n$ is asymptotically unbiased if the limit of its expectation equals the true parameter, that is, if $\lim_n) = \theta$.} E(\hat{\theta

These two properties are distinct. Consistency is the stronger requirement in the sense that it concerns the entire limiting distribution, whereas asymptotic unbiasedness concerns only the limiting mean. For the OLS estimator in the simultaneous system, the probability limit in (2.5) differs from $\beta_2$ because $\operatorname{Cov}(Y, u) \neq 0$. The estimator is therefore both biased in small samples and inconsistent: its probability limit differs from the true parameter, and the sample covariance of $Y$ and $u$ and the sample variance of $Y$ do not approach values that remove the discrepancy as the sample size increases.