E-Tivity 6 — The Two-Stage Least Squares Estimation Method (2Sls)

1. Core Concept Overview & Intuition

1.1 Introduction

The two-stage least squares (2SLS) method is an estimation technique for the parameters of a structural equation belonging to a system of simultaneous equations. It is designed principally for over-identified equations, that is, equations in which the number of excluded predetermined (exogenous) variables exceeds the number of included endogenous explanatory variables by more than the minimum required for identification. The method proceeds in two distinct stages, hence its name: in the first stage each endogenous explanatory variable appearing on the right-hand side of the structural equation is regressed on all the predetermined variables of the system, and the fitted values from that regression are retained; in the second stage the structural equation is estimated by ordinary least squares (OLS) with the endogenous regressor replaced by its fitted values. The purpose of the first stage is to purge the endogenous regressor of the stochastic component that is correlated with the structural disturbance, thereby restoring the validity of the OLS assumptions in the second stage.

1.2 Learning Outcomes

By the end of this unit the learner should be able to:

• explain why OLS applied directly to a structural equation of a simultaneous system yields biased and inconsistent estimates;
• describe the two stages of the 2SLS procedure and the role of the reduced-form regressions in the first stage;
• assemble and solve the normal equations of the second-stage regression for the structural coefficients;
• state and interpret the small-sample and large-sample properties of 2SLS estimators;
• compare 2SLS with indirect least squares (ILS) with respect to over-identified equations;
• apply the identification condition to each equation of a simultaneous system and determine the appropriate estimation method.

1.3 Simultaneous Equation Relationships

A simultaneous equation system consists of two or more structural equations in which the endogenous variables are jointly determined. A canonical two-equation system takes the form

$$
\begin{aligned}
y_1 &= \beta_{11}y_2 + \beta_{12}x_1 + u_1, \
y_2 &= \beta_{21}y_1 + \beta_{22}x_2 + \beta_{23}x_3 + v_2,
\end{aligned}
$$

where $y_1$ and $y_2$ are endogenous variables, $x_1$, $x_2$ and $x_3$ are predetermined (exogenous) variables, and $u_1$ and $v_2$ are stochastic disturbances. The essential feature of such a system is that $y_1$ appears as a regressor in the second equation and $y_2$ appears as a regressor in the first. Because $y_2$ is determined jointly with $y_1$, it is correlated with $u_1$; likewise $y_1$ is correlated with $v_2$. This contemporaneous dependence between an explanatory variable and the disturbance is the source of the estimation problem addressed by 2SLS.

The identification status of each equation is assessed by the order condition: an equation is identified if the number of predetermined variables excluded from it is at least as large as the number of endogenous variables included in it on the right-hand side, less one. An equation satisfying the condition with equality is exactly identified; one satisfying it with strict inequality is over-identified; one failing it is under-identified. In the system above, the first equation excludes $x_2$ and $x_3$ (two predetermined variables) and includes one endogenous regressor, $y_2$, so it is over-identified; the second equation excludes $x_1$ (one predetermined variable) and includes one endogenous regressor, $y_1$, so it is exactly identified.

1.4 Equation Bias

Direct application of OLS to a structural equation such as the first equation above produces what is termed equation bias (also described as simultaneous equation bias). The mechanism is as follows. The endogenous regressor $y_2$ is a random variable whose reduced-form representation contains the disturbances of the entire system, including $u_1$. Consequently the regressor and the disturbance are correlated, violating the OLS assumption of zero covariance between regressors and the error term. The OLS estimator is therefore biased in finite samples and, more seriously, inconsistent: the probability limit of the estimator does not equal the true parameter value even as the sample size grows without bound. The bias does not vanish asymptotically, so increasing the sample does not repair the defect. This is precisely the defect that the two-stage procedure is constructed to remove.

1.5 Intuition for the Two-Stage Procedure

The first stage replaces the offending endogenous regressor by a constructed variable that is highly correlated with it but, by construction, uncorrelated with the structural disturbance. Because the fitted values from the first-stage regression are linear combinations of the predetermined variables alone, and the predetermined variables are uncorrelated with the disturbances by assumption, the fitted values carry none of the stochastic dependence that contaminates the original regressor. The second stage then applies OLS to an equation in which the regressor is asymptotically uncorrelated with the error, so that the OLS assumptions are restored in large samples. The price paid is that the second-stage estimates are biased in small samples, since the fitted values are imperfect proxies for the true endogenous regressor; however, the bias disappears asymptotically, and the resulting estimator is consistent.

2. Mathematical Formalization & Core Definitions

2.1 Notation and Definitions

> Definition (Structural equation). A structural equation is an equation of a simultaneous system that expresses a behavioural or technological relationship among endogenous variables and predetermined variables, with a stochastic disturbance appended. Its coefficients are the structural parameters of interest.

> Definition (Reduced-form equation). A reduced-form equation expresses an endogenous variable solely as a function of the predetermined variables of the system and the disturbances. The reduced form is obtained by solving the structural system for the endogenous variables.

> Definition (Endogenous variable). An endogenous variable is a variable whose value is determined within the system of simultaneous equations; it is jointly dependent with the other endogenous variables and correlated with the structural disturbances.

> Definition (Predetermined variable). A predetermined variable is a variable whose value is determined outside the system, or lagged sufficiently that it may be treated as uncorrelated with the current disturbances. Such variables serve as the instruments in the first stage of 2SLS.

> Definition (Two-stage least squares estimator). The 2SLS estimator of the parameters of a structural equation is the OLS estimator obtained by regressing the dependent endogenous variable on the fitted values of the endogenous explanatory variables, where those fitted values are produced by first-stage regressions of each endogenous explanatory variable on all the predetermined variables of the system.

2.2 The First Stage

Consider a structural equation containing a single endogenous explanatory variable $y_2$ together with predetermined variables. The first stage regresses $y_2$ on all the predetermined variables of the system, say $x_1, x_2, \ldots, x_k$, obtaining

$$
y_2 = \hat{\pi}_1 x_1 + \hat{\pi}_2 x_2 + \cdots + \hat{\pi}_k x_k + \hat{e}_2,
$$

where the $\hat{\pi}_j$ are the estimated reduced-form coefficients and $\hat{e}_2$ is the residual. The fitted values are

$$
\hat{y}_2 = \hat{\pi}_1 x_1 + \hat{\pi}_2 x_2 + \cdots + \hat{\pi}_k x_k.
$$

By construction $\hat{y}_2$ is a linear combination of the predetermined variables only. It is therefore uncorrelated with the structural disturbance $u_1$ in the population, and it is strongly correlated with $y_2$ because it is the projection of $y_2$ on the predetermined variables. These two properties are exactly what the second stage requires.

2.3 The Second Stage and the Normal Equations

In the second stage the structural equation is estimated by OLS with $y_2$ replaced by $\hat{y}_2$. For the first structural equation of the system in Section 1.3, the second-stage regression is

$$
y_1 = \hat{\beta}{11}\hat{y}_2 + \hat{\beta}_1,}x_1 + \hat{u
$$

where $\hat{\beta}{11}$ and $\hat{\beta}}$ denote the 2SLS estimates of $\beta_{11}$ and $\beta_{12}$. The normal equations of this second-stage regression are obtained by minimising the sum of squared second-stage residuals with respect to the two coefficients. Differentiating with respect to $\hat{\beta{11}$ and setting the derivative to zero gives the first normal equation, and differentiating with respect to $\hat{\beta}$ gives the second. The result is

$$
\begin{aligned}
\hat{\beta}{11}\sum \hat{y}_2^2 + \hat{\beta}}\sum x_1\hat{y2 &= \sum \hat{y}_2 y_1, \
\hat{\beta}}\sum x_1\hat{y2 + \hat{\beta}\sum x_1^2 &= \sum x_1 y_1.
\end{aligned}
$$

Each right-hand side is the cross-product of the corresponding regressor with the observed dependent variable: the first equation pairs $\hat{y}2$ with $y_1$, and the second pairs $x_1$ with $y_1$. These are the standard normal equations of a two-regressor OLS regression, assembled with $\hat{y}_2$ in place of the unobservable $y_2$. The sums $\sum \hat{y}_2^2$, $\sum x_1\hat{y}_2$ and $\sum x_1^2$ are computed from the data after the first stage, and the cross-products $\sum \hat{y}_2 y_1$ and $\sum x_1 y_1$ are computed from the observed dependent variable and the fitted regressor. Solving the two equations simultaneously yields the 2SLS estimates $\hat{\beta}$.}$ and $\hat{\beta}_{12

The same construction applies to the second structural equation, which contains the endogenous regressor $y_1$. Its first stage regresses $y_1$ on all the predetermined variables $x_1, x_2, x_3$, producing fitted values $\hat{y}1$; the second stage then regresses $y_2$ on $\hat{y}_1$, $x_2$ and $x_3$, and the corresponding normal equations are solved for $\hat{\beta}}$, $\hat{\beta{22}$ and $\hat{\beta}$.

2.4 The General Normal Equations for a Two-Regressor Second Stage

For a second-stage regression of $y$ on two regressors $z_1$ and $z_2$, the normal equations in matrix form are

$$
\begin{pmatrix}
\sum z_1^2 & \sum z_1 z_2 \
\sum z_1 z_2 & \sum z_2^2
\end{pmatrix}
\begin{pmatrix}
\hat{\beta}_1 \
\hat{\beta}_2
\end{pmatrix}
=
\begin{pmatrix}
\sum z_1 y \
\sum z_2 y
\end{pmatrix}.
$$

In the 2SLS application, $z_1 = \hat{y}_2$ and $z_2 = x_1$. The solution is obtained by inverting the cross-product matrix, provided it is non-singular, which requires that the regressors not be perfectly collinear. This non-singularity is guaranteed in the 2SLS context by the requirement that the predetermined variables used in the first stage not be strongly multicollinear.

2.5 Identification and the Choice of Estimator

The identification status of each equation determines which estimator is admissible. An exactly identified equation may be estimated by ILS, which yields a unique estimate because the number of reduced-form coefficients available equals the number of structural parameters to be recovered. An over-identified equation cannot be handled by ILS without ambiguity, since ILS would deliver multiple, generally conflicting, estimates of the same structural parameter. For such equations 2SLS is the appropriate method, and it delivers a single estimate of each parameter. An under-identified equation cannot be estimated by either method without further restrictions.