In statistics and econometrics, the Frisch–Waugh–Lovell (FWL) theorem is a property of ordinary least squares estimators. Named for econometricians Ragnar Frisch, Frederick V. Waugh, and Michael C. Lovell, the theorem states that estimates of multiple regression coefficients depend only on the unique variation of each explanatory variable.
Ordinary least squares is a method of estimating coefficients in a linear regression where a single dependent variable is modeled as a linear function of one or more explanatory variables. The Frisch–Waugh–Lovell theorem states that, in a linear regression fitted by least squares, each explanatory variable's coefficient reflects the relationship between the dependent variable and the part of that explanatory variable which is not linearly related to the other explanatory variables. Specifically, each explanatory variable can be decomposed into two parts: the part linearly related to the other explanatory variables, and a residual component. Then, that variable's coefficient can be found by regressing the dependent variable on the residual component.
Coefficients in least squares-estimated regressions are often interpreted as the effect of the respective variable controlling for or holding constant the set of other explanatory variables. The Frisch–Waugh–Lovell theorem formalizes this, and as a result is sometimes called the regression anatomy theorem.
An initial version of the theorem was introduced by Udny Yule in 1907, though it was not popularized in economics until a 1933 paper by Ragnar Frisch and Frederick Waugh in the first volume of Econometrica. At the time, there was debate among economists over the proper way to adjust statistical models for the effect of time trends. Frisch and Waugh used the theorem to show that the two leading methods of adjustment were numerically equivalent, resolving the debate. Michael Lovell contributed to the theorem's development through a 1963 paper generalizing Frisch and Waugh's result beyond time trends to arbitrary sets of explanatory variables.
Background The Frisch–Waugh–Lovell theorem is a result for regressions estimated by ordinary least squares, the most commonly used estimator in applied econometrics. A regression is a statistical model where a dependent variable is modeled as a function of one or more explanatory variables plus some residual term. Ordinary least squares can be used when a dependent variable is modeled as a linear combination of one or more explanatory variables plus some residual term. For example, an individual's wages may be modeled as a linear function of a constant term, education, and parental income, with a residual term that encompasses deviations from the model's prediction. Ordinary least squares sets the values of the coefficients to minimize the sum of squared residuals. Under a certain set of assumptions, the hypotheses of the Gauss–Markov theorem, least squares estimation is the best linear unbiased estimator.
Let be any dependent variable and a set of explanatory variables, and suppose observations of are obtained. If is modeled as a linear function of the explanatory variables and a constant, the estimated model can be written as , where the hats denote estimates of the respective parameters. The least squares estimator sets the coefficients to minimize the sum of squared residuals. With observations this involves minimizing across equations, and is typically written in matrix form as , where and are vectors of dependent variable observations and residuals, respectively, is an matrix of explanatory variables' observations, and is a coefficient vector. Then, the least squares solution is yielding .
In regressions estimated by least squares, it is common to refer to an explanatory variable's coefficient as the effect of that variable "holding constant" the other explanatory variables. For example, if wage is modeled as a function of education and work experience, the coefficient on education is interpreted as the difference in the expectation of wage for a unit difference in education, "holding constant" work experience. Econometrician Arthur Goldberger frames the Frisch–Waugh–Lovell theorem as "giving content to th[is] language".
Definition and interpretation The Frisch–Waugh–Lovell theorem states that in a least squares-estimated regression of the form
any coefficient can be obtained by the two-step process of: Regress on the set of other explanatory variables, obtaining residuals Regress on , obtaining This two-step process is referred to as the residual regression or equivalently the regression anatomy theorem. This result is a numerical property of least squares estimation and does not depend on statistical properties of the data.
The variation in each explanatory variable can be decomposed into two parts: the part which is linearly related to the set of other explanatory variables and the remaining, 'residual' variation. By the theorem, an explanatory variable's coefficient can be found from the simple regression of the dependent variable on the residual part. This result is the basis for interpreting the impact of including additional variables in a regression: it is equivalent to removing from the existing variables the component of shared variance with the new variables.
Further, the Frisch–Waugh–Lovell theorem can be applied to understand how multicollinearity affects parameter estimation. When most of the variation in an explanatory variable is linearly explained by the other explanatory variables, the residual part has relatively very little variation. Consequently, the estimate of the explanatory variable's coefficient may be less precise than if fewer variables were controlled for.
Example Consider the regression of wage on education and parental income:
While the least squares estimates for and can be obtained by minimizing directly, each can be equivalently obtained by the two-step residual regression. In the case of education: Regress education on parental income, saving the residuals from this regression: the part of education not linearly related to parental income Regress wages on the residuals, obtaining the least squares estimate for This illustrates how reflects the effect of education on wages controlling for parental income: it is the relationship between wages and the part of education not linearly related to parental income.
Double residual regression The double residual regression is the three-step process: Regress on the set of other explanatory variables, obtaining residuals Regress on the set of explanatory variables excluding , obtaining residuals Regress on , estimating and Like the two-step process, this yields an identical coefficient to the full regression. It includes the additional feature that the residuals from the regression in step 3 equal the residuals in the full regression.
Multivariate definition Consider the least squares-estimated regression , where and are vectors of dependent variable observations and residuals, respectively, is an matrix of explanatory variables' observations, is an matrix of explanatory variables' observations, and and are and coefficient vectors for and , respectively. Then, the Frisch–Waugh–Lovell theorem states that
where , the residuals from the least squares regression of on , and , the residuals from the least squares regression of on . The first expression of is the residual regression, while the second is the double residual regression.
Geometric interpretation With a linear regression of the form , the fitted values can be interpreted as the orthogonal projection of onto the column space of , . The Frisch–Waugh–Lovell theorem is then (in the double residual regression case) the three step process: Project onto the orthogonal complement of , obtaining residuals Project onto the orthogonal complement of , obtaining residual vector Project onto , obtaining projection and residuals The resulting and residuals are identical to those in the full regression of on and .
Proof of equivalence between OLS and FWL solutions See proof of theorem 1 in Basu 2026.
First, split the matrix of explanatory variables (herein referred to as 'design' matrix). The split is between columns, so that the explanatory variables are split between matrices and . Then the design matrix is . Further, split up the regression coefficients:
Substitute the split design matrix and split regression coefficients into the normal equations:
Expand the equations by matrix multiplication:
1.
2.
Use row reduction (with matrices instead of scalars) to solve for . Do so by matching matrix coefficients between the terms, accomplished by multiplying the second equation by and distributing. Let , which is the projection matrix onto the column space of Z. The left-hand side and its reduction are shown.
The right-hand side reduces as follows:
So equation 2 becomes:
Let , the annihilator matrix with respect to the column space of Z. Subtract equation 2 from equation 1, then factor:
Let , the residualized X matrix where the columns of X are projected orthogonally off of the column space of Z. Recall that projection is idempotent and symmetric. These properties are used to rewrite the previous equation in the form of the normal equations, then in the form of a least squares solution:
The last line is the Frisch-Waugh-Lovell estimate of regression coefficients for variables in the matrix .
Construction of double residual regression form Consider the least squares-estimated regression and annihilator matrix . Premultiplying both sides of the regression equation by the annihilator matrix removes from and the part linearly explained by :
Where the third line follows from by construction and from , a property of least squares where is in the orthogonal complement of the column space of . Then, by the least squares solution, and . This concludes the construction of the double residual regression form of the Frisch-Waugh-Lovell theorem.
Proof of equivalence between single and double residual regression The residual regression – where is projected out of but not – returns a coefficient vector equal to the double residual regression. Their equivalence can be proved:
Where the intermediary steps follow from the idempotency and symmetry of the annihilator matrix.
History
Yule In 1907, statistician Udny Yule introduced a new system of notation for, and derived a number of algebraic results of, least squares-estimated regression coefficients. Among his results was an early form of the Frisch–Waugh–Lovell theorem. In Yule's notation, where represents the residuals from the regression of on through , and the residuals from the regression of on through , he finds that the regression of on yields the coefficient for in the full regression of on through . He notes that this relationship holds "quite generally and without reference to the form of the [variables'] frequency distribution." With this result, Yule defines the multiple regression coefficient – the coefficient on in the regression of on through – as the simple regression of on :
Having related simple regression coefficients to multiple regression coefficients, Yule describes his result as filling a gap in the interpretation of least squares coefficients and partial correlations by showing that they reflect "an actual correlation between determinate variables."
Using Yule's notation, in a 1968 text econometrician Arthur Goldberger states the residual regression form of the Frisch–Waugh–Lovell theorem as and the double residual regression form as .
Frisch and Waugh