Econometrics Economics

Summation and matrix identities

In econometrics, Σ_i denotes summation over observed units. A vector is an ordered collection of values, a matrix is a rectangular array and the transpose X′ interchanges its rows and columns. The inverse (X′X)^-1 exists only when X′X has full rank. In OLS, fitted values are orthogonal projections, so P_x=P_x′ and P_x²=P_x; this idempotence explains many algebraic least-squares identities.

For ordinary scalar variables with finite moments, covariance is Cov(X,Y)=E[XY]−E[X]E[Y]. The population simple-regression slope minimising E[(Y−a−bX)²] is Cov(X,Y)/Var(X) whenever Var(X)>0. The numerator determines direction and the denominator rescales variation in X. Conditional expectation E[Y|X] minimises E[(Y−g(X))²] across all measurable functions g with appropriate finite second moments; linear regression restricts g to a linear class.

Deriving the simple-regression OLS formula

Start with S(a,b)=Σ(Y_i−a−bX_i)². The first-order condition in a is −2Σ(Y_i−a−bX_i)=0. Dividing by n gives Ȳ−a−bX̄=0, so a=Ȳ−bX̄. The first-order condition in b is −2ΣX_i(Y_i−a−bX_i)=0. Replace a by the previous expression, expand centred variables and rearrange to obtain b=Σ(X_i−X̄)(Y_i−Ȳ)/Σ(X_i−X̄)². The second derivative in the slope is positive if the X values are not all equal, ensuring a unique minimiser.

Deriving the variance of a two-variable linear combination

For constants a and b and random variables X and Z,

Expand the squared deviation from the mean and use linearity of expectation. This identity appears repeatedly in econometrics: the standard error of a linear combination of regression coefficients, the predicted effect of an interaction and the variance of a portfolio all depend on covariance terms. Assuming coefficients are uncorrelated without checking the variance-covariance matrix is a common mistake.

Deriving the OLS asymptotic variance

From β̂−β=[(1/n)ΣX_iX_i′]^-1(1/n)ΣX_iu_i, multiply by √n. Under regularity conditions, the matrix of regressors converges to Q and the score average satisfies a multivariate CLT, (1/√n)ΣX_iu_i→ᵈN(0,Ω). Slutsky's theorem gives √n(β̂−β)→ᵈN(0,Q^-1ΩQ^-1). This result makes clear both why standard errors typically shrink at the rate 1/√n and why heteroskedasticity affects Ω. It also clarifies that an asymptotic variance formula is not a claim of finite-sample unbiasedness.

Worked exercise: a fitted regression line

Problem. Observe X=(0,1,2,3) and Y=(1,2,2,5). Calculate β̂₁ and β̂₀, residuals and R².

Solution. X̄=1.5 and Ȳ=2.5. The centred cross-product sum is (−1.5)(−1.5)+(−0.5)(−0.5)+(0.5)(−0.5)+(1.5)(2.5)=6.0. The squared-X deviations sum to 5, so β̂₁=1.2 and β̂₀=0.7. Fitted values are (0.7,1.9,3.1,4.3), and residuals (0.3,0.1,−1.1,0.7). RSS=0.09+0.01+1.21+0.49=1.8. TSS=2.25+0.25+0.25+6.25=9, giving R²=1−1.8/9=0.80. The residuals add to zero as expected for OLS with an intercept.

Worked exercise: a joint hypothesis

Problem. An unrestricted model has RSS_u=120 with n=100 observations and k=4 nonconstant regressors. A restricted model imposes q=2 restrictions and has RSS_r=144. Compute the conventional partial F statistic.

Solution. The unrestricted residual degrees of freedom are 100−4−1=95. The fit penalty per restriction is (144−120)/2=12, while the unrestricted residual mean square is 120/95=1.2632. Therefore F=12/1.2632=9.50, to compare with the relevant F(2,95) critical value under the classical assumptions. If heteroskedasticity is present, this RSS-based reference distribution may be inappropriate; a robust Wald test is required.

Worked exercise: interpreting a log-level specification

Problem. A fitted regression gives ln(wage)=1.8+0.06education. Estimate the exact fitted wage ratio for an additional three years of schooling.

Solution. The change in fitted ln wage is 3×0.06=0.18. The ratio of exponentiated fitted log wages is exp(0.18)≈1.1972, a 19.72% difference; the first-order approximation is 18%. This is a ratio of conditional geometric means when the log conditional mean is correctly specified. It also represents the ratio of conditional arithmetic means only when the retransformation factor E[exp(u)|education] is unchanged across the comparison. A causal interpretation additionally requires exogeneity, which the fitted equation does not establish.

Worked exercise: omitted-variable bias

Problem. The true model is Y=β₀+2X+3Z+u, and Cov(X,Z)/Var(X)=−0.4. Find the population coefficient from a simple regression omitting Z.

Solution. β̃₁=2+3(−0.4)=0.8. The observed slope understates the direct X effect because the omitted factor raises Y while tending to be lower among observations with higher X. The sign analysis should precede arithmetic: a positive omitted effect multiplied by negative X–Z association produces negative bias.

Worked exercise: treatment selection

Problem. Treated and untreated average outcomes are 18 and 15. The treated population would have averaged 16 without treatment. Compute ATT and selection bias.

Solution. ATT=18−16=2. Selection bias=16−15=1. The observed contrast of 3 equals 2+1. Treating the raw difference as causal exaggerates the ATT by 50% in this example.

Worked exercise: an IV ratio

Problem. A randomly assigned programme offer increases training participation by 0.20 and average annual earnings by €360. Under relevance, exclusion, monotonicity and random assignment, interpret the Wald estimator.

Solution. The IV estimate is 360/0.20=€1,800. Under the assumptions and appropriate treatment definition, it identifies an average effect of actual training for compliers whose treatment status is changed by the offer. It does not directly estimate the effect for always-takers, never-takers or the whole eligible population unless stronger homogeneity assumptions hold.

Worked exercise: panel first differences

Problem. Three firms have (X₁,Y₁)=(1,5),(2,9),(4,12) and (X₂,Y₂)=(2,7),(4,12),(5,13). Compute the observed changes, and explain why regressing ΔY on ΔX can be useful.

Solution. ΔX=(1,2,1) and ΔY=(2,3,1). If the model has an additive time-invariant firm effect, that unobserved effect disappears when outcomes and regressors are differenced. A regression on changes therefore uses within-firm rather than between-firm information, but a causal interpretation still requires ΔX to be sufficiently exogenous with respect to Δu.

Worked exercise: the omitted covariance term

Problem. An interaction estimate has β̂₁=2, β̂₃=0.5, Var(β̂₁)=0.16, Var(β̂₃)=0.01 and Cov(β̂₁,β̂₃)=−0.02. Find the estimated treatment effect and standard error at X=3.

Solution. The effect is 2+3×0.5=3.5. Its variance is 0.16+9×0.01+2×3×(−0.02)=0.13, so SE≈0.3606. Ignoring the negative covariance would instead yield √0.25=0.5 and materially overstate the uncertainty for this particular contrast.

Further guided exercises

Exercise A. Derive why the OLS sample residual mean is zero when an intercept is included. Answer. The first-order condition for the intercept is Σû_i=0; divide by n. Exercise B. Show that multicollinearity does not itself bias OLS under zero conditional mean. Answer. Conditional expectation E[β̂−β|X]=(X′X)^-1X′E[u|X]=0 whenever the matrix is invertible; collinearity affects that inverse and therefore precision.

Exercise C. Explain why a positive first-stage effect and a negative reduced form imply a negative just-identified IV slope. Answer. Their ratio is negative; the exclusion argument is still needed for causal interpretation. Exercise D. Explain why predictive cross-validation does not establish IV exclusion. Answer. A model can accurately predict D or Y while its instrument changes Y through an independent causal channel; prediction checks the wrong assumption.

Exercise E. A random walk Y_t=Y_t₋₁+ε_t starts at Y₀=0 with i.i.d. innovations of variance σ². Answer. Y_t=Σ_s₌₁ᵗ ε_s and Var(Y_t)=tσ², so variance increases with t and the process is nonstationary. Exercise F. Explain why unit FE cannot separately estimate the coefficient of a variable constant over time within each unit. Answer. Such a variable vanishes after the within transformation and is collinear with unit dummies.

Checklist for evaluating an empirical claim

State the substantive question and the numerical estimand. Distinguish outcome, treatment or focal regressor, included controls and unobserved determinants. Check units, sample construction, missingness and dependence. Write the identification assumption explicitly and argue why it might hold. Verify whether the chosen estimator targets the estimand and whether standard errors reflect sampling structure. Calculate and communicate uncertainty, explain economically meaningful magnitudes, consider sensitivity and disclose limitations. If the identification argument fails, report the findings descriptively rather than giving them an unjustified causal interpretation.