Econometrics Economics

Why large-sample theory matters

Finite-sample unbiasedness is attractive but can rely on strong assumptions or may be difficult to establish in complex models. Asymptotic theory instead asks how the distribution of an estimator behaves as the number of appropriately sampled observations grows. Consistency means convergence in probability to the parameter; asymptotic normality characterises fluctuations around that limit on the √n scale. These properties are logically distinct: an estimator can converge to a wrong pseudo-true parameter and still have a normal limiting distribution around it.

In the multiple-regression setting, the OLS identity is β̂=β+(X′X)^-1X′u. Reorganising yields

If (1/n)X′X converges in probability to a finite nonsingular matrix Q=E[X_iX_i′] and (1/n)X′u converges to zero, then β̂ converges to β. The key condition is an orthogonality moment, E[X_iu_i]=0; conditional mean independence is a sufficient but stronger requirement in standard models.

Deriving consistency

The LLN makes sample averages approach their population counterparts. For identically distributed independent observations, suitable finite moments and nonsingularity imply

Thus E[Xu]=0 gives consistency, while E[Xu]≠0 typically produces a nonzero asymptotic bias. This expression connects large-sample estimation directly to the omitted-variable and endogeneity problems introduced earlier. A sample of one million observations can estimate the wrong population slope with exceptional precision if E[Xu] is not zero.

The asymptotic distribution and the sandwich variance

Let Ω=E[X_iX_i′u_i²]. Under independent observations, appropriate moment conditions and a CLT,

This is the general heteroskedasticity-robust asymptotic covariance formula for a correctly targeted linear moment estimator. The corresponding approximate finite-sample covariance is (1/n)Q^-1ΩQ^-1. In practice, replace Q and Ω by sample analogues and u_i by estimated residuals:

The superscript HC0 labels the basic Eicker–Huber–White form. HC1 multiplies it by an appropriate degrees-of-freedom adjustment, whereas HC2 and HC3 apply leverage corrections observation by observation. The preferred choice can matter with influential observations and smaller datasets.

Why robust standard errors do not fix biased coefficients

Ordinary OLS and heteroskedasticity-robust OLS use the same fitted coefficients. The robust procedure modifies the estimated variance of the estimator, not the identification strategy. If education correlates with omitted ability, calculating HC3 standard errors does not turn the schooling association into a causal return. Inference can be robust to heteroskedasticity while the point estimate remains inconsistent because of endogeneity.

Clustered and time-dependent observations

If students share classrooms, workers share firms or observations are repeated within countries, errors may be correlated within clusters. Treating clustered observations as independent can produce severe overconfidence. Cluster-robust covariance estimators aggregate score contributions within each independent cluster; the effective amount of information often depends more on the number of independent clusters than on the raw count of observations.

Let Xg and ûg contain observations from cluster g. An uncorrected cluster sandwich has the form

Practical procedures apply small-sample corrections and use cluster-aware reference distributions. With very few clusters, standard asymptotics may be unreliable; appropriate cluster-specific bootstrap methods or alternative research designs may be necessary. For time-series data, heteroskedasticity-and-autocorrelation-consistent (HAC) covariance estimation explicitly accounts for correlation across lags.

The delta method for nonlinear quantities

Economists often report a function g(β̂), such as an elasticity, a ratio of coefficients, a turning point or a cumulative dynamic effect. If β̂ is asymptotically normal and g is differentiable at the target, the delta method gives

For a ratio g(a,b)=a/b, the gradient is (1/b, −a/b²). Ratios become unstable when the denominator is near zero, so first-order delta approximations may perform poorly—an important warning for instrumental-variable estimators with weak first stages.

Practical robust testing

Use a robust covariance estimator consistent with the observational design, not simply a default option. HC standard errors assume independence across observations (though they permit unequal error variances), while cluster and HAC estimators target different dependence structures. The relevant regression output should disclose the number of observations, controls, cluster units and standard-error estimator.

fit <- lm(y ~ x1 + x2 + x3, data = dat)
library(sandwich)
library(lmtest)
coeftest(fit, vcov. = vcovHC(fit, type = "HC3"))
# Cluster-robust inference when 'firm_id' identifies independent firms:
coeftest(fit, vcov. = vcovCL(fit, cluster = ~ firm_id))

Robust variance and the hat matrix

The leverage of observation i in linear regression is h_i=x_i′(X′X)^-1x_i. In small samples, fitted residuals can systematically understate disturbances at high-leverage observations. HC2 replaces û_i² by û_i²/(1−h_i), while HC3 uses û_i²/(1−h_i)² inside the sandwich “meat”. These corrections can produce materially different standard errors where the design contains unusual X values.

No HC estimator repairs cross-observation dependence. For example, if 500 students come from 10 classrooms, an HC3 calculation still treats the rows as independent. With shared classroom shocks, clustering at the classroom level is more relevant. Ten clusters may be too few for conventional cluster-asymptotic t-reference approximations, so finite-cluster procedures or stronger design-based evidence may be required.

The meaning of asymptotic normality

A statement √n(β̂−β)→ᵈN(0,V) implies that sampling deviations become small at a 1/√n rate and approach a normal form after standardisation. It does not imply that β̂ is unbiased for each finite n, or that the sample-size threshold at which the approximation becomes good is universal. Near-singular regressors, heavy tails and weak instruments are examples in which finite-sample behaviour can diverge sharply from an appealing asymptotic formula.

A particularly important caution concerns ratios. A 2SLS estimator divides one random sample moment by another. If instrument relevance is weak, the denominator may be close to zero with nonnegligible probability, making the ratio distribution highly skewed or heavy-tailed. Treating this estimator as approximately normal can produce invalid confidence intervals even with many observations.

HAC covariance and the choice of a lag window

With time-series observations, the covariance of the regression score s_t=x_tu_t includes products across dates. A heteroskedasticity-and-autocorrelation-consistent estimator, abbreviated HAC, estimates their long-run covariance rather than retaining only same-date products. Newey–West combines sample score autocovariances up to a lag window L and downweights more distant lags; the Bartlett weight at lag ℓ is 1−ℓ/(L+1).

A very short window can miss persistent dependence, whereas a long window estimates many noisy autocovariances. Report the ordering of observations, bandwidth or lag rule, any prewhitening, and a meaningful sensitivity check. HAC does not correct endogenous regressors, unit roots or an omitted causal mechanism. For an appropriately specified stationary regression in R, sandwich::NeweyWest(fit, lag = 4, prewhite = FALSE) requests a four-lag Bartlett calculation; four is an illustration, not a universal choice. Clustered panels and a single ordered time series have different information structures, so a HAC option should not be substituted mechanically for clustering.