Econometrics Economics

Linearity in parameters versus linearity in regressors

The classical least-squares formula estimates models linear in the unknown coefficients, not necessarily linear in the original data variables. A model with X², log(X), a treatment interaction or year indicators remains linear in parameters if these transformed regressors enter additively with linear coefficients. Conversely Y=exp(β₀+β₁X)+u is nonlinear in parameters and cannot be estimated as written by ordinary linear least squares without changing the model and error assumptions.

Functional form choices impose substantive statements about marginal effects. Adding a polynomial is not a purely cosmetic technique: it says that the relationship may bend in a systematic way. Using logs can create a proportional-effect interpretation, handle substantial skewness and express economic elasticities. A model must still be checked for sensible predictions, influential observations and any new restrictions imposed by the chosen transformation.

The main logarithmic models

For positive Y and X, four common forms are useful. Level–level: Y=β₀+β₁X+u has a constant β₁-unit marginal change in Y. Log–level: ln(Y)=β₀+β₁X+u has approximately a 100β₁-percent change in Y per unit X. Level–log: Y=β₀+β₁ln(X)+u has an approximate β₁/100-unit change in Y per 1% increase in X. Log–log: ln(Y)=β₀+β₁ln(X)+u has constant elasticity β₁, so a 1% increase in X corresponds to approximately a β₁% difference in Y.

When the dependent variable is logged, exponentiating the fitted conditional mean of ln(Y) does not generally reproduce E[Y|X]. Jensen's inequality implies that E[exp(log Y)|X] is not simply exp(E[log Y|X]). Forecasts in original units may require a retransformation or smearing correction, especially with heteroskedastic log residuals.

Quadratic and higher-order terms

A quadratic specification is

Its marginal effect is β₁+2β₂X. If β₂<0, the function is concave and a positive β₁ may eventually turn into a negative marginal effect. The turning point is X*=−β₁/(2β₂), if β₂≠0. One should report whether this point falls inside the observed data range before interpreting it economically. Arbitrarily high-degree polynomials may overfit, fluctuate sharply near boundaries and extrapolate poorly.

Interactions between two continuous regressors

Let E[Y|X,Z]=β₀+β₁X+β₂Z+β₃XZ. Holding Z fixed, the marginal effect of X is

The coefficient on X describes its effect at Z=0, which may lie outside the substantively interesting range. Centering Z around its mean makes β₁ refer to the average observed level of Z, improving interpretability. The interaction coefficient measures how X's marginal effect changes per unit increase in Z; its sampling uncertainty and the covariance between estimates matter when computing standard errors for a particular marginal effect.

Binary × continuous and binary × binary interactions

The model E[Y|D,X]=β₀+β₁D+β₂X+β₃(DX) permits different slopes in treated and untreated groups. For D=0 the slope in X is β₂; for D=1 it is β₂+β₃. The treatment-control contrast at X=x is β₁+β₃x, not simply β₁. The standard error of this contrast uses Var(β̂₁)+x²Var(β̂₃)+2xCov(β̂₁,β̂₃).

With two binary variables A and B, adding A×B allows the group possessing both characteristics to differ from the sum of their separate main effects. In a fully interacted model with two binary indicators, fitted means match the four group-specific means. This is an example of a saturated linear model. The same algebra underpins the simplest difference-in-differences specification discussed in the panel-data chapter.

Flexible regression and partial interpretation

A statistical model can flexibly approximate the conditional mean without identifying a causal mechanism. Adding many transformations does not eliminate omitted confounders or endogeneity. An estimated nonlinear regression also requires evaluating the relevant marginal effect at economically appropriate values, accompanied by uncertainty intervals, rather than declaring the coefficient on the original term to be the whole effect.

The multivariate marginal-effect formula

For a smooth conditional mean m(x)=E[Y|X=x], the marginal effect of X_j is ∂m(x)/∂x_j. For a discrete or binary treatment, derivatives can be inappropriate; instead calculate the change in predicted outcome between treatment states, holding the remaining covariates fixed. Average marginal effects average unit-specific partial effects across the observed sample; marginal effects at the mean evaluate a nonlinear function at a particular artificial covariate vector. These quantities differ because nonlinear functions do not generally commute with averaging.

If m(X)=β₀+β₁X+β₂X², the derivative β₁+2β₂X is individual-specific. The average marginal effect is β₁+2β₂X̄ in this quadratic case, but more complex functions may require averaging calculated effects over all observations. When models contain multiple interacted covariates, state exactly which are held constant and where the derivative is evaluated.

Inference for turning points and elasticities

The turning point of a quadratic, X*=−β₁/(2β₂), is a ratio of estimated coefficients. If β₂ is imprecisely estimated near zero, this ratio can have a highly skewed distribution; the standard delta method may be unreliable. A graph of the estimated function with confidence bands often communicates uncertainty more responsibly than reporting a single turning-point value. For a predicted effect L(β̂) involving multiple coefficients, incorporate the entire estimated covariance matrix.

Consider a log–log model with estimated slope 0.75 and robust SE 0.12. A simple 95% interval for the elasticity is approximately 0.75±1.96(0.12), or (0.515,0.985). This interval characterises sampling uncertainty in the coefficient under the model, not whether a policy-induced price change has the same effect as a historical market association.

From log predictions to outcomes in original units

A fitted log model describes m(x)=E[ln(Y)|X=x]. Its exponent exp(m(x)) is a conditional geometric mean, and is a conditional median when the log disturbance has conditional median zero. To obtain the conditional arithmetic mean, write Y=exp(m(X))exp(u) and average the second factor conditional on X. The resulting correction can vary across covariate values; exponentiation alone generally understates the conditional mean.

If u conditional on X is normal with constant variance σ², the correction is exp(σ²/2). With a constant error distribution but without normality, a smearing estimate uses the sample mean of exp(û_i). A single global smearing factor is generally insufficient when the distribution of u changes with X. For example, if m(x)=2 and σ²=0.25, exp(2) is about 7.39 but the conditional mean is exp(2.125), about 8.37. Multiplicative comparisons between two covariate values cancel the correction only if it is identical at those values.

Binary outcomes: linear probability, logit and probit

For a binary outcome, the conditional mean is a probability. A linear probability model uses p(x)=x′β and interprets a coefficient of 0.04 as a four-percentage-point change, holding other regressors fixed. The model is simple but can predict below zero or above one, and its error variance p(x)[1−p(x)] is generally heterogeneous.

Logit uses the logistic distribution function Λ and probit uses the standard normal distribution function Φ. Both keep fitted probabilities between zero and one. With independent observations, maximum likelihood chooses coefficients to maximise the Bernoulli log-likelihood shown below, where p_i is the fitted probability and y_i is zero or one. A logit coefficient describes a change in log odds, not a percentage-point change. For an untransformed continuous regressor without interactions, its probability marginal effect is β_jp_i(1−p_i); the probit analogue is β_jφ(x_i′β), where φ is the standard normal density. For binary regressors, compare predicted probabilities at zero and one and average these differences if the estimand is an average effect. These models still require a credible identification argument for causal claims.