Econometrics Economics

A general approach to hypothesis testing

Testing starts with a population-level hypothesis. The null may claim that education has zero conditional association with wages, that a policy affects neither employment nor investment, or that two groups have the same regression slope. The hypothesis must be stated before interpreting results so that the number of restrictions, relevant estimators and reference distribution can be chosen appropriately.

For a single linear coefficient, H₀: β_j=β_j,₀, the usual test statistic is

When the standard error consistently estimates sampling uncertainty, t is often approximately N(0,1) under the null in large samples. Exact Student t inference additionally requires the relevant normal and classical linear-model assumptions. The p-value measures how surprising the observed statistic would be if the null and its sampling assumptions were true.

Confidence intervals and the link with two-sided tests

An approximate 100(1−α)% interval for β_j is β̂_j ± z₁₋α/₂ SE(β̂_j). At the 5% level, the two-sided test H₀:β_j=0 rejects exactly when zero is outside its corresponding two-sided 95% interval, provided both use the same SE and critical distribution. The interval displays the range of parameter values not rejected by the corresponding procedure and provides more information about possible economic magnitudes than a binary significant/nonsignificant statement.

Joint restrictions and why separate t-tests are insufficient

A model may contain a quadratic polynomial or a set of industry indicators whose joint effect is of interest. Testing each coefficient individually at 5% does not test the set of null restrictions at a 5% family level. The appropriate joint null can be written

where R is a q×(k+1) matrix identifying the q restrictions and r is a q-vector of hypothesised values. A general Wald statistic measures how far Rβ̂ is from r relative to the joint covariance of the estimates:

For asymptotic tests, W converges to a χ² distribution with q degrees of freedom when the null holds and suitable regularity assumptions apply. Many packages report F=W/q and use an appropriate reference distribution. A heteroskedasticity-robust V̂ should be used when constant error variance is not justified.

The restricted versus unrestricted F-test

Under the classical normal homoskedastic model, one can compare a restricted regression satisfying the null to an unrestricted regression that includes all parameters. The standard partial-F statistic is

Restrictions cannot reduce the minimal RSS, so RSS_r≥RSS_u. The numerator quantifies deterioration in fit per imposed restriction; the denominator estimates disturbance variance from the unrestricted model. Under the null hypothesis and the classical normal, spherical-error assumptions, F follows the central F distribution with q and n−k−1 degrees of freedom. Under a false restriction it generally has a noncentral F distribution. This convenient RSS formula is not generally heteroskedasticity-robust; use the robust Wald formulation when necessary.

A joint F-test on two coefficients can reject even when neither individual t-test rejects, because the estimates' covariance matters. Conversely several isolated significant t-tests are not automatically evidence against a carefully defined joint restriction after accounting for multiple comparisons.

Tests of linear combinations and interactions

Suppose Y=β₀+β₁D+β₂X+β₃DX+u. The treatment contrast at X=x is L=β₁+xβ₃. Its estimate is L̂=β̂₁+xβ̂₃ and its variance is

Ignoring the covariance term can materially distort the standard error. The same principle applies to sums of coefficients representing total dynamic effects, post-policy effects and returns at a specific value of a moderating variable.

Multiple testing and substantive interpretation

When researchers examine many outcomes, control sets or subgroups and report only the most significant results, nominal p-values can seriously overstate evidence. Adjusting familywise error rates by Bonferroni or using false-discovery-rate procedures may be appropriate for multiple exploratory tests. More generally, transparent research design, reporting complete specifications and distinguishing preregistered confirmatory analysis from exploratory analysis reduces misleading claims.

Power, statistical significance and scale

Statistical power is the probability of rejecting the null for a specified alternative. Larger independent samples and lower noise increase power for a fixed effect, whereas high regressor collinearity, limited treatment variation and correlated errors can reduce it. An insignificant estimate does not prove an effect is exactly zero; it may reveal limited information. A very narrow interval around a small effect can demonstrate that large effects are implausible, which may be more useful than a binary test decision.

A systematic derivation of the partial F statistic

Let the unrestricted model estimate k slope coefficients and an intercept, with unrestricted residual sum of squares RSS_u. Imposing q independent linear restrictions produces RSS_r, with RSS_r≥RSS_u because the restricted candidate set is smaller. Under the null restrictions and the spherical normal model, the extra sum of squares divided by σ² is central χ² with q degrees of freedom, independent of RSS_u/σ², which is χ² with n−k−1 degrees of freedom. The ratio of these mean squares therefore has an F(q,n−k−1) distribution. Normality and spherical errors support the exact reference distribution and independence of these quadratic forms. The sample identity TSS=ESS+RSS follows instead from OLS orthogonality with an intercept and needs neither homoskedasticity nor stochastic independence.

The robust Wald test instead uses the estimated covariance of Rβ̂. This generalises beyond the classical covariance formula and allows tests whose restrictions mix several coefficients. A practical R model comparison can report both classical ANOVA and robust Wald results, but the scientifically appropriate one depends on whether the conventional variance assumptions are defensible.

Misreading a nonsignificant coefficient

Suppose a minimum-wage effect on employment is estimated at −0.5% with an approximate 95% interval from −7% to +6%. Calling the result “no effect” is not justified: the data may be compatible with both economically important employment losses and gains. If the corresponding interval were (−0.7%,−0.3%), the effect would be more precisely estimated but perhaps economically small. The width and substantive range of a confidence interval frequently provide more policy information than its relation to an arbitrary zero threshold.