10. Double Selection and High-Dimensional Controls
Post-Lasso inference for a treatment coefficient with many candidate controls
The problem that motivates double selection
Standard low-dimensional regression works when the set of meaningful controls is modest compared with the sample size and the researcher can prespecify a credible adjustment set. Contemporary economic datasets may contain hundreds or thousands of candidate controls, transformations and interactions. If the number of regressors is large relative to n, ordinary least squares can be unstable or impossible to estimate with a unique solution. Purely predictive variable selection is tempting, but selecting controls only because they predict the outcome can jeopardise inference on the treatment effect.
The central model is a partially linear relationship
where D is a treatment or focal regressor, X contains potentially many pretreatment controls and g0 is a high-dimensional nuisance function. The parameter α0 is the target. An auxiliary treatment equation relates D to controls:
The second relationship matters because some controls may be weak predictors of Y but strong predictors of D. Omitting such variables from the outcome model can bias the treatment coefficient when the omitted variables also affect Y.
Why outcome-only selection can be misleading
Imagine Xj has a modest association with test scores but a very strong association with district class size. Selecting variables by their power to predict test scores alone may discard Xj, although it is a confounder relevant to the treatment assignment process. Conversely, selecting only by predictors of D may ignore major determinants of Y that help improve precision. A high-dimensional procedure should protect against both problems instead of treating predictive accuracy in one equation as sufficient.
In a low-dimensional model, the classical omitted-variable bias formula warns that leaving out a confounder requires both a link to D and a link to Y. Double selection addresses this dual requirement systematically by selecting controls from both relationships.
Lasso as a screening device
The Lasso minimises squared errors plus an L1 penalty on regression coefficients:
The tuning parameter λ controls the degree of shrinkage. Because the absolute-value penalty creates corners, coefficients of some predictors can be set exactly to zero, generating a selected set of controls. Standardising predictors before penalisation is usually necessary so that coefficient penalties are comparable. The treatment D is not penalised in the final causal parameter regression; it is the variable whose effect we seek to estimate.
The double-selection algorithm
In a standard partially linear implementation, first run Lasso of Y on the controls X and collect the controls selected for outcome prediction, Sᵧ. Second run Lasso of D on X and collect the controls selected for treatment prediction, Sᴅ. Form their union, S=Sᵧ∪Sᴅ, and estimate an ordinary linear regression of Y on D and all controls in S. The coefficient on D is the post-double-selection estimator of α0.
The union is essential: selecting by either equation alone omits a class of potential confounders. The method is particularly meaningful when there is an approximately sparse representation of the nuisance relationships, appropriate regularity conditions, and a sufficient effective sample size. It does not overcome endogenous D caused by genuinely unobserved confounders; it can only adjust for information represented among the candidate controls.
Orthogonalisation and the double-machine-learning connection
A related technique residualises the outcome and treatment on X and regresses one residual on the other. Let mᵧ(X)=E[Y|X] and mᴅ(X)=E[D|X]. Define residuals Ỹ=Y−mᵧ(X) and D̃=D−mᴅ(X). Under the partially linear model, α0 can be estimated using the moment equation
This moment is Neyman-orthogonal with respect to suitably defined nuisance perturbations: small errors in estimating the nuisance functions have no first-order effect on the target moment at the truth. Cross-fitting—estimating nuisance functions on folds not used for the corresponding residual-based target regression—reduces overfitting bias and is especially important with flexible learners. Double selection and partialling-out double machine learning share an identification framework, but their algorithms and regularity conditions should not be casually conflated.
Conditions, inference and limitations
Post-selection confidence intervals cannot generally be constructed by pretending that a naive single-equation variable-selection step did not take place. Valid double-selection results require assumptions on approximate sparsity, penalty selection, regressor moments, restricted eigenvalues or related design conditions, and suitable rates for nuisance estimation. The exact theorem depends on the estimator and the sample design. Robust or clustered inference must accommodate the actual dependence structure.
If X contains variables caused by D, conditioning on them may remove part of the causal effect or induce collider bias. Thus even a mathematically sophisticated selection algorithm does not choose a causal graph for the researcher. The controls' economic and temporal meaning remains indispensable.
Practical implementation in R
The following code is a schematic implementation for a complete-case dataset containing y, d and numeric controls x1 to x5. All variables must use the same retained rows before model.matrix and cv.glmnet are called. It uses a penalty chosen by cross-validation for illustration; the displayed summary is a software check, not a validated post-selection inference procedure. Published inferential implementations may use theoretically motivated penalty loadings, cross-fitting or dedicated double-selection packages.
library(glmnet)
X <- model.matrix(~ x1 + x2 + x3 + x4 + x5 - 1, data = dat)
fit_y <- cv.glmnet(X, dat$y, alpha = 1)
fit_d <- cv.glmnet(X, dat$d, alpha = 1)
sel_y <- which(as.vector(coef(fit_y, s = "lambda.min"))[-1] != 0)
sel_d <- which(as.vector(coef(fit_d, s = "lambda.min"))[-1] != 0)
keep <- sort(unique(c(sel_y, sel_d)))
final <- if (length(keep) == 0L) lm(dat$y ~ dat$d) else lm(dat$y ~ dat$d + X[, keep, drop = FALSE])
summary(final)For production work, verify variable mapping and intercept handling, inspect model selection stability, use appropriate heteroskedasticity-robust or clustered standard errors, and consult an inferentially validated implementation rather than assuming the illustrative code constitutes a proof of asymptotic coverage.
Double selection compared with ordinary stepwise regression
Traditional stepwise regression selects variables through sequential significance tests or fit criteria. The procedure often neglects the special inferential status of the focal treatment coefficient and can produce unstable selected models with invalid naive post-selection p-values. Double selection instead builds on the omitted-confounder logic: include controls that predict the outcome or treatment and then estimate the target parameter after taking the union of the selected adjustment variables.
It does not follow that every variable chosen by a machine-learning algorithm is a valid pretreatment control. Economic theory and the temporal ordering of events determine which features can be candidates. If a firm responds to a subsidy by increasing employment and wages, post-subsidy employment should not automatically be controlled for when measuring the subsidy’s total effect on wages, even if an algorithm deems it a powerful predictor.
When sample splitting is useful
In double machine learning, dividing the sample into folds prevents nuisance predictions for observation i from being estimated using that observation's outcome. Each fold's treatment and outcome residuals are constructed with nuisance models trained on other folds. The resulting orthogonal moment reduces first-order sensitivity to regularisation bias under suitable convergence rates. Cross-fitting can improve robustness to overfitting but cannot overcome a failure of conditional ignorability or the absence of overlap.
If the treatment is continuous, the conditional expectation mᴅ(X) is a regression function; for binary treatment it is the propensity score. The same residualisation logic leads to a partialling-out estimator, but the modelling strategy and inference must match the treatment type and statistical design.