Econometric Analysis of Microdata

Indicators, categories and the comparison they encode

A binary or dummy variable takes the value 1 when a defined condition holds and 0 otherwise. Employment status, an apartment’s ocean view and the presence of high blood pressure are examples. Binary variables are a special case of categorical variables, which classify observations into a limited set of groups. The coding must be stated explicitly: reversing 0 and 1 reverses the interpretation of the associated comparison.

A numeric category code is not necessarily a quantitative measurement. Coding blood types A, B, AB and O as 1, 2, 3 and 4 does not make type O “four times” type A, nor establish equally spaced biological differences. Treating these codes as one continuous regressor would impose an arbitrary ordering and constant steps. Separate indicators allow each group to have its own conditional mean relative to a chosen reference category.

There are two distinct uses of binary variables in regression. An indicator on the right-hand side represents a group comparison while leaving the nature of the outcome unchanged. An indicator on the left-hand side makes the conditional mean a probability, introducing restrictions that an ordinary straight line may fail to respect. The estimation principles remain connected, but the interpretation and appropriate diagnostics differ.

A dummy variable as an explanatory variable

Suppose monthly apartment rent, in currency units, is modelled using the number of rooms and an ocean-view indicator:

For a fixed room count, the fitted rent is ${\beta }_0+{\beta }_{1}Rooms$ without an ocean view and ${\beta }_0+{\beta }_{1}Rooms+{\beta }_2$ with a view. Thus ${\beta }_2$ is the conditional difference in monthly rent, measured in the same currency units as rent. The model assumes that this difference does not vary with the number of rooms. Adding $Rooms\times View$ would relax that restriction, allowing the view premium to vary with apartment size.

“Holding rooms constant” does not mean all other apartment characteristics are identical. If ocean-view properties also have better locations or higher construction quality and those variables are omitted, the coefficient may reflect those differences. The group indicator does not supply random assignment. Its coefficient is a conditional association unless the identifying assumptions justify more.

If the dependent variable is log rent, the coefficient changes meaning: the exact fitted proportional difference between view and no-view apartments is $100(e^{{\beta }_2}-1)$%, subject to the logged-outcome qualifications in Chapter 5. If an indicator interacts with another regressor, its standalone coefficient applies when that other regressor is zero, and the full group contrast must include the interaction.

Multiple categories and the dummy-variable trap

Consider a school-district example that groups computer availability into low, middle and high categories using the 25th and 75th percentiles. To define a complete classification, assign values below the 25th percentile to low, values from the 25th through the 75th percentile to middle, and values above the 75th percentile to high. Ties require a consistent boundary rule; the groups need not contain exactly 25%, 50% and 25% of observations.

Let $D_M$ and $D_H$ indicate middle and high availability. With low availability as the reference, estimate:

The intercept is the low group’s mean, and the two slopes are differences relative to it. Including all three group indicators together with an intercept produces perfect multicollinearity because their sum is exactly 1. This is the dummy-variable trap. Omitting one category with an intercept solves it; alternatively, omitting the intercept and retaining all categories estimates separate group means. Neither option discards observations from the reference group.

The fitted means are approximately 653.96 points for low availability, 654.82 for middle availability and 653.04 for high availability. Both difference intervals include zero. The joint test of the two category coefficients gives $F(2,417)=0.31$, with $p=0.7319$; $R^2=0.0015$, adjusted $R^2=-0.0033$, and the root mean squared error is 19.085 score points. These results provide little evidence of differences in this simple grouping. They do not prove that access to computers has no causal effect: the specification is an unadjusted group comparison, and the measure is computer availability rather than a randomly assigned intervention.

Binary outcomes and the linear probability model

For a binary outcome $Y$, its conditional expectation equals its probability of being 1: $E[Y\mid X]=P(Y=1\mid X)$. Running OLS with a binary outcome therefore gives a linear probability model, or LPM. In $P(Y=1\mid X)={\beta }_0+{\beta }_{1}X$, a one-unit increase in $X$ changes the fitted probability by ${\beta }_1$, or $100{\beta }_1$ percentage points. The scale is essential: a coefficient of 0.05 means five percentage points, not automatically a 5% relative increase.

For school districts, define $HighTest=1$ when the district’s average test score is above the chosen overall average and 0 otherwise. A regression on average district income produces the following results. Income is kept in the dataset’s recorded units, so the slope applies to one unit on that scale; any currency conversion must be applied before interpreting a different unit change.

There are 420 observations; $R^2=0.2253$, root mean squared error 0.44108, and the robust overall test is $F(1,418)=125.81$, with $p<0.001$. A one-unit income increase is associated with about 3.288 percentage points higher probability of an above-average score. This comparison is about crossing a binary threshold, not the number of test-score points gained.

Probability bounds, heteroskedasticity and percentage points

A straight line need not stay between 0 and 1. The fitted school model is $\hat{p}=0.0034945+0.0328825I$, which exceeds 1 when recorded income exceeds approximately 30.31. At $I=40$, it predicts about 1.319, an impossible probability. Such predictions reveal a limitation of the functional form; merely truncating them to 1 does not change the model that was estimated or provide a principled marginal-effect calculation.

The LPM also imposes a constant probability slope. An income increase has the same fitted effect near a probability of 0.05 as near 0.95, although the room for further decreases or increases is very different. The importance of this restriction depends on the range of covariates and the purpose of the analysis. A linear approximation may be useful over a central range even if it behaves badly in the tails.

With a correctly specified conditional probability $p\left(X\right)$, the binary error variance is $Var(u\mid X)=p\left(X\right)[1-p\left(X\right)]$. It therefore generally varies with the regressors, making heteroskedasticity-robust standard errors appropriate. Robust inference addresses this variance problem; it does not repair impossible probabilities, omitted variables or an incorrect conditional mean. If observations are dependent within groups, the standard-error calculation must address that dependence as well.

Percentage points and percent changes answer different questions. A probability moving from 40% to 50% increases by 10 percentage points and by 25% relative to its starting value, because $(0.50-0.40)/0.40=0.25$. In a group of fixed size 100, moving from 40 to 50 members with a characteristic illustrates both statements. Without a baseline probability, an LPM coefficient cannot be converted into a unique relative percentage change.

Probit, logit and maximum likelihood

Probit and logit replace the unrestricted probability line with a smooth cumulative distribution function. With an index $z_i={\beta }_0+X_i\prime \beta$, the probit model uses the standard normal cumulative distribution function $\Phi$:

The function $\Phi \left(z\right)$ is the area under the standard normal density to the left of $z$. It maps every real-valued index into a probability strictly between 0 and 1 for finite values. The logit model uses the logistic function instead:

Both curves flatten near their lower and upper bounds. The raw coefficients describe changes in an index, not percentage-point changes in probability. For a continuous regressor without interactions, the probit marginal effect is $\varphi \left(z_i\right){\beta }_j$, where $\varphi$ is the standard normal density; the logit marginal effect is $p_i(1-p_i){\beta }_j$. Their magnitude depends on all covariates through the index. An average marginal effect averages these observation-specific derivatives, while an effect evaluated at average covariates is a different quantity.

For a binary regressor, calculate the discrete difference between predicted probabilities when its value is set to 1 and to 0, holding other covariates fixed. Interactions require still more care: the sign of a coefficient on a product term need not by itself equal the sign of every probability-scale interaction effect. Presenting fitted probabilities or explicitly calculated contrasts is often clearer.

These models are usually estimated by maximum likelihood rather than OLS. For independent observations with probabilities $p_i\left(\beta \right)$, the binary log-likelihood is:

Maximisation selects the parameter values that give the observed outcomes the greatest likelihood within the specified model. Probit, logit and LPM predictions can be similar over a central range, but this is not guaranteed, especially in the tails. Choose between them by considering the target comparison, predicted probability range, specification and interpretability. A nonlinear probability model supplies sensible bounds, not a causal research design.

Open figure at full size ↗

Reading a complete regression output

A coefficient table contains several distinct pieces of information. The coefficient gives an estimated conditional relationship in the model’s units. Its standard error measures sampling uncertainty under the chosen variance estimator. A test statistic compares the estimate with a hypothesised value, usually zero; the p value describes compatibility with that null under the testing assumptions. A 95% confidence interval summarises a range generated by a procedure with 95% long-run coverage. Neither the p value nor the interval measures the probability that a particular causal story is true.

The number of observations states the sample actually used. $R^2$ is the fraction of sample outcome variation explained by the fitted regression; adjusted $R^2$ penalises added regressors and may be negative. Root mean squared error expresses residual spread in the dependent variable’s units. The overall F test assesses a joint restriction on slope coefficients. None of these statistics substitutes for substantive effect sizes, a valid research design or a careful description of the sample.

Logged-outcome comparisons refer directly to fitted log values and their implied geometric means; arithmetic-mean comparisons require the retransformation conditions discussed in Chapter 5. Every interpretation also holds the other included regressors fixed and remains associational unless additional assumptions establish causality.

Worked example: annual medical expenditure in levels

Three models use 2,970 observations and the same explanatory variables: age in years, the natural logarithm of personal income, and an obesity indicator equal to 1 for obese individuals and 0 otherwise. The first outcome is total annual medical expenditure measured in 2005 US dollars. Its coefficients therefore combine three interpretation rules: a level–level age effect, a level–log income effect and a discrete group difference.

Holding income and obesity status fixed, an additional year of age is associated with approximately USD 202.11 higher annual expenditure. A 1% income increase corresponds to approximately USD 2.60 lower expenditure, because the coefficient multiplies the change in log income; the exact fitted change is $-260.2222ln\left(1.01\right)$. The income estimate is very imprecise and its interval includes substantial positive and negative values, so the negative point estimate should not be presented as established evidence that income reduces expenditure.

At the same age and income, the obese group has fitted annual expenditure USD 1,251.30 higher. Its interval ranges from approximately USD 97.56 to USD 2,405.04. The intercept describes a non-obese individual at age zero and log income zero, meaning income equal to one in the recorded currency unit. This combination is generally not substantively representative; a negative intercept does not imply negative expenditure for a typical person.

The regression has $R^2=0.0082$, adjusted $R^2=0.0072$ and root mean squared error 14,058 USD. The joint test gives $F(3,2966)=8.20$, with $p<0.001$. Statistical evidence of some association can coexist with low explanatory power and large individual prediction errors. The estimates do not isolate causal effects of ageing, income or obesity without further assumptions about omitted health conditions, selection and other determinants of expenditure.

Worked example: medical expenditure in logarithms

The second model uses the natural logarithm of annual expenditure. It therefore concerns proportional expenditure differences and requires positive expenditure observations for the logarithm to be defined. Although the reported sample size remains 2,970, one should never assume that an arbitrary dataset contains no zero expenditure merely because a logged specification is convenient.

The age coefficient corresponds to approximately 4.19% higher expenditure per additional year, or an exact fitted geometric-mean difference of $100(e^{0.0419183}-1)\approx 4.28$%. The income coefficient is an elasticity: a 1% income increase is associated with approximately 0.170% lower expenditure, holding age and obesity status fixed. The exact finite-change figure is about −0.169%. Its confidence interval includes zero and the p value is 0.101, so the data do not establish a nonzero income association at the conventional 5% level.

The obesity coefficient is large enough that the common small-coefficient approximation is misleading. The exact fitted difference is $100(e^{0.420106}-1)\approx 52.21$%, rather than 42%. This compares conditional geometric means. Describing it as a 52.21% difference in arithmetic mean expenditure would additionally require an appropriate retransformation assumption or calculation. The intercept is the fitted log expenditure for the baseline covariate combination, not expenditure in dollars.

The model reports $R^2=0.0232$, adjusted $R^2=0.0222$, root mean squared error 1.8836 log units and $F(3,2966)=23.48$, with $p<0.001$. Its $R^2$ cannot be compared directly with Model 1’s to decide which better predicts dollar expenditure, because the dependent variable and residual scale have changed.

Worked example: the probability of high blood pressure

The third outcome equals 1 if an individual has high blood pressure and 0 otherwise. OLS therefore estimates an LPM. The age and obesity coefficients describe probability differences; the income coefficient combines a probability outcome with a logged regressor.

An additional year of age is associated with $100\times 0.0053784=0.53784$ percentage points higher probability. A 1% income increase changes the fitted probability by approximately $0.01\times 0.0914678=0.000914678$, or 0.0914678 percentage points. Using the exact log change gives about 0.0910135 percentage points. Multiplying by 100 without also accounting for the 0.01 log change would overstate this comparison by a factor of 100.

The obese group has an estimated probability 19.87462 percentage points higher, conditional on age and income. This is not a 19.87% relative increase: the relative figure would require a specified baseline probability. The negative intercept illustrates again why a baseline outside the useful data range is not a plausible standalone probability. Predictions should be examined over the actual covariate values.

The output reports $R^2=0.0432$, adjusted $R^2=0.0423$, root mean squared error 0.46012 and $F(3,2966)=44.67$, with $p<0.001$. The displayed standard errors are conventional OLS standard errors, unlike the explicitly robust school-district example. Given the binary outcome, a sound empirical analysis should recompute heteroskedasticity-robust or otherwise appropriate uncertainty before relying on these reported tests. The coefficient interpretations remain as stated, but their statistical precision must be assessed using an adequate variance estimator.