The Mechanics of Omitted Variable Bias: Bias Amplification and Cancellation of Offsetting Biases

Bias amplification

Omitting U but including IV in the regression model, Yˆ=γˆ+τˆZ+αˆIVIV, also results in bias [17]:

However, conditioning on IV amplifies any bias left due to an unblocked backdoor path because 0<1−αIV2<1. Thus, the absolute OVB after adjusting for IV is always greater than the absolute initial OVB: αUβU1−αIV2>αUβU. If we were to condition on U in addition to IV (in case U would be observed), no OVB would be left because U blocks the backdoor path Z ← U → Y. Thus, if all confounders (or at least a set of variables that blocks all backdoor paths) are reliably measured, conditioning on an IV does not result in any OVB because there is no bias left to be amplified (provided the functional form of the regression is correctly specified). However, adjusting for the IV still reduces the efficiency of the treatment effect estimate [21, 23].

Imbalance in the unobserved confounder U

Bias amplification occurs because conditioning on the IV increases the imbalance in the unobserved confounder U. For our linear framework, we define imbalance as the difference in the expected value of U for subpopulations with Z = z and Z = z + 1 (if Z would be dichotomous the imbalance would measure the mean difference between the two groups). That is, without conditioning on the IV or any other covariate the imbalance in U is obtained by regressing U on Z: Imbalance(U|{})=E(U|Z=z+1)−E(U|Z=z)=αU. After conditioning on IV, we get Imbalance(U|IV)=E(U|Z=z+1,IV)−E(U|Z=z,IV)=αU1−αIV2 (Proof 1 in Appendix C). The comparison of the two imbalance formulas reveals that conditioning on the IV amplifies U’s imbalance by the factor 1/(1−αIV2). Thus, we can write the OVB as the product of the amplified imbalance in U and U’s direct effect on the outcome: OVB(τˆ|IV)=αU1−αIV2×βU. This formula highlights that conditioning on IV turns U into a relatively stronger confounder.

The increased imbalance in U can be explained as follows (similar explanations can be found in [21], and [24]): Since Z=αIVIV+αUU+εZ is a function of IV, U, and the error term εZ, conditioning on the IV removes IV’s effect on Z such that the remaining variation in Z is determined by U and the error term alone. With only two sources of variation left (U and εZ), U now explains a larger portion of variance in Z. Hence, the association between U and Z for a given IV = v is necessarily greater than before conditioning on IV. The increased association between U and Z implies an increase in U’s absolute imbalance: Imbalance(U|IV)=αU1−αIV2>Imbalance(U|{})=αU. Appendix A contains a more intuitive explanation within the context of matching or stratifying treatment and control cases on an IV.

Reliably measured confounder X

Adjusting for a reliably measured confounder X results in a biased regression estimator with

A comparison of this bias formula (Proof 3 in Appendix C) with the initial OVB indicates that conditioning on X has two effects: First, a bias-reducing effect because X blocks the backdoor path Z ← X → Y and thus eliminates its own confounding bias (αXβX). Second, a bias-increasing effect because the bias due to the unblocked backdoor path Z ← U → Y (αUβU) is amplified by the factor 1/(1−αX2).

If the bias-increasing effect dominates the bias-reducing effect then conditioning on X leads to an increase in the absolute OVB, that is, the OVB after conditioning on the confounder X is greater than without conditioning on X: αUβU1−αX2>αXβX+αUβU. The discussion of the conditions under which the absolute OVB actually increases requires a distinction between the case where X and U induce bias in the same direction (no offsetting biases) and where they induce bias in different directions such that their respective confounding biases partially or fully offset each other.

Biases in the Same Direction. If both confounders induce bias in the same direction, sgn(αXβX)=sgn(αUβU), then conditioning on X results in an increasing OVB only if the bias-amplifying effect dominates the bias-reducing effect, which is the case if ^[4]

Conditioning on X very likely increases the absolute OVB in two situations. First, if the bias induced by U (αUβU) is much larger than the bias induced by X (αXβX), implying that the bias ratio on the left-hand side in (4) is large. And second, if X strongly determines Z (αX close to 1) such that the right-hand side in (4) is close to zero. Thus, adjusting for a confounder with αX close to 1 and βX close to zero (i. e., a near-IV) very likely increases the absolute bias.

In the upper left plot of Figure 3 the two dark grey areas show combinations of αX values and bias ratios αUβUαXβX for which the absolute OVB increases. The two light grey areas indicate areas of decreasing OVB. The line separating the dark and light grey areas represents the 100% bias contour line where conditioning on X neither reduces nor increases OVB (i. e., 100% of the initial OVB is left). The darker shade of the two dark grey areas indicates the region where conditioning on X leads to a bias that is at least twice as large as the initial bias. Thus, the contour line that separates the two dark grey areas represents the 200% bias contour line. Similarly, the very light grey area indicates that less than 50% of the initial bias is remaining. The contour line separating the two light grey areas represents the 50% bias contour line. For example, conditioning on a confounder with αX=.1 results in an increasing OVB only if the bias ratio αUβUαXβX is greater than 1−.12.12=101, that is, if the bias induced by the unobserved confounder U is at least 101 times greater than the bias induced by X. However, if X is strongly related to treatment, αX=.9, conditioning on X results in an increasing OVB if the bias induced by U is at least one fourth (1−.92.92=.23) of X’s bias. In this case, bias amplification dominates bias reduction: though conditioning on X removes its own bias αXβX which amounts to 81% (= 1/(1 + .23)) of the total confounding bias, ^[5] the amplification of the remaining 19% (= .23/(1 + .23)) due to omitting U (αUβU) is strong enough to offset the bias-reducing effect because the bias amplification factor is 1/(1 − .9²) = 5.26.

Figure 3:

Increasing and decreasing OVB due to conditioning on an uncorrelated confounder X. The two dark grey areas indicate an increasing OVB, with 100%-200% (lighter shade) and 200% or more (darker shade) remaining bias. The two light grey areas indicate a decreasing OVB, with 50%-100% (darker shade) and 50% or less (lighter shade) remaining bias.

Offsetting Biases. For sgn(αXβX)≠sgn(αUβU), the confounding biases induced by X and U partially or even completely offset each other such that αXβX+αUβU<max(αXβX,αUβU). If U induces less bias than X, αUβU≤αXβX, adjusting for the observed confounder X increases rather than reduces OVB only if

But if U induces more bias than X, αUβU>αXβX, then conditioning on X always increases OVB because the remaining bias due to the unblocked backdoor path Z ← U → Y is necessarily greater than the initial bias: αUβU>αXβX+αUβU.

The upper right plot in Figure 3 shows areas of increasing and decreasing absolute OVB when biases (partially) offset each other. For αX→0, OVB increases as long as the bias induced by U is at least half of X’s bias: limαX→01−αX22−αX2=12. For αX=.5, OVB increases if the bias ratio exceeds 1−.522−.52=.43. If αX is close to 1, say .95, then OVB increases as long as the bias induced by U is at least about one tenth of X’s bias (1−.9522−.952=.09).

To summarize, for offsetting biases, the absolute OVB increases in two situations: First, if the confounding biases induced by X and U nearly offset each other (αXβX≈−αUβU). In fact, independent of the value of αX, OVB always increases if the bias induced by the unobserved confounder U is at least half of X’s bias (αUβU>αXβX/2). And second, if X strongly determines Z such that αX is close to 1, then the absolute OVB increases even when αXβX>>αUβU. The increase in the absolute OVB is mostly a result of the cancellation of the bias-offsetting effect, but the amplification of the remaining bias adds to the increase. Also note that the sign of the initial and adjusted OVB may differ. For instance, the initial OVB might be positive, but adjusting for X might turn the positive OVB into a negative OVB.

Unreliably measured confounder X

The OVB formula in (3) only holds for a reliably measured uncorrelated confounder X. The right graph in Figure 2 shows the case with a fallibly measured X. The node of X now turns into a vacant node (open circle) indicating that X is not directly observed. Instead, we only have an unreliable measure X* which is given by X∗=X+e, where e is an independent error with mean zero and variance σe2. ^[6] Since Var(X) = 1, the reliability of X* is given by γ=1/(1+σe2). Measurement error in X* has no influence on the initial OVB, OVB(τˆ|{})=αXβX+αUβU, but affects OVB after adjusting for the fallible X* (Proof 3 in Appendix C):

In comparison to the OVB for a reliably measured confounder X in (3), measurement error has two effects: First, the bias left due to (partially) unblocked backdoor paths now consists of two components, αUβU and αXβX(1−γ). Besides the open backdoor path Z ← U → Y (due to omitting U), adjusting for X* no longer fully blocks the backdoor path Z ← X → Y such that(1−γ)% of X’s bias is left. That is, X* removes the bias induced by X only to the degree of its reliability (γ). The less reliable the measurement, the more of X’s bias will remain. Second, measurement error attenuates the bias amplification factor since 1/(1−αX2γ) is always less than 1/(1−αX2) because 0≤γ≤1. A completely unreliable measure X* with γ→0 neither removes nor amplifies any bias such that the initial OVB remains: limγ→0OVB(τˆ|X∗)=αXβX+αUβU (also see [25]). On the other extreme, with a perfectly reliable measure X (γ=1), the OVB formula in (6) reduces to the OVB formula in (3).

Biases in the Same Direction. The second and third row of plots in Figure 3 show the areas of increasing OVB (the two dark grey areas) and decreasing OVB (the two light grey areas) for an unreliably measured confounder X (γ=.75 in the second row and γ=.5 in the third row). In the left columns of plots for sgn(αXβX)=sgn(αUβU), the 100% bias contour lines are the same as for the reliably measured confounder (upper left plot), but the 200% and 50% bias contour lines change. Unreliability in X does not change the 100 % contour line because measurement error always results in an attenuation of OVB toward the initial OVB [26] (see Proof 5 in Appendix C which also contains a more detailed discussion). Since the 100% contour line represents situations where conditioning on X does not alter the initial OVB (i. e., bias reduction is exactly offset by bias amplification), measurement error has no effect. But if adjusting for the reliable X increases OVB then measurement error attenuates the increase as shown by the retreating 200% contour line (as one moves from the plot in the first row to the plots in the second and third row). If conditioning on the reliable X reduces OVB then measurement error attenuates bias reduction as indicated by the retreating 50% contour line.

Offsetting Biases. For offsetting biases (sgn(αXβX)≠sgn(αUβU), shown in the right column of Figure 3), all bias contour lines depend on the extent of measurement error. In comparison to the reliably measured confounder (upper right plot), more measurement error in X* results in an expansion of the light grey areas of diminishing OVB, that is, measurement error makes an increasing OVB less likely because the cancellation of the offsetting biases is attenuated. Though unreliability decreases the chances of an increasing OVB, it does not imply that the fallible X* necessarily removes more bias than the corresponding reliable measure. A comparison of the 50% bias contour lines (or the very light grey area) across the three plots reveals that the fallibly X* can remove less OVB than the reliably X.

Imbalance in confounders U and X

For both reliably and unreliably measured confounders X, bias amplification operates via increasing the imbalance in U and X. For an unreliably measured confounder X*, the initial imbalance in U (αU) and remaining imbalance in X (αX(1−γ)) are inflated by the factor 1/(1−αX2γ): Imbalance(U|X∗)=αU1−αX2γ and Imbalance(X|X∗)=αX(1−γ)1−αX2γ (Proof 1 in Appendix C). The imbalance formula for U indicates that adjusting for X* always increases the absolute imbalance in U because the amplification factor 1/(1−αX2γ) is less than one (but note that measurement error attenuates bias amplification and thus the decrease in U’s absolute imbalance). Regarding the imbalance in X, conditioning on X* cannot fully balance X because the unreliable X* fails to completely remove the association between Z and X. However, the unreliable measure X* will be balanced, Imbalance(X∗|X∗)=0. Thus, balance in a fallible covariate X* does not imply that the underlying data-generating confounder X will be balanced. Particularly if αX>>0 or γ<.75, then the absolute imbalance in X after adjusting for X* may still be large but it will never exceed the absolute initial imbalance, Imbalance(X|{})=αX (Proof 2 in Appendix C). This result does not generalize to the more general case with multiple observed confounders. If one conditions not only on a single unreliable confounder but on multiple, possibly uncorrelated confounders simultaneously, the resulting imbalance in the latent X might exceed the initial imbalance. This is so because the remaining imbalance in X after conditioning on X*, Imbalance(X|X∗), is further amplified by any other confounder we condition on (just like the imbalance in U).

Reliably measured confounder X

Adjusting for the reliably measured confounder X but omitting U results in ^[7]

The OVB formula indicates that conditioning on a correlated confounder X has three effects. First, it eliminates its own confounding bias (αXβX) but also the entire confounding bias induced by X’s correlation with U (αXρβU+αUρβX). That is, conditioning on X blocks all backdoor paths going through X (i. e., Z ← X → Y, Z ← X U → Y, and Z ← U X → Y). Second, because of X and U’s correlation, X partially blocks the backdoor path Z ← U → Y to the extent of the squared correlation ρ2, thus the bias due to the unobserved U reduces to αUβU(1−ρ2). And third, the correlation also affects the bias amplification factor 1/(1−(αX+αUρ)2) because conditioning on X triggers U’s bias-amplifying potential to the extent of their correlation as reflected by the additional term αUρ in the denominator.

Depending on the sign of αUρ, the correlation can strengthen, weaken, or even neutralize the bias amplification factor. If sgn(αUρ)=sgn(αX) then the correlation boosts bias amplification in comparison to the uncorrelated case because αX+αUρ>αX. The stronger the correlation and the larger αU, the stronger the bias-amplifying effect. If sgn(αUρ)≠sgn(αX), the correlation can strengthen (if αX+αUρ>αX), weaken (if αX+αUρ<αX) or completely cancel bias amplification (if αX=−αUρ). Thus, even with highly correlated confounders X and U, there is no guarantee that conditioning on a correlated X reduces OVB (examples are briefly discussed at the end of the following subsection).

Unreliably measured confounder X

The right graph in Figure 4 shows the same causal diagram as before but with the fallible covariate X*. In this case, one can show (Proof 3 in Appendix C) that conditioning on X* results in an OVB of

All four terms of the initial bias appear in the OVB formula, but the biases induced by the four backdoor paths are not fully effective. First, the correlation of the unreliable X* with the unobserved confounder U, Cor(X∗,U)=ρ˜=ργ, reduces the bias induced by U to the extent of the squared correlation ρ˜2, leaving a bias of αUβU(1−ρ˜2). Second, the unreliable X* blocks the three backdoor paths via X only to the extent of its reliability (γ) and thus leaves a bias of (αXβX+αXρβU+αUρβX)(1−γ). Finally, the remaining bias due to the four partially unblocked backdoor paths is amplified but the bias amplification factor is attenuated by the reliability γ.

Due to the increased complexity of the OVB formulas, an easily interpretable inequality as for the uncorrelated confounder case is not derivable. Thus, we illustrate the effect of correlated confounders with two examples. The first row of plots in Figure 5 shows for two different parameter settings the areas of increasing (dark grey) and decreasing OVB (light grey) as a function of the correlation ρ (abscissa) and the unobserved confounder’s coefficient αU (ordinate). For both plots we set βX=βU=.1, but αX=.3 in the left plot and αX=.9 in the right plot (making X a near-IV in the latter case). In each plot, quadrant I (with ρ≥0 and αU≥0) represents the situation where all biases induced by X and U go into the same direction because all five data-generating parameters are positive. Quadrants II, III, and IV show the results for partial or completely offsetting biases (because the signs of the parameters differ).

Figure 5:

Increasing and decreasing OVB due to conditioning on a correlated confounder X. The two dark grey areas indicate an increasing OVB, with 100%-200% (lighter shade) and 200% or more (darker shade) remaining bias. The two light grey areas indicate a decreasing OVB, whith 50%-100% (darker shade) and 50% or less (lighter shade) remaining bias. The white areas indicate parameter combinations that are impossible for standardized path coefficients.

Consider quadrant I of the top right plot in Figure 3, where the confounder X strongly affects Z (αX=.9): OVB can exceed the initial bias even if one conditions on a confounder X that is almost perfectly correlated with U. In general, it is hard to derive a generalizable pattern from the two example plots. Without knowing the sign and magnitude of the five parameters it is impossible to predict whether conditioning on a correlated X or X* reduces or increases OVB even if X is highly correlated with U. The second and third row in Figure 3 shows the effect of measurement error, which is the same as for the uncorrelated case (i. e., attenuation to the initial bias; Proof 5 in Appendix C).

Imbalance in confounders U and X

As for the case of uncorrelated confounders, the bias-amplifying effect of conditioning on a reliably or unreliably measured confounder X can be explained by the amplified imbalance in U and X. The absolute initial imbalance in U, Imbalance(U|{})=αU+αXρ, might increase or decrease once one conditions on X*, even when U is correlated with X*. Adjusting for the correlated X* changes the initial imbalance in U to αU(1−ρ˜2)+αXρ(1−γ), which then is amplified by the factor 1/(1−(αX+αUρ)2γ) such that we obtain Imbalance(U|X∗)=αU(1−ρ˜2)+αXρ(1−γ)1−(αX+αUρ)2γ. Compared to the absolute value of the initial imbalance (before adjusting for X*), the absolute imbalance in U after adjusting for X* might be smaller or larger (Proof 2 in Appendix C). Despite the correlation, conditioning on X* can increase the imbalance in U because the term αUρ may strengthen the bias amplification factor.

Correspondingly, conditioning on X* first reduces the absolute initial imbalance in X from Imbalance(X|{})=αX+αUρ to (αX+αUρ)(1−γ), which again is amplified such that Imbalance(X|X∗)=(αX+αUρ)(1−γ)1−(αX+αUρ)2γ. Multiplying U’s imbalance by βU and X’s imbalance by βX, and then adding the two terms results in the OVB formula (8). As for the uncorrelated confounder case, the absolute imbalance in X after adjusting for X* will always be smaller than before the adjustment, Imbalance(X|X∗)≤Imbalance(X|{}) (Proof 2 in Appendix C). Again, this only holds for the case with a single observed confounder X. Conditioning on multiple confounders, including X*, can actually increase the imbalance in X (but as for the imbalance in U, whether the imbalance in X decreases or increases depends on the correlation among the observed confounders).

With a perfectly reliably measured X (γ=1), X will be fully balanced but U remains imbalanced with Imbalance(U|X∗)=αU(1−ρ2)1−(αX+αUρ)2. Note that neither the imbalance in U nor in X (given it is unreliably measured) can be tested empirically since both are unobserved.