# Abstract

Conventional wisdom dictates that the more we know about a problem domain the easier it is to predict the effects of policies in that domain. Strangely, this wisdom is not sanctioned by formal analysis, when the notions of “knowledge” and “policy” are given concrete definitions in the context of nonparametric causal analysis. This note describes this peculiarity and speculates on its implications.

## 1 Introduction

In her book, *Hunting Causes and Using Them* [1], Nancy Cartwright expresses several objections to the *do*-operator represents an ideal, atomic intervention, different from the one implementable by most policies under evaluation. According to Cartwright, for policy evaluation “we generally want to know what would happen were the policy really set in place,” and “the policy may affect a host of changes in other variables in the system, some envisaged and some not.”

In my answer to Cartwright [2, p. 363], I stressed two points. First, the *do*-calculus enables us to evaluate the effect of compound interventions as well, as long as they are described in the model and are not left to guesswork. Second, I claimed that “in many studies our goal is not to predict the effect of the crude, non-atomic intervention that we are about to implement but, rather, to evaluate an ideal, atomic policy that cannot be implemented given the available tools, but that represents nevertheless scientific knowledge that is pivotal for our understanding of the domain.”

The example I used was as follows: Smoking cannot be stopped by any legal or educational means available to us today; cigarette advertising can. That does not stop researchers from aiming to estimate “the effect of smoking on cancer,” and doing so from experiments in which they vary the instrument – cigarette advertisement – not smoking. The reason they would be interested in the atomic intervention

This note takes another look at this argument, in light of recent results in transportability theory (Bareinboim and Pearl [3], hereafter BP).

## A theorem and its implications

The question investigated in BP was whether one can infer the causal effect of *X* on *Y* by randomizing a surrogate variable *Z*, which is more easily controllable than *X*. This problem was addressed earlier in Pearl [2, pp. 88–89] where a sufficient condition was derived for a variable *Z* to act as an experimental surrogate for *X*. BP have obtained a condition that is both necessary and sufficient for surrogacy, which reads as follows:

**Theorem 1***(BP [3])*,

*The causal effect**can be inferred from experiments on Z if and only if*:

*1*. *can be inferred from observational studies alone, or*

*2(i). All directed paths from Z to Y go through X, and*

*2(ii)*. *can be inferred from observational studies*.

**Remark**: Condition 2(i), in effect, turns *Z* into an instrumental variable, when randomized.

If *X* stands for a treatment, then *Z* plays the role of an “intent-to-treat” variable in noncompliance situations. Condition 2(i) ensures that *Z* has no side effects on *Y*; i.e. it acts as an instrumental variable when randomized. Condition 2(ii) ensures a nonparametric identification of treatment effects, using *Z* as an instrument [4–6].

Figure 1(a) and (b) illustrates models where both 2(i) and 2(ii) are satisfied, while in Figure 1(c) 2(i) fails, because a directed path exists from *Z* to *Y*. For example, if *Z* represents cigarette tax and *X* represents smoking, then we can infer the causal effect of smoking on cancer, *Z* (i.e. deleting all arrows pointing to *Z*.)

### Figure 1

We now return to the question of whether scientific knowledge can be useful in evaluating practical policies. We ask: Suppose

Formally, the problem amounts to reversing the role of *X* and *Z* in Theorem 1 and yields:

**Theorem 2***The causal effect**can be inferred from observational studies and knowledge of**if and only if*:

*1*. *can be inferred from observational studies alone, or*

*2(i). All directed paths from X to Y go through Z, and*

*2(ii)*. *is identifiable in observational studies*.

This is a surprising result, saying in effect that knowing how *X* affects *Y* (i.e. *X* on *Y*. Put differently, knowing how effective a treatment is does not tell us how effective any policy is, which is intended to administer that treatment in practice. This can be seen by noting that 2(i) cannot be satisfied unless *Z* contains descendants of *X*, and this will never be the case when *Z* is chosen so as to influence *Y* through *X*. Therefore, the causal effect

To see the ramification of this impossibility result, consider again the smoking-cancer example, depicted in Figure 2. Here *Z* represents cigarette tax, *X* represents smoking, and *Y* represents cancer. Our aim is to estimate the effect of policy *Z* and *Y* represents confounding factors, for example, factors that render communities that impose high cigarette taxes more diet-conscience, hence, less cancer prone. In model 2(a), neither *Z* does not block the directed path from *X* to *Y*, thus violating Condition 2(i).

### Figure 2

## Discussion

This result is peculiar, for it implies that policies such as imposing cigarette taxes cannot be informed by knowing the extent to which smoking causes cancer. It reflects an idiosyncratic property of nonparametric analysis in which knowledge of causal effects (such as

Things are different in parametric systems, as can be seen from Figure 2(b), which represents a linear version of Figure 2(a), with parameters *Z* on *Y* is *X* on *Y* is given,

Another exception to this impossibility result is the case where *X* has zero effect on *Y*, namely, *X* to *Y*, as shown in Figure 2(c) and the conclusion

This observation mitigates substantially our initial disappointment with formal analysis. It implies that, whereas knowledge of *point-estimate* of *interval estimate* that vanishes when *X* is known to have no effect on *Y* at the population level, i.e.

Finally, another exception to Theorem 2 occurs when a policy *Z* has no side effects on *Y* (see Pearl [2, p. 358]).

# Acknowledgements

This paper benefited greatly from discussions with Elias Bareinboim who proved the “only if” part of Theorem 1. This research was supported in parts by grants from NSF #IIS-0914211 and #IIS-1018922 and ONR #N000-14-09-1-0665 and #N00014-10-1-0933.

### References

1. CartwrightN. Hunting causes and using them: approaches in philosophy and economics. New York, NY: Cambridge University Press, 2007.Search in Google Scholar

2. PearlJ. Causality: models, reasoning, and inference, 2nd ed. New York: Cambridge University Press, 2009.Search in Google Scholar

3. BareinboimE, PearlJ. Causal inference by surrogate experiments: z-identifiability. In: de FreitasN, MurphyK, editors. Proceedings of the twenty-eighth conference on uncertainty in artificial intelligence. Corvallis, OR: AUAI Press, 2012:113–20.Search in Google Scholar

4. AngristJ, ImbensG, RubinD. Identification of causal effects using instrumental variables (with comments). J Am Stat Assoc1996;91:444–72.Search in Google Scholar

5. BalkeA, PearlJ. Universal formulas for treatment effect from noncompliance data. In: JewellN, KimberA, LeeM-L, WhitmoreG, editors. Lifetime data: models in reliability and survival analysis. Dordrecht: Kluwer Academic Publishers, 1995:39–43.Search in Google Scholar

6. BalkeA, PearlJ. Bounds on treatment effects from studies with imperfect compliance. J Am Stat Assoc1997;92:1172–6.Search in Google Scholar

7. ShpitserI, PearlJ. Complete identification methods for the causal hierarchy. J Mach Learn Res2008;9:1941–79.Search in Google Scholar

**Published Online:**2014-4-18

**Published in Print:**2014-3-1

©2014 by Walter de Gruyter Berlin / Boston