Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Causal Inference

Ed. by Imai, Kosuke / Pearl, Judea / Petersen, Maya Liv / Sekhon, Jasjeet / van der Laan, Mark J.

Online
ISSN
2193-3685
See all formats and pricing
More options …

Regression Adjustments for Estimating the Global Treatment Effect in Experiments with Interference

Alex Chin
Published Online: 2019-05-17 | DOI: https://doi.org/10.1515/jci-2018-0026

Abstract

Standard estimators of the global average treatment effect can be biased in the presence of interference. This paper proposes regression adjustment estimators for removing bias due to interference in Bernoulli randomized experiments. We use a fitted model to predict the counterfactual outcomes of global control and global treatment. Our work differs from standard regression adjustments in that the adjustment variables are constructed from functions of the treatment assignment vector, and that we allow the researcher to use a collection of any functions correlated with the response, turning the problem of detecting interference into a feature engineering problem. We characterize the distribution of the proposed estimator in a linear model setting and connect the results to the standard theory of regression adjustments under SUTVA. We then propose an estimator that allows for flexible machine learning estimators to be used for fitting a nonlinear interference functional form. We propose conducting statistical inference via bootstrap and resampling methods, which allow us to sidestep the complicated dependences implied by interference and instead rely on empirical covariance structures. Such variance estimation relies on an exogeneity assumption akin to the standard unconfoundedness assumption invoked in observational studies. In simulation experiments, our methods are better at debiasing estimates than existing inverse propensity weighted estimators based on neighborhood exposure modeling. We use our method to reanalyze an experiment concerning weather insurance adoption conducted on a collection of villages in rural China.

Keywords: causal inference; peer effects; SUTVA; A/B testing; exposure models; off-policy evaluation

References

  • 1.

    Cox DR. Planning of experiments. 1958.Google Scholar

  • 2.

    Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;66(5):688.CrossrefGoogle Scholar

  • 3.

    Rubin DB. Randomization analysis of experimental data: The Fisher randomization test comment. J Am Stat Assoc. 1980.

  • 4.

    Rosenbaum PR. Interference between units in randomized experiments. J Am Stat Assoc. 2007;102(477):191–200.CrossrefGoogle Scholar

  • 5.

    Walker D, Muchnik L. Design of randomized experiments in networks. Proc IEEE. 2014;102(12):1940–51.CrossrefGoogle Scholar

  • 6.

    Aral S. Networked experiments. Oxford, UK: Oxford University Press; 2016.Google Scholar

  • 7.

    Taylor SJ, Eckles D. Randomized experiments to detect and estimate social influence in networks. 1709.09636 (2017).Google Scholar

  • 8.

    Hudgens MG, Halloran ME. Toward causal inference with interference. J Am Stat Assoc. 2008;103(482):832–42.CrossrefGoogle Scholar

  • 9.

    VanderWeele TJ, Tchetgen Tchetgen EJ. Effect partitioning under interference in two-stage randomized vaccine trials. Stat Probab Lett. 2011;81(7):861–9.CrossrefGoogle Scholar

  • 10.

    Tchetgen EJT, VanderWeele TJ. On causal inference in the presence of interference. Stat Methods Med Res. 2012;21(1):55–75.CrossrefGoogle Scholar

  • 11.

    Liu L, Hudgens MG. Large sample randomization inference of causal effects in the presence of interference. J Am Stat Assoc. 2014;109(505):288–301.CrossrefGoogle Scholar

  • 12.

    Baird S, Bohren JA, McIntosh C, Özler B. Optimal design of experiments in the presence of interference. Rev Econ Stat. 2016;(0).

  • 13.

    Basse G, Feller A, Toulis P. Exact tests for two-stage randomized designs in the presence of interference. 1709.08036 (2017).Google Scholar

  • 14.

    Basse G, Feller A. Analyzing two-stage experiments in the presence of interference. J Am Stat Assoc. 2018;113(521):41–55.CrossrefGoogle Scholar

  • 15.

    Ogburn EL, VanderWeele TJ. Causal diagrams for interference. Stat Sci. 2014;29(4):559–78.CrossrefGoogle Scholar

  • 16.

    van der Laan MJ. Causal inference for a population of causally connected units. J Causal Infer. 2014;2(1):13–74.Google Scholar

  • 17.

    Ogburn EL, Sofrygin O, Diaz I, van der Laan MJ. Causal inference for social network data. 1705.08527 (2017).Google Scholar

  • 18.

    Tchetgen Tchetgen EJ, Fulcher I, Shpitser I. Auto-g-computation of causal effects on a network. 1709.01577 (2017).Google Scholar

  • 19.

    Choi D. Estimation of monotone treatment effects in network experiments. J Am Stat Assoc. 2017;1–9.

  • 20.

    Jagadeesan R, Pillai N, Volfovsky A. Designs for estimating the treatment effect in networks with interference. 1705.08524 (2017).Google Scholar

  • 21.

    Aronow PM, Samii C. Estimating average causal effects under general interference, with application to a social network experiment. Ann Appl Stat. 2017;11(4):1912–47.CrossrefGoogle Scholar

  • 22.

    Manski CF. Identification of treatment response with social interactions. Econom J. 2013;16(1).

  • 23.

    Sussman DL, Airoldi EM. Elements of estimation theory for causal effects in the presence of network interference. 1702.03578 (2017).Google Scholar

  • 24.

    Forastiere L, Airoldi EM, Mealli F. Identification and estimation of treatment and interference effects in observational studies on networks. 1609.06245 (2016).Google Scholar

  • 25.

    Paluck EL, Shepherd H, Aronow PM. Changing climates of conflict: A social network experiment in 56 schools. Proc Natl Acad Sci USA. 2016;113(3):566–71.CrossrefGoogle Scholar

  • 26.

    Banerjee A, Chandrasekhar AG, Duflo E, Jackson MO. The diffusion of microfinance. Science. 2013;341(6144):1236498.Google Scholar

  • 27.

    Cai J, De Janvry A, Sadoulet E. Social networks and the decision to insure. Am Econ J Appl Econ. 2015;7(2):81–108.CrossrefGoogle Scholar

  • 28.

    Kim DA, Hwong AR, Stafford D, Hughes DA, O’Malley AJ, Fowler JH, Christakis NA. Social network targeting to maximise population behaviour change: a cluster randomised controlled trial. Lancet. 2015;386(9989):145–53.CrossrefGoogle Scholar

  • 29.

    Beaman L, BenYishay A, Magruder J, Mobarak AM. Can network theory-based targeting increase technology adoption? Technical report. National Bureau of Economic Research; 2018.

  • 30.

    Eckles D, Karrer B, Ugander J. Design and analysis of experiments in networks: Reducing bias from interference. J Causal Infer. 2017;5(1).

  • 31.

    Ugander J, Karrer B, Backstrom L, Kleinberg J. Graph cluster randomization: Network exposure to multiple universes. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2013. p. 329–37.Google Scholar

  • 32.

    Fortunato S. Community detection in graphs. Phys Rep. 2010;486(3–5):75–174.CrossrefGoogle Scholar

  • 33.

    Ugander J, Backstrom L. Balanced label propagation for partitioning massive graphs. In: Proceedings of the sixth ACM International Conference on Web Search and Data Mining. ACM; 2013. p. 507–16.Google Scholar

  • 34.

    Ugander J, Karrer B, Backstrom L, Marlow C. The anatomy of the Facebook social graph. 1111.4503 (2011).Google Scholar

  • 35.

    Backstrom L, Boldi P, Rosa M, Ugander J, Vigna S. Four degrees of separation. In: Proceedings of the 4th Annual ACM Web Science Conference. ACM; 2012. p. 33–42.Google Scholar

  • 36.

    Bhagat S, Burke M, Diuk C, Filiz IO, Edunov S. Three and a half degrees of separation. In: Facebook research note. 2016. https://research.fb.com/three-and-a-half-degrees-of-separation/.Google Scholar

  • 37.

    Saveski M, Pouget-Abadie J, Saint-Jacques G, Duan W, Ghosh S, Xu Y, Airoldi EM. Detecting network effects: Randomizing over randomized experiments. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM; 2017. p. 1027–35.Google Scholar

  • 38.

    Traud AL, Kelsic ED, Mucha PJ, Porter MA. Comparing community structure to characteristics in online collegiate social networks. SIAM Rev. 2011;53(3):526–43.CrossrefGoogle Scholar

  • 39.

    Traud AL, Mucha PJ, Porter MA. Social structure of Facebook networks. Physica A. 2012;391(16):4165–80.CrossrefGoogle Scholar

  • 40.

    Tyner S, Briatte F, Hofmann H. Network visualization with ggplot2. R Journal. 2017.

  • 41.

    Fruchterman TM, Reingold EM. Graph drawing by force-directed placement. Softw Pract Exp. 1991;21(11):1129–64.CrossrefGoogle Scholar

  • 42.

    Freedman DA. On regression adjustments to experimental data. Adv Appl Math. 2008a;40(2):180–93.CrossrefGoogle Scholar

  • 43.

    Freedman DA. On regression adjustments in experiments with several treatments. Ann Appl Stat. 2008b;2(1):176–96.CrossrefGoogle Scholar

  • 44.

    Lin W. Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. Ann Appl Stat. 2013;7(1):295–318.CrossrefGoogle Scholar

  • 45.

    Berk R, Pitkin E, Brown L, Buja A, George E, Zhao L. Covariance adjustments for the analysis of randomized field experiments. Eval Rev. 2013;37(3–4):170–96.CrossrefGoogle Scholar

  • 46.

    Bloniarz A, Liu H, Zhang C-H, Sekhon JS, Yu B. Lasso adjustments of treatment effect estimates in randomized experiments. Proc Natl Acad Sci USA. 2016;113(27):7383–90.CrossrefGoogle Scholar

  • 47.

    Wager S, Du W, Taylor J, Tibshirani RJ. High-dimensional regression adjustments in randomized experiments. Proc Natl Acad Sci USA. 2016;113(45):12673–8.CrossrefGoogle Scholar

  • 48.

    Wu E, Gagnon-Bartsch J. The LOOP estimator: Adjusting for covariates in randomized experiments. 1708.01229 (2017).Google Scholar

  • 49.

    Athey S, Imbens GW, Wager S. Approximate residual balancing: De-biased inference of average treatment effects in high dimensions. 1604.07125 (2017).Google Scholar

  • 50.

    Chernozhukov V, Chetverikov D, Demirer M, Duflo E, Hansen C, Newey W, Robins J. Double/debiased machine learning for treatment and structural parameters. Econom J. 2018;21(1):C1–68.CrossrefGoogle Scholar

  • 51.

    Manski CF. Identification of endogenous social effects: The reflection problem. Rev Econ Stud. 1993.

  • 52.

    Bramoullé Y, Djebbari H, Fortin B. Identification of peer effects through social networks. J Econom. 2009;150(1):41–55.CrossrefGoogle Scholar

  • 53.

    Neyman J. On the application of probability theory to agricultural experiments. Essay on Principles. Section 9 (translated and edited by Dabrowska DM, Speed TP. Stat Sci. 1990;5:465–80). Ann Agric Sci. 1923;10:1–51.Google Scholar

  • 54.

    Imbens GW. Nonparametric estimation of average treatment effects under exogeneity: A review. Rev Econ Stat. 2004;86(1):4–29.CrossrefGoogle Scholar

  • 55.

    Pearl J. Causality. Cambridge University Press; 2009.Google Scholar

  • 56.

    Greenland S. Interpretation and choice of effect measures in epidemiologic analyses. Am J Epidemiol. 1987;125(5):761–8.CrossrefGoogle Scholar

  • 57.

    Robins J, Greenland S. The probability of causation under a stochastic model for individual risk. Biometrics. 1989;1125–38.

  • 58.

    Robins JM, Greenland S. Causal inference without counterfactuals: comment. J Am Stat Assoc. 2000;95(450):431–5.Google Scholar

  • 59.

    VanderWeele TJ, Robins JM. Stochastic counterfactuals and stochastic sufficient causes. Stat Sin. 2012;22(1):379.Google Scholar

  • 60.

    Sävje F, Aronow PM, Hudgens MG. Average treatment effects in the presence of unknown interference. 1711.06399 (2017).Google Scholar

  • 61.

    Chin A. Central limit theorems via Stein’s method for randomized experiments under interference. 1808.08683 (2018).Google Scholar

  • 62.

    Montgomery JM, Nyhan B, Torres M. How conditioning on posttreatment variables can ruin your experiment and what to do about it. Am J Polit Sci. 2018;62(3):760–75.CrossrefGoogle Scholar

  • 63.

    Kivelä M, Arenas A, Barthelemy M, Gleeson JP, Moreno Y, Porter MA. Multilayer networks. J Complex Netw. 2014;2(3):203–71.CrossrefGoogle Scholar

  • 64.

    Granovetter MS. The strength of weak ties. Am J Sociol. 1973;78(6):1360–80.CrossrefGoogle Scholar

  • 65.

    Haythornthwaite C, Wellman. Work friendship B. and media use for information exchange in a networked organization. J Am Soc Inf Sci. 1998;49(12):1101–14.CrossrefGoogle Scholar

  • 66.

    Eicker F. Limit theorems for regressions with unequal and dependent errors. 1967.

  • 67.

    Huber PJ. The behavior of maximum likelihood estimates under nonstandard conditions. 1967.

  • 68.

    White H. A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica. 1980;817–38.

  • 69.

    Hahn J. On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica. 1998;315–31.

  • 70.

    Abadie A, Athey S, Imbens GW, Wooldridge J. When should you adjust standard errors for clustering? Technical report. National Bureau of Economic Research; 2017a.

  • 71.

    Abadie A, Athey S, Imbens GW, Wooldridge JM. Sampling-based vs. design-based uncertainty in regression analysis. 1706.01778 (2017b).Google Scholar

  • 72.

    Aronow PM, Middleton JA. A class of unbiased estimators of the average treatment effect in randomized experiments. J Causal Infer. 2013;1(1):135–54.Google Scholar

  • 73.

    Chen K, Lei J. Network cross-validation for determining the number of communities in network data. J Am Stat Assoc. 2018;113(521):241–51.CrossrefGoogle Scholar

  • 74.

    Li T, Levina E, Zhu J. Network cross-validation by edge sampling. 1612.04717 (2018).Google Scholar

  • 75.

    Efron B. Bootstrap methods: Another look at the jackknife. Ann Stat. 1979;7(1):1–26.CrossrefGoogle Scholar

  • 76.

    Künsch HR. The jackknife and the bootstrap for general stationary observations. Ann Stat. 1989; 1217–41.

  • 77.

    Efron B. Better bootstrap confidence intervals. J Am Stat Assoc. 1987;82(397):171–85.CrossrefGoogle Scholar

  • 78.

    Jackknife C-FJWu. bootstrap and other resampling methods in regression analysis. Ann Stat. 1986;1261–95.

  • 79.

    Horvitz DG, Thompson DJ. A generalization of sampling without replacement from a finite universe. J Am Stat Assoc. 1952;47(260):663–85.CrossrefGoogle Scholar

  • 80.

    Hájek J. Comment on ‘An essay on the logical foundations of survey sampling, part 1’. In Basu D, Godambe V, Sprott DA, editors. Foundations of Statistical Inference. Toronto: Holt, Rinehart and Winston; 1971. p. 236.Google Scholar

  • 81.

    Blume LE. The statistical mechanics of best-response strategy revision. Games Econ Behav. 1995;11(2):111–45.CrossrefGoogle Scholar

  • 82.

    Watts DJ, Strogatz SH. Collective dynamics of ‘small-world’ networks. Nature. 1998;393(6684):440.CrossrefGoogle Scholar

  • 83.

    Hastie T, Tibshirani R. Generalized additive models. Stat Sci. 1986;1(3):297–310.CrossrefGoogle Scholar

  • 84.

    VanderWeele TJ, Tchetgen Tchetgen EJ, Halloran ME. Interference and sensitivity analysis. Stat Sci. 2014;29(4):687.CrossrefGoogle Scholar

  • 85.

    Robins JM, Rotnitzky A, Scharfstein DO. Sensitivity analysis for selection bias and unmeasured confounding in missing data and causal inference models. In: Statistical Models in Epidemiology, the Environment, and Clinical Trials. Springer; 2000. p. 1–94.Google Scholar

  • 86.

    Egami N. Unbiased estimation and sensitivity analysis for network-specific spillover effects: Application to an online network experiment. 1708.08171 (2017).Google Scholar

  • 87.

    Aronow PM. A general method for detecting interference between units in randomized experiments. Sociol Methods Res. 2012;41(1):3–16.CrossrefGoogle Scholar

  • 88.

    Athey S, Eckles D, Imbens GW. Exact p-values for network interference. J Am Stat Assoc. 2017;1–11.

About the article

Received: 2018-09-20

Revised: 2019-04-12

Accepted: 2019-04-12

Published Online: 2019-05-17

Published in Print: 2019-09-25


Funding Source: National Science Foundation

Award identifier / Grant number: IIS-1657104

This work was supported in part by NSF grant IIS-1657104.


Citation Information: Journal of Causal Inference, Volume 7, Issue 2, 20180026, ISSN (Online) 2193-3685, DOI: https://doi.org/10.1515/jci-2018-0026.

Export Citation

© 2019 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in