Unbiased Estimation of the Average Treatment Effect in Cluster-Randomized Experiments

Joel A. Middleton 1  and Peter M. Aronow 2
  • 1 Department of Political Science, University of California Berkeley, Berkeley, CA, USA
  • 2 Department of Political Science, Yale University, New Haven, CT, USA
Joel A. Middleton and Peter M. Aronow

Abstract

Many estimators of the average treatment effect, including the difference-in-means, may be biased when clusters of units are allocated to treatment. This bias remains even when the number of units within each cluster grows asymptotically large. In this paper, we propose simple, unbiased, location-invariant, and covariate-adjusted estimators of the average treatment effect in experiments with random allocation of clusters, along with associated variance estimators. We then analyze a cluster-randomized field experiment on voter mobilization in the US, demonstrating that the proposed estimators have precision that is comparable, if not superior, to that of existing, biased estimators of the average treatment effect.

  • Angrist, J. D. and J. Pischke (2009) Mostly Harmless Econometrics. Princeton: Princeton University Press.

    • Crossref
    • Export Citation
  • Aronow, P. M., D. P. Green and D. K. K. Lee (2014) “Sharp Bounds on the Variance in Randomized Experiments,” Annals of Statistics, 42(3):850–871.

    • Crossref
    • Export Citation
  • Bates, D. and M. Maechler (2010) lme4: Linear mixed-effects models using S4 classes. R package, version 0.999375-37.

  • Brewer, K. R. W. (1979) “A Class of Robust Sampling Designs for Large-Scale Surveys,” Journal of the American Statistical Association, 74:911–915.

    • Crossref
    • Export Citation
  • Chaudhuri, A. and H. Stenger (2005) Survey Sampling. Boca Raton: Chapman and Hall.

    • Crossref
    • Export Citation
  • Cochran, W. G. (1977) Sampling Techniques, 3rd ed. New York: John Wiley.

  • Des Raj. (1965) “On A Method of Using Multi-Auxiliary Information in Sample Surveys,” Journal of The American Statistical Association, 60:270–277.

    • Crossref
    • Export Citation
  • Ding, P. (2014) “A Paradox from Randomization-Based Causal Inference,” arXiv preprint arXiv:1402.0142.

  • Donner, A. and N. Klar (2000) Design and Analysis of Cluster Randomization Trials in Health Research. New York: Oxford Univ. Press.

    • Crossref
    • Export Citation
  • Freedman, D. A. (2006) “On the So-Called ‘Huber Sandwich Estimator’ and ‘Robust’ Standard Errors,” American Statistician, 60:299–302.

    • Crossref
    • Export Citation
  • Freedman, D. A. (2008a) “On Regression Adjustments to Experimental Data,” Advances in Applied Mathematics, 40:180–193.

    • Crossref
    • Export Citation
  • Freedman, D. A. (2008b) “On Regression Adjustments in Experiments with Several Treatments.” Annals of Applied Statistics, 2:176–196.

    • Crossref
    • Export Citation
  • Freedman, D. A., R. Pisani and R. A. Purves (1998) Statistics, 3rd ed. New York: W. W. Norton, Inc.

  • Green, D. P. and L. Vavreck (2008) “Analysis of Cluster-Randomized Experiments: A Comparison of Alternative Estimation Approaches,” Political Analysis, 16:138–152.

    • Crossref
    • Export Citation
  • Hansen, B. and J. Bowers (2008) “Covariate Balance in Simple, Stratified and Clustered Comparative Studies,” Statistical Science, 23:219–236.

    • Crossref
    • Export Citation
  • Hansen, B. and J. Bowers (2009) “Attributing Effects to a Cluster-Randomized Get-Out-the-Vote Campaign,” Journal of the American Statistical Association, 104:873–885.

    • Crossref
    • Export Citation
  • Hartley, H. O. and A. Ross (1954) “Unbiased Ratio Estimators,” Nature, 174:270.

    • Crossref
    • Export Citation
  • Hoffman, E. B., P. K. Sen and C. R. Weinberg (2001) “Within-Cluster Resampling,” Biometrika, 88: 1121–1134.

    • Crossref
    • Export Citation
  • Horvitz, D. G. and D. J. Thompson (1952) “A Generalization of Sampling Without Replacement From a Finite Universe,” Journal of the American Statistical Association, 47:663–684.

    • Crossref
    • Export Citation
  • Humphreys, M. (2009) Bounds on Least Squares Estimates of Causal Effects in the Presence of Heterogeneous Assignment Probabilities. Working paper. Available at: http://www.columbia.edu/~mh2245/papers1/monotonicity4.pdf.

  • Imai, K., G. King and C. Nall (2009) “The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation,” Statistical Science, 24:29–53.

    • Crossref
    • Export Citation
  • King, G. and M. Roberts (2014) “How Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It,” Political Analysis, 1–12.

  • Lachin, J. M. (1988) “Properties of Simple Randomization in Clinical Trials,” Controlled Clinical Trials, 9(4):312–326.

    • Crossref
    • Export Citation
  • Lin, W. (2013) “Agnostic Notes on Regression Adjustments to Experimental Data: Reexamining Freedman’s Critique,” Annals of Applied Statistics, 7(1):295–318.

    • Crossref
    • Export Citation
  • Middleton, J. A. (2008) “Bias of the Regression Estimator for Experiments Using Clustered Random Assignment,” Statistics and Probability Letters, 78:2654–2659.

    • Crossref
    • Export Citation
  • Miratrix, L., J. Sekhon and B. Yu (2013) “Adjusting Treatment Effect Estimates by Post- Stratification in Randomized Experiments,” Journal of the Royal Statistical Society. Series B (Methodological), 75(2):369–396.

    • Crossref
    • Export Citation
  • Neyman, J. (1923) “On the Application of Probability Theory to Agricultural Experiments: Essay on Principles, Section 9,” Statistical Science, 5:465–480. (Translated in 1990).

    • Crossref
    • Export Citation
  • Neyman, J. (1934) “On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection,” Journal of the Royal Statistical Society, 97(4):558–625.

    • Crossref
    • Export Citation
  • R Development Core Team. (2010) R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, ISBN 3-900051-07-0. Version 2.12.0.

  • Rosenbaum, P. R. (2002) Observational Studies, 2nd ed. New York: Springer.

    • Crossref
    • Export Citation
  • Rubin, D. (1974) “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies,” Journal of Educational Psychology, 66:688–701.

    • Crossref
    • Export Citation
  • Rubin, D. B. (1978) “Bayesian Inference for Causal Effects: The Role of Randomization,” The Annals of Statistics, 6:34–58.

    • Crossref
    • Export Citation
  • Rubin, D. B. (2005) “Causal Inference Using Potential Outcomes: Design, Modeling, Decisions,” Journal of the American Statistical Association, 100:322–331.

    • Crossref
    • Export Citation
  • Samii, C. and P. M. Aronow (2012) “On Equivalencies Between Design-Based and Regression-Based Variance Estimators for Randomized Experiments,” Statistics and Probability Letters, 82:365–370.

    • Crossref
    • Export Citation
  • Sarndal, C.-E. (1978) “Design-Based and Model-Based Inference in Survey Sampling,” Scandinavian Journal of Statistics, 5(1):27–52.

  • Williams, W. H. (1961) “Generating Unbiased Ratio and Regression Estimators,” Biometrics, 17:267–274.

    • Crossref
    • Export Citation
Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Search