Jump to ContentJump to Main Navigation
Show Summary Details

Journal of Official Statistics

The Journal of Statistics Sweden

IMPACT FACTOR 2015: 0.467
5-year IMPACT FACTOR: 0.740

SCImago Journal Rank (SJR) 2015: 0.410
Source Normalized Impact per Paper (SNIP) 2015: 0.810
Impact per Publication (IPP) 2015: 0.540

Open Access
See all formats and pricing

The Relative Impacts of Design Effects and Multiple Imputation on Variance Estimates: A Case Study with the 2008 National Ambulatory Medical Care Survey

1 / Elizabeth Goldberg1 / Nathaniel Schenker1 / Vladislav Beresovsky1 / Susan Schappert1 / Sandra Decker1 / Nancy Sonnenfeld1 / Iris Shimizu1

1National Center for Health Statistics, 3311 Toledo Road, Hyattsville, MD 20782, U.S.A

This content is open access.

Citation Information: Journal of Official Statistics. Volume 30, Issue 1, Pages 147–161, ISSN (Online) 2001-7367, DOI: https://doi.org/10.2478/jos-2014-0008, February 2014

Publication History

Published Online:


The National Ambulatory Medical Care Survey collects data on office-based physician care from a nationally representative, multistage sampling scheme where the ultimate unit of analysis is a patient-doctor encounter. Patient race, a commonly analyzed demographic, has been subject to a steadily increasing item nonresponse rate. In 1999, race was missing for 17 percent of cases; by 2008, that figure had risen to 33 percent. Over this entire period, single imputation has been the compensation method employed. Recent research at the National Center for Health Statistics evaluated multiply imputing race to better represent the missing-data uncertainty. Given item nonresponse rates of 30 percent or greater, we were surprised to find many estimates’ ratios of multiple-imputation to single-imputation estimated standard errors close to 1. A likely explanation is that the design effects attributable to the complex sample design largely outweigh any increase in variance attributable to missing-data uncertainty.

Keywords: Health survey; missing data; item nonresponse; fraction of missing information


  • Andridge, R. and Little, R. (2010). A Review of Hot Deck Imputation for Survey Non-Response. International Statistical Review, 78, 40-64, DOI: http://www.dx.doi. org/ 10.1111/j.1751-5823.2010.00103.x. [Crossref] [Web of Science]

  • Cleveland, W. (1979). Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association, 74, 829-836, DOI: http://www.dx.doi. org/10.1080/01621459.1979.10481038. [Crossref]

  • Efron, B. (1994). Missing Data, Imputation, and the Bootstrap. Journal of the American Statistical Association, 89, 463-475, DOI: http://www.dx.doi.org/10.1080/01621459.1994.10476768. [Crossref]

  • Groves, R., Dillman, D., Eltinge, J., and Little, R., (Eds.) (2002). Survey Nonresponse. New York, NY: Wiley.

  • Kalton, G. and Kasprzyk, D. (1986). The Treatment of Missing Survey Data. Survey Methodology, 12, 1-16.

  • Kish, L. (1965). Survey Sampling. New York, NY: Wiley.

  • Kozak, J. (1995). Underreporting of Race in the National Hospital Discharge Survey. Advance Data from Vital and Health Statistics, No. 265. Hyattsville, MD: National Center for Health Statistics.

  • Li, Y., Lynch, C., Shimizu, I., and Kaufman, S. (2004). Imputation Variance Estimation by Bootstrap Method for the National Ambulatory Medical Care Survey, Proceedings of the Survey Research Methods Section of the American Statistical Association.

  • Lohr, S. (1999). Sampling: Design and Analysis. Pacific Grove, CA: Brooks/Cole.

  • McCarthy, P. and Snowden, C. (1985). The Bootstrap and Finite Population Sampling. Vital Health Statistics, 2(95). Hyattsville, MD: National Center for Health Statistics.

  • National Center for Health Statistics (2009). 2008 NAMCS Micro-Data File Documentation. Division of Health Care Surveys, National Center for Health Statistics, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services, Hyattsville, MD, Available online at: ftp://ftp.cdc.gov/pub/Health_Statistics/ NCHS/Dataset_Documentation/NAMCS/doc08.pdf (accessed January 2014).

  • Office of Management and Budget (1997). Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity, Federal Register 62FR58781-58790. Available at: http://www.gpo.gov/fdsys/pkg/FR-1997-10-30/pdf/97-28653.pdf (accessed January 2014).

  • Raghunathan, T., Lepkowski, J., Van Hoewyk, J., and Solenberger, P. (2001). A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models. Survey Methodology, 27, 85-95.

  • Reiter, J., Raghunathan, T., and Kinney, S. (2006). The Importance of Modeling the Sampling Design in Multiple Imputation for Missing Data. Survey Methodology, 32, 143-150.

  • Rubin, D. (1987). Multiple Imputation for Nonresponse in Surveys. New York, NY: Wiley.

  • Rubin, D. (1996). Multiple Imputation After 18 þ Years (with discussion). Journal of the American Statistical Association, 91, 473-489.

  • Rubin, D. and Schenker, N. (1986). Multiple Imputation for Interval Estimation from Simple Random Samples with Ignorable Nonresponse. Journal of the American Statistical Association, 81, 366-374, DOI: http://www.dx.doi.org/10.1080/01621459.1986.10478280. [Crossref]

  • Schenker, N., Borrud, L., Burt, V., Curtin, L., Flegal, K., Hughes, J., Johnson, C., Looker, A., and Mirel, L. (2011). Multiple Imputation of Missing Dual-Energy X-Ray Absorptiometry Data in the National Health and Nutrition Examination Survey. Statistics in Medicine, 30, 260-276, DOI: http://www.dx.doi.org/10.1002/sim.4080. [Crossref]

  • Shao, J. and Sitter, R. (1996). Bootstrap for Imputed Survey Data. Journal of the American Statistical Association, 91, 1278-1288, DOI: http://www.dx.doi.org/10.1080/01621459.1996.10476997. [Crossref]

  • Valverde, R. and Marsteller, J. (2007). A Revised Matching Routine for Imputing Missing Race and Ethnicity in the National Ambulatory Medical Care Survey, Unpublished internal manuscript of the National Center for Health Statistics.

  • Wagner, J. (2010). The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data. Public Opinion Quarterly, 74, 233-243, DOI: http://www.dx.doi.org/10.1093/poq/nfq007. [Web of Science] [Crossref]

Comments (0)

Please log in or register to comment.