Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Official Statistics

The Journal of Statistics Sweden

4 Issues per year

IMPACT FACTOR 2016: 0.411
5-year IMPACT FACTOR: 0.776

CiteScore 2016: 0.63

SCImago Journal Rank (SJR) 2016: 0.710
Source Normalized Impact per Paper (SNIP) 2016: 0.975

Open Access
See all formats and pricing
More options …

The Relative Impacts of Design Effects and Multiple Imputation on Variance Estimates: A Case Study with the 2008 National Ambulatory Medical Care Survey

Taylor Lewis / Elizabeth Goldberg / Nathaniel Schenker / Vladislav Beresovsky / Susan Schappert / Sandra Decker / Nancy Sonnenfeld / Iris Shimizu
Published Online: 2014-02-14 | DOI: https://doi.org/10.2478/jos-2014-0008


The National Ambulatory Medical Care Survey collects data on office-based physician care from a nationally representative, multistage sampling scheme where the ultimate unit of analysis is a patient-doctor encounter. Patient race, a commonly analyzed demographic, has been subject to a steadily increasing item nonresponse rate. In 1999, race was missing for 17 percent of cases; by 2008, that figure had risen to 33 percent. Over this entire period, single imputation has been the compensation method employed. Recent research at the National Center for Health Statistics evaluated multiply imputing race to better represent the missing-data uncertainty. Given item nonresponse rates of 30 percent or greater, we were surprised to find many estimates’ ratios of multiple-imputation to single-imputation estimated standard errors close to 1. A likely explanation is that the design effects attributable to the complex sample design largely outweigh any increase in variance attributable to missing-data uncertainty.

Keywords: Health survey; missing data; item nonresponse; fraction of missing information


  • Andridge, R. and Little, R. (2010). A Review of Hot Deck Imputation for Survey Non-Response. International Statistical Review, 78, 40-64, DOI: http://www.dx.doi. org/ 10.1111/j.1751-5823.2010.00103.x.CrossrefWeb of ScienceGoogle Scholar

  • Cleveland, W. (1979). Robust Locally Weighted Regression and Smoothing Scatterplots. Journal of the American Statistical Association, 74, 829-836, DOI: http://www.dx.doi. org/10.1080/01621459.1979.10481038.CrossrefGoogle Scholar

  • Efron, B. (1994). Missing Data, Imputation, and the Bootstrap. Journal of the American Statistical Association, 89, 463-475, DOI: http://www.dx.doi.org/10.1080/01621459.1994.10476768.CrossrefGoogle Scholar

  • Groves, R., Dillman, D., Eltinge, J., and Little, R., (Eds.) (2002). Survey Nonresponse. New York, NY: Wiley.Google Scholar

  • Kalton, G. and Kasprzyk, D. (1986). The Treatment of Missing Survey Data. Survey Methodology, 12, 1-16.Google Scholar

  • Kish, L. (1965). Survey Sampling. New York, NY: Wiley.Google Scholar

  • Kozak, J. (1995). Underreporting of Race in the National Hospital Discharge Survey. Advance Data from Vital and Health Statistics, No. 265. Hyattsville, MD: National Center for Health Statistics.Google Scholar

  • Li, Y., Lynch, C., Shimizu, I., and Kaufman, S. (2004). Imputation Variance Estimation by Bootstrap Method for the National Ambulatory Medical Care Survey, Proceedings of the Survey Research Methods Section of the American Statistical Association.Google Scholar

  • Lohr, S. (1999). Sampling: Design and Analysis. Pacific Grove, CA: Brooks/Cole.Google Scholar

  • McCarthy, P. and Snowden, C. (1985). The Bootstrap and Finite Population Sampling. Vital Health Statistics, 2(95). Hyattsville, MD: National Center for Health Statistics.Google Scholar

  • National Center for Health Statistics (2009). 2008 NAMCS Micro-Data File Documentation. Division of Health Care Surveys, National Center for Health Statistics, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services, Hyattsville, MD, Available online at: ftp://ftp.cdc.gov/pub/Health_Statistics/ NCHS/Dataset_Documentation/NAMCS/doc08.pdf (accessed January 2014).Google Scholar

  • Office of Management and Budget (1997). Revisions to the Standards for the Classification of Federal Data on Race and Ethnicity, Federal Register 62FR58781-58790. Available at: http://www.gpo.gov/fdsys/pkg/FR-1997-10-30/pdf/97-28653.pdf (accessed January 2014).Google Scholar

  • Raghunathan, T., Lepkowski, J., Van Hoewyk, J., and Solenberger, P. (2001). A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models. Survey Methodology, 27, 85-95.Google Scholar

  • Reiter, J., Raghunathan, T., and Kinney, S. (2006). The Importance of Modeling the Sampling Design in Multiple Imputation for Missing Data. Survey Methodology, 32, 143-150.Google Scholar

  • Rubin, D. (1987). Multiple Imputation for Nonresponse in Surveys. New York, NY: Wiley.Google Scholar

  • Rubin, D. (1996). Multiple Imputation After 18 þ Years (with discussion). Journal of the American Statistical Association, 91, 473-489.Google Scholar

  • Rubin, D. and Schenker, N. (1986). Multiple Imputation for Interval Estimation from Simple Random Samples with Ignorable Nonresponse. Journal of the American Statistical Association, 81, 366-374, DOI: http://www.dx.doi.org/10.1080/01621459.1986.10478280.CrossrefGoogle Scholar

  • Schenker, N., Borrud, L., Burt, V., Curtin, L., Flegal, K., Hughes, J., Johnson, C., Looker, A., and Mirel, L. (2011). Multiple Imputation of Missing Dual-Energy X-Ray Absorptiometry Data in the National Health and Nutrition Examination Survey. Statistics in Medicine, 30, 260-276, DOI: http://www.dx.doi.org/10.1002/sim.4080.CrossrefGoogle Scholar

  • Shao, J. and Sitter, R. (1996). Bootstrap for Imputed Survey Data. Journal of the American Statistical Association, 91, 1278-1288, DOI: http://www.dx.doi.org/10.1080/01621459.1996.10476997.CrossrefGoogle Scholar

  • Valverde, R. and Marsteller, J. (2007). A Revised Matching Routine for Imputing Missing Race and Ethnicity in the National Ambulatory Medical Care Survey, Unpublished internal manuscript of the National Center for Health Statistics.Google Scholar

  • Wagner, J. (2010). The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data. Public Opinion Quarterly, 74, 233-243, DOI: http://www.dx.doi.org/10.1093/poq/nfq007. Web of ScienceCrossrefGoogle Scholar

About the article

Published Online: 2014-02-14

Published in Print: 2014-03-01

Citation Information: Journal of Official Statistics, Volume 30, Issue 1, Pages 147–161, ISSN (Online) 2001-7367, DOI: https://doi.org/10.2478/jos-2014-0008.

Export Citation

This content is open access.

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Ellen T. Kurtzman, Burt S. Barnow, Jean E. Johnson, Samuel J. Simmens, Donna Lind Infeld, and Fitzhugh Mullan
Health Services Research, 2017, Volume 52, Page 437

Comments (0)

Please log in or register to comment.
Log in