Jump to ContentJump to Main Navigation
Show Summary Details
More options …

The International Journal of Biostatistics

Ed. by Chambaz, Antoine / Hubbard, Alan E. / van der Laan, Mark J.

IMPACT FACTOR 2017: 0.840
5-year IMPACT FACTOR: 1.000

CiteScore 2017: 0.97

SCImago Journal Rank (SJR) 2017: 1.150
Source Normalized Impact per Paper (SNIP) 2017: 1.022

Mathematical Citation Quotient (MCQ) 2016: 0.09

See all formats and pricing
More options …

A Serious Flaw in Nutrition Epidemiology: A Meta-Analysis Study

Karl E. Peace
  • Corresponding author
  • Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ JingJing Yin
  • Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Haresh Rochani
  • Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Sarbesh Pandeya
  • Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Stanley Young
Published Online: 2018-11-22 | DOI: https://doi.org/10.1515/ijb-2018-0079



Many researchers have studied the relationship between diet and health. Specifically, there are papers showing an association between the consumption of sugar sweetened beverages and Type 2 diabetes. Many meta-analyses use individual studies that do not attempt to adjust for multiple testing or multiple modeling. Hence the claims reported in a meta-analysis paper may be unreliable as the base papers do not ensure unbiased statistics.


Determine (i) the statistical reliability of 10 papers and (ii) indirectly the reliability of the meta-analysis study.


We obtained copies of each of the 10 papers used in a metaanalysis paper and counted the numbers of outcomes, predictors, and covariates. We estimate the size of the potential analysis search space available to the authors of these papers; i. e. the number of comparisons and models available. The potential analysis search space is the number of outcomes times the number of predictors times 2 c , where c is the number of covariates. This formula was applied to information found in the abstracts (Space A) as well as the text (Space T) of each base paper.


The median and range of the number of comparisons possible across the base papers are 6.5 and (2 12,288), respectively for Space A, and 196,608 and (3072–117,117,952), respectively for Space T. It is noted that the median of 6.5 for Space A may be misleading as each study has 60–165 foods that could be predictors.


Given that testing is at the 5% level and the number of comparisons is very large, nominal statistical significance is very weak support for a claim. The claims in these papers are not statistically supported and hence are unreliable so the meta-analysis paper is also unreliable.

Keywords: observational studies; nutritional epidemiology; reliability of claims; multiple testing; multiple modeling; meta-analysis


  • [1]

    Chen DGD, Peace KE. Applied meta-analysis with R. Boca Raton: CRC Press, 2013.Google Scholar

  • [2]

    Ehm W. Meta-analysis of mind-matter experiments: a statistical modeling perspective. Mind Matter. 2005;3:85–132.Google Scholar

  • [3]

    Boos D, Stefanski L. Bayesian inference. Essential statistical inference Vol. 2013. In: EDs Boos D, Stefanski L (eds.). New York: Springer, 2013:163–203.Google Scholar

  • [4]

    Feinstein A. Scientific standards in epidemiologic studies of the menace of daily life. Science. 1988;242:1247–64.Google Scholar

  • [5]

    Mayes L, Horwitz R, Fhnstein A. A collection of 56 topics with contradictory results in case-control research. Int J Epidemiol. 1988;3:680–85.Google Scholar

  • [6]

    Taubes G, Mann C. Epidemiology faces its limits. Science. 1995;269:164.PubMedCrossrefGoogle Scholar

  • [7]

    Ioannidis J. Contradicted and initially stronger effects in highly cited clinical research. Jama. 2005;2:218–28.Google Scholar

  • [8]

    Kaplan S, Billimek J, Sorkin DH, Ngo-Metzger Q, Greenfield S. Who can respond to treatment?: identifying patient characteristics related to heterogeneity of treatment effects. Med Care. 2010;48:S9–S16.CrossrefWeb of ScienceGoogle Scholar

  • [9]

    Young S, Deming KA. Data and observational studies: a process out of control and needing fixing. Significance. 2011;8:116–20.CrossrefGoogle Scholar

  • [10]

    Breslow N. Are statistical contributions to medicine undervalued?. Biometrics. 2003;59:1–8.CrossrefPubMedGoogle Scholar

  • [11]

    Breslow N. Commentary. Biostatistics. 2010;3:379–80.Google Scholar

  • [12]

    Taubes G. 2007. Do we really know what makes us healthy. New York Times, 2007.Google Scholar

  • [13]

    Hughes S. 2007. New York times magazine focuses on pitfalls of epidemiological trials. 2007. http://www.theheart.org/article/813719.

  • [14]

    Wikipedia. 2016. Replication Crisis. https://en.wikipedia.org/wiki/Replicationcrisis.

  • [15]

    Glaeser E. 2006. Researcher incentives and empirical methods. National Bureau of Economic Research.Google Scholar

  • [16]

    Cardwell C, Abnet C, Cantwell M, Murray LJ. Exposure to oral bisphosphonates and risk of esophageal cancer. Jama. 2010;304:657–63.CrossrefPubMedWeb of ScienceGoogle Scholar

  • [17]

    Green J, Czanner G, Reeves G, Watson J, Wise L, Beral V. Oral bisphosphonates and risk of cancer of oesophagus, stomach, and colorectum: case-control analysis within a UK primary care cohort. BMJ. 2010;341:c4444.CrossrefPubMedWeb of ScienceGoogle Scholar

  • [18]

    Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533:452–54.CrossrefPubMedGoogle Scholar

  • [19]

    Malik VS, Popkin B, Bray G, Despre´S J-P, Willett W, Hu F. Sugar-sweetened beverages and risk of metabolic syndrome and type 2 diabetes. Diabetes Care. 2010;33:2477–83.PubMedWeb of ScienceCrossrefGoogle Scholar

  • [20]

    Nettleton J, Lutsey P, Wang Y, Lima J, Michos E, Jacobs D. Diet soda intake and risk of incident metabolic syndrome and type 2 diabetes in the multi-ethnic study of atherosclerosis (MESA). Diabetes Care. 2009;32:688–94.PubMedWeb of ScienceCrossrefGoogle Scholar

  • [21]

    Lutsey P, Stevens J. Dietary intake and the development of the metabolic syndrome. Diabetes Care. 2008;117:754–61.Google Scholar

  • [22]

    Dhingra R, Sullivan L, Jacques P, Wang T, Fox CS, Meigs J, et al. Soft drink consumption and risk of developing cardiometabolic risk factors and the metabolic syndrome. Circulation. 2007;116:480–88.Web of ScienceCrossrefPubMedGoogle Scholar

  • [23]

    Montonen J, Ja¨Rvinen R, Knekt P, Helio¨Vaara M, Reunanen A. Consumption of sweetened beverages and intakes of fructose and glucose predict type 2 diabetes occurrence. J Nutr. 2007;137:1447–54.PubMedCrossrefGoogle Scholar

  • [24]

    Paynter N, Yeh H-C, Voutilainen S, Schmidt M, Heiss G, Folsom A, et al. Coffee and sweetened beverage consumption and the risk of type 2 diabetes mellitus the atherosclerosis risk in communities study. Am J Epidemiol. 2006;164:1075–84.CrossrefPubMedGoogle Scholar

  • [25]

    Schulze M, Manson J, Ludwig DS, Colditz GA, Stampfer M, Willett W, et al. Sugar-sweetened beverages, weight gain, and incidence of type 2 diabetes in young and middle-aged women. Jama. 2004;292:927–34.CrossrefGoogle Scholar

  • [26]

    Palmer JR, Boggs D, Krishnan S, Hu F, Singer M, Rosenberg L. Sugar-sweetened beverages and incidence of type 2 diabetes mellitus in African American women. Arch Intern Med. 2008;164:1075–84.Web of ScienceGoogle Scholar

  • [27]

    Bazzano LA, Li T, Joshipura K, Hu F. Intake of fruit, vegetables, and fruit juices and risk of diabetes in women. Diabetes Care. 2008;31:1311–17.CrossrefPubMedWeb of ScienceGoogle Scholar

  • [28]

    Odegaard A, Koh W-P, Arakawa K, Mimi C, Pereira M. Drink and juice consumption and risk of physician-diagnosed incident type 2 diabetes the Singapore Chinese health study. Am J Epidemiol. 2010;171:701–8.PubMedCrossrefGoogle Scholar

  • [29]

    de Koning L, Malik V, Rimm E, Willett W, Hu FB. Sugarsweetened and artificially sweetened beverage consumption and risk of type 2 diabetes in men. Am J Clin Nutr. 2011;93:1321–27.CrossrefGoogle Scholar

  • [30]

    Royston J. Algorithm AS 177: expected normal order statistics (exact and approximate). J R Stat Soc Ser C (Appl Stat). 1982;31:161–65.Google Scholar

  • [31]

    Blom G. Statistical estimates and transformed beta-variables. Stockholm: Almqvist & Wiksell, 1958:174.Google Scholar

  • [32]

    Lander E, Kruglyak L. Genetic dissection of complex traits: guidelines for interpreting and reporting linkage results. Nat Genet. 1995;11:241–47.CrossrefPubMedGoogle Scholar

  • [33]

    Schweder T, Tvoll ES. Plots of p-values to evaluate many tests simultaneously. Biometrika. 1982;3:493–502.Google Scholar

  • [34]

    Kabat G. Getting risk right: understanding the science of elusive health risks. New York: Columbia University Press, 2016.Google Scholar

  • [35]

    Kass R, Caffo B, Davidian M, Meng XL, Yu B, Reid N. Ten simple rules for effective statistical practice. PLoS Comput Biol. 2016;12(6):e1004961. DOI: .CrossrefWeb of SciencePubMedGoogle Scholar

  • [36]

    Berger R, Casella G. Statistical InferenceWadsworth statistics/probability series. Pacific Grove: Brooks/Cole Publishing Company, 1990.Google Scholar

About the article

Received: 2018-08-10

Accepted: 2018-11-04

Revised: 2018-10-31

Published Online: 2018-11-22

Citation Information: The International Journal of Biostatistics, Volume 14, Issue 2, 20180079, ISSN (Online) 1557-4679, DOI: https://doi.org/10.1515/ijb-2018-0079.

Export Citation

© 2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in