Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Statistical Applications in Genetics and Molecular Biology

Editor-in-Chief: Sanguinetti, Guido

IMPACT FACTOR 2018: 0.536
5-year IMPACT FACTOR: 0.764

CiteScore 2018: 0.49

SCImago Journal Rank (SJR) 2018: 0.316
Source Normalized Impact per Paper (SNIP) 2018: 0.342

Mathematical Citation Quotient (MCQ) 2018: 0.02

See all formats and pricing
More options …
Volume 7, Issue 1


Volume 10 (2011)

Volume 9 (2010)

Volume 6 (2007)

Volume 5 (2006)

Volume 4 (2005)

Volume 2 (2003)

Volume 1 (2002)

Calculating Confidence Intervals for Prediction Error in Microarray Classification Using Resampling

Wenyu Jiang / Sudhir Varma
  • Genomics and Bioinformatics Group, Laboratory of Molecular Pharmacology, National Cancer Institute
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Richard Simon
  • Biometric Research Branch, Division of Cancer Treatment and Diagnosis, National Cancer Institute
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2008-03-01 | DOI: https://doi.org/10.2202/1544-6115.1322

Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy.We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction.The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.

Keywords: microarray study; class prediction; prediction error; confidence interval; resampling; bootstrap; cross-validation

About the article

Published Online: 2008-03-01

Citation Information: Statistical Applications in Genetics and Molecular Biology, Volume 7, Issue 1, ISSN (Online) 1544-6115, DOI: https://doi.org/10.2202/1544-6115.1322.

Export Citation

©2011 Walter de Gruyter GmbH & Co. KG, Berlin/Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Jonathan D Rosenblatt, Yuval Benjamini, Roee Gilron, Roy Mukamel, and Jelle J Goeman
Biostatistics, 2019
Daniel Stahl, Andrew Pickles, Mayada Elsabbagh, Mark H. Johnson, and The BASIS Team
Developmental Neuropsychology, 2012, Volume 37, Number 3, Page 274
Wytze Laméris, Adrienne van Randen, Thomas M. van Gulik, Olivier R. C. Busch, Jasper Winkelhagen, Patrick M. M. Bossuyt, Jaap Stoker, and Marja A. Boermeester
Diseases of the Colon & Rectum, 2010, Volume 53, Number 6, Page 896
Linda S Lee, Anna E Szafranska-Schwarzbach, Dennis Wylie, Leona A Doyle, Andrew M Bellizzi, Vivek Kadiyala, Shadeah Suleiman, Peter A Banks, Bernard F Andruss, and Darwin L Conwell
Clinical and Translational Gastroenterology, 2014, Volume 5, Number 1, Page e47
Kevin K. Dobbin and Stephanie Cooke
Bioinformatics, 2011, Volume 27, Number 22, Page 3129
Emilie Chary, Hélène Amieva, Karine Pérès, Jean-Marc Orgogozo, Jean-François Dartigues, and Hélène Jacqmin-Gadda
Alzheimer's & Dementia, 2013, Volume 9, Number 5, Page 562
Edoardo Missiaglia, Dan Williamson, Julia Chisholm, Pratyaksha Wirapati, Gaëlle Pierron, Fabien Petel, Jean-Paul Concordet, Khin Thway, Odile Oberlin, Kathy Pritchard-Jones, Olivier Delattre, Mauro Delorenzi, and Janet Shipley
Journal of Clinical Oncology, 2012, Volume 30, Number 14, Page 1670
Le Chang and Doris Y. Tsao
Cell, 2017, Volume 169, Number 6, Page 1013
Simone Wahl, Anne-Laure Boulesteix, Astrid Zierer, Barbara Thorand, and Mark A. van de Wiel
BMC Medical Research Methodology, 2016, Volume 16, Number 1
Chao Wang, Michelle E. Costanzo, Paul E. Rapp, David Darmon, Kylee Bashirelahi, Dominic E. Nathan, Christopher J. Cellucci, Michael J. Roy, and David O. Keyser
Frontiers in Psychiatry, 2017, Volume 8
Blaise Hanczar and Edward R. Dougherty
Pattern Recognition, 2013, Volume 46, Number 3, Page 1067
Jyothi Subramanian and Richard Simon
Statistics in Medicine, 2011, Volume 30, Number 6, Page 642

Comments (0)

Please log in or register to comment.
Log in