Jump to ContentJump to Main Navigation

Clinical Chemistry and Laboratory Medicine (CCLM)

Published in Association with the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM)

Editor-in-Chief: Plebani, Mario

Ed. by Gillery, Philippe / Lackner, Karl J. / Lippi, Giuseppe / Melichar, Bohuslav / Schlattmann, Peter / Tate, Jillian R. / Tsongalis, Gregory J.

12 Issues per year


IMPACT FACTOR 2014: 2.707
Rank 6 out of 30 in category Medical Laboratory Technology in the 2014 Thomson Reuters Journal Citation Report/Science Edition

SCImago Journal Rank (SJR) 2014: 0.741
Source Normalized Impact per Paper (SNIP) 2014: 1.011
Impact per Publication (IPP) 2014: 2.310

VolumeIssuePage

Issues

Statistical methods for assessment of added usefulness of new biomarkers

Michael J. Pencina1, 4, 5 / Ralph B. D'Agostino2, 4 / Ramachandran S. Vasan3, 4

1Department of Biostatistics, Boston University, Boston, MA, USA

2Department of Mathematics and Statistics, Boston University, Boston, MA, USA

3School of Medicine, Boston University, Boston, MA, USA

4NHLBI's Framingham Heart Study, Framingham, MA, USA

5Harvard Clinical Research Institute, Boston, MA, USA

Corresponding authors: Michael J. Pencina, PhD, Department of Biostatistics, Boston University, Framingham Heart Study, Harvard Clinical Research Institute, 111 Cummington St., Boston, MA 02215, USA Phone: +1 617-358-3386, Fax: +1 617-353-4767, Ramachandran S. Vasan, MD, DM, FACC, Boston University School of Medicine, Framingham Heart Study, 73 Mount Wayte Avenue, Suite 2, Framingham, MA 01702-5803, USA Phone: +1 508-935-3450,

Citation Information: Clinical Chemistry and Laboratory Medicine. Volume 48, Issue 12, Pages 1703–1711, ISSN (Online) 1437-4331, ISSN (Print) 1434-6621, DOI: 10.1515/CCLM.2010.340, August 2010

Publication History

Received:
2010-02-11
Accepted:
2010-06-18
Published Online:
2010-08-18

Abstract

The discovery and development of new biomarkers continues to be an exciting and promising field. Improvement in prediction of risk of developing disease is one of the key motivations in these pursuits. Appropriate statistical measures are necessary for drawing meaningful conclusions about the clinical usefulness of these new markers. In this review, we present several novel metrics proposed to serve this purpose. We use reclassification tables constructed on the basis of clinically meaningful disease risk categories to discuss the concepts of calibration, risk separation, risk discrimination, and risk classification accuracy. We discuss the notion that the net reclassification improvement (NRI) is a simple yet informative way to summarize information contained in risk reclassification tables. In the absence of meaningful risk categories, we suggest a ‘category-less’ version of the NRI and integrated discrimination improvement as metrics to summarize the incremental value of new biomarkers. We also suggest that predictiveness curves be preferred to receiver operating characteristic curves as visual descriptors of a statistical model's ability to separate predicted probabilities of disease events. Reporting of standard metrics, including measures of relative risk and the c statistic, is still recommended. These concepts are illustrated with a risk prediction example using data from the Framingham Heart Study.

Clin Chem Lab Med 2010;48:1703–11.

Keywords: calibration; discrimination; integrated discrimination improvement; net reclassification improvement; reclassification; risk prediction

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Petter Bjornstad, Robert H. Eckel, Laura Pyle, Marian Rewers, David M. Maahs, and Janet K. Snell-Bergeon
The American Journal of Cardiology, 2015
[2]
Behnam Heidari, Arash Aghajani Nargesi, Nima Hafezi-Nejad, Sara Sheikhbahaei, Atieh Pajouhi, Manouchehr Nakhjavani, and Alireza Esteghamati
American Heart Journal, 2015
[3]
Roberto Elosua, Gavin Lucas, and Carla Lluis-Ganella
Current Cardiovascular Risk Reports, 2013, Volume 7, Number 1, Page 33
[4]
Gethin W. Hodges, Casper N. Bang, Kristian Wachtell, Jesper Eugen-Olsen, and Jørgen L. Jeppesen
Canadian Journal of Cardiology, 2015
[5]
Koki Mise, Junichi Hoshino, Toshiharu Ueno, Ryo Hazue, Keiichi Sumida, Rikako Hiramatsu, Eiko Hasegawa, Masayuki Yamanouchi, Noriko Hayami, Tatsuya Suwabe, Naoki Sawa, Takeshi Fujii, Shigeko Hara, Kenichi Ohashi, Kenmei Takaichi, and Yoshifumi Ubara
Diabetes/Metabolism Research and Reviews, 2015, Page n/a
[6]
Michael C. Kontos, Benjamin M. Scirica, Anita Y. Chen, Laine Thomas, Monique L. Anderson, Deborah B. Diercks, James G. Jollis, and Matthew T. Roe
American Heart Journal, 2015, Volume 169, Number 4, Page 515
[7]
Mohammadreza Bozorgmanesh, Mahsa Sardarinia, Farhad Hajsheikholeslami, Fereidoun Azizi, and Farzad Hadaegh
European Journal of Nutrition, 2015
[8]
Setor K. Kunutsor, Stephan J.L. Bakker, Jenny E. Kootstra-Ros, Ronald T. Gansevoort, and Robin P.F. Dullaart
Atherosclerosis, 2015, Volume 238, Number 2, Page 356
[9]
I. Drion, N. Kleefstra, G. W. D. Landman, A. Alkhalaf, J. Struck, K. H. Groenier, S. J. L. Bakker, and H. J. G. Bilo
Diabetes Care, 2012, Volume 35, Number 11, Page 2354
[10]
Frederick R. Appelbaum, Claudio Anasetti, Joseph H. Antin, Harold Atkins, Stella Davies, Steven Devine, Sergio Giralt, Helen Heslop, Ginna Laport, Stephanie J. Lee, Brent Logan, Marcelo Pasquini, Michael Pulsipher, Edward Stadtmauer, John R. Wingard, and Mary M. Horowitz
Biology of Blood and Marrow Transplantation, 2015, Volume 21, Number 2, Page 202
[11]
Justin L. Grodin, Sarah Neale, Yuping Wu, Stanley L. Hazen, and W.H. Wilson Tang
The American Journal of Medicine, 2015, Volume 128, Number 3, Page 276
[12]
Andrei P. Drabovich, Eduardo Martínez-Morillo, and Eleftherios P. Diamandis
Biochimica et Biophysica Acta (BBA) - Proteins and Proteomics, 2015, Volume 1854, Number 6, Page 677
[13]
Thor Ueland, Lars Gullestad, Ståle H. Nymo, Arne Yndestad, Pål Aukrust, and Erik T. Askevold
Clinica Chimica Acta, 2015, Volume 443, Page 71
[14]
Ewout W. Steyerberg, Moniek M. Vedder, Maarten J. G. Leening, Douwe Postmus, Ralph B. D'Agostino, Ben Van Calster, and Michael J. Pencina
Biometrical Journal, 2014, Page n/a
[15]
Gerard D. van Rijsbergen, Huibert Burger, Steven D. Hollon, Hermien J. Elgersma, Gemma D. Kok, Jack Dekker, Peter J. de Jong, and Claudi L.H. Bockting
Psychiatry Research, 2014, Volume 220, Number 1-2, Page 287
[16]
Petter Bjornstad, David M. Maahs, Christopher J. Rivard, Laura Pyle, Marian Rewers, Richard J. Johnson, and Janet K. Snell-Bergeon
Acta Diabetologica, 2014, Volume 51, Number 5, Page 783
[17]
Tariq Ahmad, Mona Fiuzat, Michael J. Pencina, Nancy L. Geller, Faiez Zannad, John G.F. Cleland, James V. Snider, Stephan Blankenberg, Kirkwood F. Adams, Rita F. Redberg, Jae B. Kim, Alice Mascette, Robert J. Mentz, Christopher M. O'Connor, G. Michael Felker, and James L. Januzzi
JACC: Heart Failure, 2014, Volume 2, Number 5, Page 477
[18]
Petra G. van Peet, Anton J.M. de Craen, Jacobijn Gussekloo, and Wouter de Ruijter
AGE, 2014, Volume 36, Number 3
[19]
Vanessa Xanthakis, Lisa M. Sullivan, Ramachandran S. Vasan, Emelia J. Benjamin, Joseph M. Massaro, Ralph B. D’Agostino, and Michael J. Pencina
Statistics in Medicine, 2014, Volume 33, Number 15, Page 2577
[20]
Alireza Esteghamati, Nima Hafezi-Nejad, Ali Zandieh, Sara Sheikhbahaei, Maryam Ebadi, and Manouchehr Nakhjavani
Journal of Cardiology, 2014, Volume 64, Number 4, Page 290
[21]
Zhi-De Hu, Xiao-Fei Liu, Xiao-Cui Liu, Chun-Mei Ding, and Cheng-Jin Hu
Clinica Chimica Acta, 2014, Volume 433, Page 44
[22]
Lori B. Daniels, Paul Clopton, Gail A. Laughlin, Alan S. Maisel, and Elizabeth Barrett-Connor
American Heart Journal, 2014, Volume 167, Number 5, Page 674
[23]
A. Massot, D. Giralt, A. Penalba, T. Garcia-Berrocoso, M. Navarro-Sobrino, J.F. Arenillas, M. Ribó, C.A. Molina, J. Alvarez-Sabín, J. Montaner, and P. Delgado
Atherosclerosis, 2014, Volume 233, Number 1, Page 186
[24]
Xiao-Fei Liu, Zhi-De Hu, Xiao-Cui Liu, Yuan Cao, Chun-Mei Ding, and Cheng-Jin Hu
Clinical Biochemistry, 2014, Volume 47, Number 3, Page 196
[25]
E. Giannitsis and H. A. Katus
European Heart Journal, 2014, Volume 35, Number 5, Page 268
[26]
Andre Pascal Kengne, Katya Masconi, Vivian Nchanchou Mbanya, Alain Lekoubou, Justin Basile Echouffo-Tcheugui, and Tandi E. Matsha
Critical Reviews in Clinical Laboratory Sciences, 2014, Volume 51, Number 1, Page 1
[27]
Alejandro Bustamante, Carmen Dominguez, Víctor Rodriguez-Sureda, Angel Vilches, Ana Penalba, Dolors Giralt, Teresa García-Berrocoso, Victor Llombart, Alan Flores, Marta Rubiera, Carlos Molina, José Alvarez-Sabín, and Joan Montaner
International Journal of Stroke, 2014, Volume 9, Number 7, Page 910
[28]
Richa Sharma, Stephanie Macy, Kara Richardson, Yuliya Lokhnygina, and Daniel T. Laskowitz
Journal of Stroke and Cerebrovascular Diseases, 2014, Volume 23, Number 5, Page 910
[29]
Steven Grover, Jean Grégoire, and Todd J. Anderson
Canadian Journal of Cardiology, 2014, Volume 30, Number 2, Page 248.e5
[31]
Sophie Paczesny, Nisha Raiker, Sam Brooks, and Christy Mumaw
International Journal of Hematology, 2013, Volume 98, Number 3, Page 275
[32]
J. M. T. Hendriksen, G. J. Geersing, K. G. M. Moons, and J. A. H. de Groot
Journal of Thrombosis and Haemostasis, 2013, Volume 11, Page 129
[33]
Stefania Mondello, Kara Schmid, Rachel P. Berger, Firas Kobeissy, Domenico Italiano, Andreas Jeromin, Ronal L. Hayes, Frank C. Tortella, and Andras Buki
Medicinal Research Reviews, 2014, Volume 34, Number 3, Page 503
[34]
Michael G. Shlipak and Erica C. Day
Nature Reviews Nephrology, 2013, Volume 9, Number 8, Page 478
[35]
Arkendra De, Kristen Meier, Rong Tang, Meijuan Li, Thomas Gwise, Shanti Gomatam, and Gene Pennello
Journal of Cardiovascular Translational Research, 2013, Volume 6, Number 4, Page 449
[36]
Siri Lillegraven, Nina Paynter, Femke H. M. Prince, Nancy A. Shadick, Espen A. Haavardsholm, Michelle L. Frits, Christine K. Iannaccone, Tore K. Kvien, Michael E. Weinblatt, and Daniel H. Solomon
Arthritis Care & Research, 2013, Volume 65, Number 4, Page 526
[37]
David Salat, Anna Penalba, Teresa García-Berrocoso, Mireia Campos-Martorell, Alan Flores, Jorge Pagola, Alejandro Bustamante, Manuel Quintana, Dolors Giralt, Carlos Molina, José Alvarez-Sabín, and Joan Montaner
Cerebrovascular Diseases, 2013, Volume 35, Number 3, Page 220
[38]
S. Apostolakis, M. Zubaid, W.A. Rashed, A.A. Alsheikh-Ali, W. Almahmeed, A. Shehab, K. Sulaiman, A. AlQudaimi, N. Asaad, H. Amin, D.A. Lane, and G.Y.H. Lip
International Journal of Cardiology, 2013, Volume 168, Number 2, Page 1644
[39]
Carlos Iribarren and Sabee Molloi
Current Cardiovascular Risk Reports, 2013, Volume 7, Number 2, Page 126
[40]
H. E. Wang, G. Jain, R. J. Glassock, and D. G. Warnock
Nephrology Dialysis Transplantation, 2013, Volume 28, Number 6, Page 1447
[41]
J. R. Dahlen, M. J. Price, H. Parise, and P. A. Gurbel
Thrombosis and Haemostasis, 2012, Volume 109, Number 5, Page 808
[42]
Rakesh K. Mishra, Yongmei Li, Ana C. Ricardo, Wei Yang, Martin Keane, Magdalena Cuevas, Robert Christenson, Christopher DeFilippi, Jing Chen, Jiang He, Radhakrishna R. Kallem, Dominic S. Raj, Jeffrey R. Schelling, Jackson Wright, Alan S. Go, and Michael G. Shlipak
The American Journal of Cardiology, 2013, Volume 111, Number 3, Page 432
[43]
Julio A. Chirinos, Jan G. Kips, David R. Jacobs, Lyndia Brumback, Daniel A. Duprez, Richard Kronmal, David A. Bluemke, Raymond R. Townsend, Sebastian Vermeersch, and Patrick Segers
Journal of the American College of Cardiology, 2012, Volume 60, Number 21, Page 2170
[44]
Davide Capodanno, Carmelita Marcantoni, Margherita Ministeri, Fabio Dipasqua, Luca Zanoli, Stefania Rastelli, Sarah Mangiafico, Maria Sanfilippo, Gaetano Romano, and Corrado Tamburino
International Journal of Cardiology, 2013, Volume 168, Number 1, Page 396
[45]
Caroline Courtais, Nils Kuster, Anne-Marie Dupuy, Margit Folschveiller, Riad Jreige, Anne-Sophie Bargnoux, Julie Guiot, Sophie Lefebvre, Jean-Paul Cristol, and Mustapha Sebbane
The American Journal of Emergency Medicine, 2013, Volume 31, Number 1, Page 215
[46]
Thomas G. Liman, Vera Zietemann, Silke Wiedmann, Gerhard J. Jungehuelsing, Matthias Endres, Frank A. Wollenweber, Ian Wellwood, Martin Dichgans, and Peter U. Heuschmann
International Journal of Stroke, 2013, Volume 8, Number 6, Page 484
[47]
Gregory E. Rice, Sebastian E. Illanes, and Murray D. Mitchell
International Journal of Endocrinology, 2012, Volume 2012, Page 1
[48]
Thor Ueland, Asbjørn Svardal, Erik Øie, Erik T. Askevold, Ståle H. Nymoen, Bodil Bjørndal, Christen P. Dahl, Lars Gullestad, Rolf K. Berge, and Pål Aukrust
International Journal of Cardiology, 2013, Volume 167, Number 5, Page 1892
[49]
Niku Oksala, Ilkka Seppälä, Jussi Hernesniemi, Leo-Pekka Lyytikäinen, Mika Kähönen, Kari-Matti Mäkelä, Antti Reunanen, Antti Jula, Mika Ala-Korpela, and Terho Lehtimäki
Annals of Medicine, 2013, Volume 45, Number 2, Page 141
[50]
Liisa Byberg, Rolf Gedeborg, Thomas Cars, Johan Sundström, Lars Berglund, Lena Kilander, Håkan Melhus, and Karl Michaëlsson
Journal of Bone and Mineral Research, 2012, Volume 27, Number 4, Page 797
[51]
A. Ganna, M. Reilly, U. de Faire, N. Pedersen, P. Magnusson, and E. Ingelsson
American Journal of Epidemiology, 2012, Volume 175, Number 7, Page 715
[52]
Andreas P. Kalogeropoulos, Vasiliki V. Georgiopoulou, Christopher R. deFilippi, John S. Gottdiener, and Javed Butler
JACC: Cardiovascular Imaging, 2012, Volume 5, Number 2, Page 131
[53]
K. T. HayGlass
Clinical & Experimental Allergy, 2011, Volume 41, Number 12, Page 1656
[54]
G. Lippi, E. J Favaloro, and M. Plebani
International Journal of Clinical Practice, 2011, Volume 65, Number 12, Page 1221
[55]
Daniela Giannessi
Pharmacological Research, 2011, Volume 64, Number 1, Page 11
[56]
Maria Carmo P. Nunes, Rodrigo Citton Padilha Reis, Enrico Antônio Colosimo, Antônio Luiz P. Ribeiro, and Manoel O.C. Rocha
International Journal of Cardiology, 2011, Volume 150, Number 2, Page 224
[57]
Stanley S. Levinson
Clinical Chemistry and Laboratory Medicine, 2011, Volume 49, Number 7
[58]
Adriana Unić, Dunja Rogić, and Gordana Rajsman
Journal of Medical Biochemistry, 2011, Volume 30, Number 3
[59]
John P. A. Ioannidis
PROTEOMICS - Clinical Applications, 2011, Volume 5, Number 5-6, Page 241
[60]
Karl J. Lackner
Clinical Chemistry and Laboratory Medicine, 2010, Volume 48, Number 12

Comments (0)

Please log in or register to comment.