Jump to ContentJump to Main Navigation
Show Summary Details

Journal of Quantitative Analysis in Sports

An official journal of the American Statistical Association

Editor-in-Chief: Mark Glickman PhD

SCImago Journal Rank (SJR) 2015: 0.288
Source Normalized Impact per Paper (SNIP) 2015: 0.358
Impact per Publication (IPP) 2015: 0.250

See all formats and pricing


Ranking rankings: an empirical comparison of the predictive power of sports ranking methods

Daniel Barrow1 / Ian Drayer2 / Peter Elliott2 / Garren Gaut2 / 2

1Pitzer College, Department of Mathematics, 1050 North Mills Avenue, Claremont, CA 91711, USA

2UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA

Corresponding author: Braxton Osting, UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA, Tel.: +3108252601

Citation Information: Journal of Quantitative Analysis in Sports. Volume 9, Issue 2, Pages 187–202, ISSN (Online) 1559-0410, ISSN (Print) 2194-6388, DOI: https://doi.org/10.1515/jqas-2013-0013, May 2013

Publication History

Published Online:


In this paper, we empirically evaluate the predictive power of eight sports ranking methods. For each ranking method, we implement two versions, one using only win-loss data and one utilizing score-differential data. The methods are compared on 4 datasets: 32 National Basketball Association (NBA) seasons, 112 Major League Baseball (MLB) seasons, 22 NCAA Division 1-A Basketball (NCAAB) seasons, and 56 NCAA Division 1-A Football (NCAAF) seasons. For each season of each dataset, we apply 20-fold cross validation to determine the predictive accuracy of the ranking methods. The non-parametric Friedman hypothesis test is used to assess whether the predictive errors for the considered rankings over the seasons are statistically dissimilar. The post-hoc Nemenyi test is then employed to determine which ranking methods have significant differences in predictive power. For all datasets, the null hypothesis – that all ranking methods are equivalent – is rejected at the 99% confidence level. For NCAAF and NCAAB datasets, the Nemenyi test concludes that the implementations utilizing score-differential data are usually more predictive than those using only win-loss data. For the NCAAF dataset, the least squares and random walker methods have significantly better predictive accuracy at the 95% confidence level than the other methods considered.

Keywords: cross validation; Friedman test; Nemenyi test; hypothesis testing; sports rankings

Comments (0)

Please log in or register to comment.