Jump to ContentJump to Main Navigation
Show Summary Details

Statistics, Politics and Policy

Editor-in-Chief: Wagschal, Uwe

Online
ISSN
2151-7509
See all formats and pricing




30,00 € / $42.00 / £23.00

Get Access to Full Text

Substantive Importance and the Veil of Statistical Significance

1 / Carlisle Rainey1

1Department of Political Science, Texas A&M University, 2010 Allen Building, College Station, TX 77843, USA

Corresponding author: Kelly McCaskey, PhD Student, Department of Political Science, Texas A&M University, 2010 Allen Building, College Station, TX 77843, USA, e-mail:

Citation Information: Statistics, Politics and Policy. Volume 6, Issue 1-2, Pages 77–96, ISSN (Online) 2151-7509, ISSN (Print) 2194-6299, DOI: https://doi.org/10.1515/spp-2015-0001, November 2015

Publication History

Published Online:
2015-11-06

Abstract

Political science is gradually moving away from an exclusive focus on statistical significance and toward an emphasis on the magnitude and importance of effects. While we welcome this change, we argue that the current practice of “magnitude-and-significance,” in which researchers only interpret the magnitude of a statistically significant point estimate, barely improves the much-maligned “sign-and-significance” approach, in which researchers focus only on the statistical significance of an estimate. This exclusive focus on the point estimate hides the uncertainty behind a veil of statistical significance. Instead, we encourage researchers to explicitly account for uncertainty by interpreting the range of values contained in the confidence interval. Especially when making judgments about the importance of estimated effects, we advise researchers to make empirical claims if and only if those claims hold for the entire confidence interval.

Comments (1)

Please log in or register to comment.
  • Confidence intervals that don't cover the null value suffer from problems just as significance values do. See my paper Shaffer, J.P. (2004), Confidence intervals on subsets may be misleading, Journal of Modern Applied Statistical Methods, 3, 261-270, plus the errata (2006), 5, 281.

    posted by: Juliet Shaffer on 2015-12-05 06:53 PM (Europe/Berlin)