Abstract
The Literary Digest poll of 1936, which incorrectly predicted that Landon would defeat Roosevelt in the 1936 US presidential election, has long been held up as an example of how not to sample. The sampling frame was constructed from telephone directories and automobile registration lists, and the survey had a 24% response rate. But if information collected by the poll about votes cast in 1932 had been used to weight the results, the poll would have predicted a majority of electoral votes for Roosevelt in 1936, and thus would have correctly predicted the winner of the election. We explore alternative weighting methods for the 1936 poll and the models that support them. While weighting would have resulted in Roosevelt being projected as the winner, the bias in the estimates is still very large. We discuss implications of these results for today’s low-response-rate surveys and how the accuracy of the modeling might be reflected better than current practice.
Acknowledgments
The authors are grateful to the editor and referees for their helpful comments.
References
Amateur Statistician (1936) “The Literary Digest Poll: Mr. Franklin’s Mathematical Analysis Evokes Some Criticism,” Letter to the Editor, The New York Times (October 31), 18.Search in Google Scholar
Belli, R. F., M. W. Traugott and M. N. Beckmann (2001) “What Leads to Voting Overreports? Contrasts of Overreporters to Validated Voters and Admitted Nonvoters in the American National Election Studies,” Journal of Official Statistics, 17:479–498.Search in Google Scholar
Berent, M. K., J. A. Krosnick and A. Lupia (2016) “Measuring Voter Registration and Turnout in Surveys: Do Official Government Records Yield More Accurate Assessments?” Public Opinion Quarterly, 80:597–621.10.1093/poq/nfw021Search in Google Scholar
Bowley, A. L. (1926) “Measurement of the Precision Attained in Sampling,” Bulletin of the International Statistical Institute, 22(Supplement to Book I):1–62.Search in Google Scholar
Bryson, M. C. (1976) “The Literary Digest Poll: Making of a Statistical Myth,” The American Statistician, 30:184–185.Search in Google Scholar
Cahalan, D. (1989) “Comment: The Digest Poll Rides Again !” Public Opinion Quarterly, 53:129–133.10.1086/269146Search in Google Scholar
Campbell, J. E. (2010) “Explaining Politics, Not Polls: Reexamining Macropartisanship with Recalibrated NES Data,” Public Opinion Quarterly, 74:616–642.10.1093/poq/nfq042Search in Google Scholar
Cohn, N. (2016a) “Why the Surprise Over ‘Brexit’? Don’t Blame the Polls,” The New York Times (June 24). Available at: https://www.nytimes.com/2016/06/25/upshot/why-the-surprise-over-brexit-dont-blame-the-polls.html.Search in Google Scholar
Cohn, N. (2016b) “How One Illinois Man Distorts National Polls,” The New York Times (October 13), p. A18.Search in Google Scholar
Cornfield, J. (1942) “On Certain Biases in Samples of Human Populations,” Journal of the American Statistical Association, 37:63–68.10.1080/01621459.1942.10500613Search in Google Scholar
Crum, W. L. (1933) “On Analytical Interpretation of Straw-Vote Samples,” Journal of the American Statistical Association, 28:152–163.10.1080/01621459.1933.10502253Search in Google Scholar
Deming, W. E. (1986) Out of the Crisis. Cambridge, MA: MIT Press.Search in Google Scholar
Franklin, F. (1936) “Refiguring the Digest Poll: Returns Otherwise Examined Found to Show Roosevelt Lead,” Letter to the Editor, The New York Times (October 28), 24.Search in Google Scholar
Gallup, G. (1938) “Government and the Sampling Referendum,” Journal of the American Statistical Association, 33:131–142.10.1080/01621459.1938.10503381Search in Google Scholar
Hansen, M. H., W. N. Hurwitz and W. G. Madow (1953) Sample Survey Methods and Theory (Vol. 1: Methods and Applications). New York, NY: John Wiley & Sons.Search in Google Scholar
Harford, T. (2016) “When Forecasters Get it Wrong,” Financial Times (November 26), 45.Search in Google Scholar
Hjort, N. L. and G. Claeskens (2003) “Frequentist Model Average Estimators,” Journal of the American Statistical Association, 98:879–899.10.1198/016214503000000828Search in Google Scholar
Hoeting, J. A., D. Madigan, A. E. Raftery and C. T. Volinsky (1999) “Bayesian Model Averaging: A Tutorial” (with discussion), Statistical Science, 14:382–417.Search in Google Scholar
Laplace, P. (1814) Essai Philosophique sur les Probabilités (no.57), Paris, France: MME VE Courcier, Imprimeur-Libraire pour les Mathématiques, quai des Augustins.Search in Google Scholar
Lauter, D. (2016) “No, One 19-year-old Trump Supporter Probably Isn’t Distorting the Polling Averages All By Himself,” The Los Angeles Times (October 13) [online]. Available at: http://www.latimes.com/politics/la-na-pol-daybreak-poll-questions-20161013-snap-story.html.Search in Google Scholar
Levy, F. F. (1936) “Forecasting the Election: Method Suggested to Correct the Poll Taken by the Literary Digest,” Letter to the Editor, The New York Times (October 16), 24.Search in Google Scholar
Literary Digest (1928a) “Semi-final Figures in ‘The Digest’s’ Big Poll,” Literary Digest, 99 (October 27):10–12.Search in Google Scholar
Literary Digest (1928b) “Final Returns in ‘The Digest’s’ Presidential Poll,” Literary Digest, 99 (November 3):5–7.Search in Google Scholar
Literary Digest (1932a) “‘Digest’ Poll Scrutinized by an Expert,” Literary Digest, 114 (October 8): 37–39.Search in Google Scholar
Literary Digest (1932b) “Roosevelt Bags 41 States Out of 48,” Literary Digest, 114 (November 5): 8, 9, 44, 46, 47.Search in Google Scholar
Literary Digest (1936a) “‘The Digest’ Presidential Poll is On!” Literary Digest, 122 (August 22): 3–4.Search in Google Scholar
Literary Digest (1936b) “First Votes in ‘Digest’s’ 1936 Poll,” Literary Digest, 122 (September 5): 7–8.Search in Google Scholar
Literary Digest (1936c) “Landon, 1,293,669; Roosevelt, 972,897: Final Returns in ‘The Digest’s’ Poll of Ten Million Voters,” Literary Digest, 122 (October 31):5–6.Search in Google Scholar
Literary Digest (1936d) “What Went Wrong with the Polls?” Literary Digest, 122 (November 14): 7–8.Search in Google Scholar
Little, R. J. A. (1993) “Post-Stratification: A Modeler’s Perspective,” Journal of the American Statistical Association, 88:1001–1012.10.1080/01621459.1993.10476368Search in Google Scholar
Little, B. (2016) “Four of History’s Worst Political Predictions,” National Geographic [online] Available at: http://news.nationalgeographic.com/2016/11/presidential-election-predictions-history/.Search in Google Scholar
Lohr, S. (2010) Sampling: Design and Analysis (2nd ed.). Boston, MA: Brooks/Cole.Search in Google Scholar
Lohr, S. and T. Raghunathan (2017) “Combining Survey Data with Other Data Sources,” Statistical Science, in press. Available at: http://imstat.org/sts/future_papers.html.10.1214/16-STS584Search in Google Scholar
Lusinchi, D. (2012) “‘President’ Landon and the 1936 Literary Digest Poll: Were Automobile and Telephone Owners to Blame?” Social Science History, 36:23–54.10.1215/01455532-1461650Search in Google Scholar
National Council of Public Polls (2017) “Election Results,” Available at: http://www.ncpp.org/?q=node/101.Search in Google Scholar
New York Times (1924) “Contest Accuracy of Digest’s Poll,” The New York Times (October 20), 2.Search in Google Scholar
New York Times (1937) “Literary Digest Bought by Shaws,” The New York Times (June 17), 21.Search in Google Scholar
Neyman, J. (1934) “On the Two Different Aspects of the Representative Method: The Method of Stratified Sampling and the Method of Purposive Selection. Journal of the Royal Statistical Society, 97:558–625.10.2307/2342192Search in Google Scholar
Parten, M. (1950) Surveys, Polls, and Samples. New York, NY: Harper & Brothers.Search in Google Scholar
Presser, S., M. W. Traugott and S. Traugott (1990) “Vote ‘Over’ Reporting in Surveys: The Records or the Respondents?” ANES Technical Report Series No. nes010157. Available at: http://electionstudies.org/Library/papers/documents/nes010157.pdf.Search in Google Scholar
Robinson, C. (1932) Straw Votes. New York, NY: Columbia University Press.Search in Google Scholar
Spiegelhalter, D. J. and H. Riesch (2011), “Don’t Know, Can’t Know: Embracing Deeper Uncertainties When Analyzing Risks,” Philosophical Transactions: Mathematical, Physical and Engineering Sciences, 369:4730–4750.10.1098/rsta.2011.0163Search in Google Scholar
Squire, P. (1988) “Why the 1936 Literary Digest Poll Failed,” Public Opinion Quarterly, 52:125–133.10.1086/269085Search in Google Scholar
Sturgis, P., N. Baker, M. Callegaro, S. Fisher, J. Green, W. Jennings, J. Kuha, B. Lauderdale and P. Smith (2016) Report of the Inquiry into the 2015 British General Election Opinion Polls. London: Market Research Society and British Polling Council. Available at: http://eprints.ncrm.ac.uk/3789/1/Report_final_revised.pdf.Search in Google Scholar
Walton Jr., H., C. V. Gray and L. McLemore (2001) “African American Public Opinion and the Pre-Scientific Polls: The Literary Digest Magazine’s Straw-Vote Presidential Polls, 1916–1936,” National Political Science Review, 8:221–243.Search in Google Scholar
Willcox, W. F. (1931) “An Attempt to Measure Public Opinion About Repealing the Eighteenth Amendment,” Journal of the American Statistical Association, 26:243–261.10.1080/01621459.1931.10502544Search in Google Scholar
Wright, G. C. (1993), “Errors in Measuring Vote Choice in the National Election Studies, 1952–88,” American Journal of Political Science, 37:291–316.10.2307/2111533Search in Google Scholar
©2017 Walter de Gruyter GmbH, Berlin/Boston
