Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter June 26, 2017

Did Shy Trump Supporters Bias the 2016 Polls? Evidence from a Nationally-representative List Experiment

Alexander Coppock ORCID logo

Abstract

Explanations for the failure to predict Donald Trump’s win in the 2016 Presidential election sometimes include the “Shy Trump Supporter” hypothesis, according to which some Trump supporters succumb to social desirability bias and hide their vote preference from pollsters. I evaluate this hypothesis by comparing direct question and list experimental estimates of Trump support in a nationally representative survey of 5290 American adults fielded from September 2 to September 13, 2016. Of these, 32.5% report supporting Trump’s candidacy. A list experiment conducted on the same respondents yields an estimate 29.6%, suggesting that Trump’s poll numbers were not artificially deflated by social desirability bias as the list experiment estimate is actually lower than direct question estimate. I further investigate differences across measurement modes for relevant demographic and political subgroups and find no evidence in support of the “Shy Trump Supporter” hypothesis.

Appendix

Table A1:

Comparing Direct Question and List Experimental Estimates of Trump Support.

SubgroupNUnadjusted estimatesAdjusted estimates
Direct questionList experimentDifferenceDirect questionList experimentDifference
Entire sample529032.5 (0.8)29.6 (3.4)3.0 (3.4)33.6 (0.7)33.0 (2.6)0.6 (2.6)
Strong democrat7264.5 (0.8)−4.3 (7.5)8.8 (7.6)5.0 (0.6)0.8 (2.4)4.2 (2.4)
Not very strong democrat11409.2 (1.2)5.0 (7.0)4.2 (6.9)8.0 (0.7)2.5 (3.2)5.5 (3.2)
Lean democrat40913.0 (2.1)8.4 (11.0)4.6 (11.1)13.0 (1.4)7.6 (5.9)5.5 (6.1)
Independent113120.1 (1.5)24.3 (8.0)−4.2 (8.1)18.8 (1.4)26.5 (6.9)−7.7 (6.9)
Lean republican35173.4 (2.7)55.9 (11.0)17.6 (11.1)59.8 (2.4)48.5 (12.5)11.3 (12.3)
Not very strong republican99068.1 (1.9)72.1 (8.2)−4.0 (8.0)72.4 (1.3)73.2 (6.9)−0.8 (6.9)
Strong republican54390.4 (1.3)80.5 (9.7)9.9 (9.7)84.1 (1.3)89.6 (6.5)−5.5 (6.6)
Less than high school15730.7 (4.2)−4.3 (18.4)35.1 (18.4)33.6 (4.2)15.3 (9.1)18.3 (9.7)
High school or some college279334.4 (1.1)35.8 (4.7)−1.4 (4.6)36.2 (1.0)36.7 (4.2)−0.5 (4.2)
College146530.1 (1.4)27.4 (5.0)2.7 (5.0)32.5 (1.3)35.5 (4.7)−3.0 (4.6)
Graduate school87525.8 (1.7)10.9 (6.7)14.9 (6.5)27.1 (1.6)20.5 (4.7)6.6 (4.5)
Below 20th income percentile103830.7 (1.8)15.6 (8.2)15.1 (7.8)33.3 (1.2)31.0 (4.0)2.4 (3.9)
20th–40th Income percentile136136.6 (1.7)38.4 (6.3)−1.8 (6.2)35.6 (1.1)34.6 (2.8)0.9 (2.7)
40th–60th Income percentile93634.0 (2.1)19.7 (7.3)14.3 (7.3)34.0 (1.3)34.8 (3.3)−0.8 (3.1)
60th–80th Income percentile115133.9 (1.8)40.7 (7.4)−6.8 (7.4)35.4 (1.4)39.7 (5.0)−4.3 (4.9)
Above 80th income percentile80427.2 (2.0)30.5 (8.8)−3.4 (9.0)27.5 (1.4)21.4 (4.7)6.1 (4.7)
Men233236.9 (1.3)33.5 (5.3)3.5 (5.2)38.0 (1.1)33.5 (3.7)4.5 (3.7)
Women295828.4 (1.1)26.4 (4.4)2.0 (4.5)30.1 (1.0)32.7 (4.6)−2.6 (4.6)
White354439.4 (1.1)38.0 (4.1)1.4 (4.1)39.9 (0.9)40.5 (3.3)−0.6 (3.3)
Black44610.3 (1.7)16.6 (10.9)−6.3 (11.0)11.2 (1.7)22.0 (9.0)−10.7 (9.0)
Hispanic80424.4 (2.0)2.8 (9.7)21.6 (9.4)26.2 (1.9)9.0 (5.5)17.2 (5.4)
Other race49620.3 (2.2)24.0 (11.6)−3.8 (11.2)20.7 (2.2)28.5 (10.1)−7.8 (10.0)
Unlikely voter142024.2 (1.4)20.7 (7.1)3.5 (7.1)24.3 (1.3)24.9 (5.9)−0.5 (5.9)
Likely voter387036.5 (1.0)33.6 (3.7)2.9 (3.7)37.0 (0.9)36.0 (2.9)1.0 (2.8)

  1. All estimates incorporate sampling weights.

  2. Bootstrapped standard errors are in parentheses.

  3. Adjusted direction question estimates are predictions from a logistic regression.

  4. Adjusted list experiment estimates are predictions from Imai’s (2011) NLS regression model.

References

Aronow, P. M., A. Coppock, F. W. Crawford and D. P. Green (2015) “Combining List Experiment and Direct Question Estimates of Sensitive Behavior Prevalence,” Journal of Survey Statistics and Methodology, 3:43–66.10.1093/jssam/smu023Search in Google Scholar

Blair, G. and K. Imai (2012) “Statistical Analysis of List Experiments,” Political Analysis, 20:47–77.10.1093/pan/mpr048Search in Google Scholar

Curtice, J. (1997) “So How Well Did They Do? The Polls in the 1997 Election,” Journal of the Market Research Society, 39:449–462.10.1177/147078539703900303Search in Google Scholar

Dropp, K. (2015) “Why Does Donald Trump Perform Better in Online Versus Live Telephone Polling?”.Search in Google Scholar

Dropp, K. (2016) “Yes, There Are Shy Trump Voters. No, They Won’t Swing the Election,” URL https://morningconsult.com/2016/11/03/yes-shy-trump-voters-no-wont-swing-election/.Search in Google Scholar

Durand, C., A. Blais and S. Vachon (2001) “Review: A Late Campaign Swing or a Failure of the Polls? The Case of the 1998 Quebec Election,” The Public Opinion Quarterly, 65:108–123.10.1086/320041Search in Google Scholar

Frye, T., S. Gehlbach, K. L. Marquardt and O. J. Reuter (2016) “Is Putin’s Popularity Real?” Post-Soviet Affairs, 1–15.Search in Google Scholar

Glynn, A. N. (2013) “What Can We Learn with Statistical Truth Serum? Design and Analysis of the List Experiment,” Public Opinion Quarterly, 77:159–172.10.1093/poq/nfs070Search in Google Scholar

Hopkins, D. J. (2009) “No More Wilder Effect, Never a Whitman Effect: When and Why Polls Mislead about Black and Female Candidates,” The Journal of Politics, 71:769–781.10.1017/S0022381609090707Search in Google Scholar

Imai, K. (2011) “Multivariate Regression Analysis for the Item Count Technique,” Journal of the American Statistical Association, 106:407–416.10.1198/jasa.2011.ap10415Search in Google Scholar

Kuklinski, J. H., P. M. Sniderman, K. Knight, T. Piazza, P. E. Tetlock, G. R. Lawrence and B. Mellers (1997) “Racial Prejudice and Attitudes Toward Affirmative Action,” American Journal of Political Science, 41:402–419.10.2307/2111770Search in Google Scholar

LaBrie, J. W. and M. Earleywine (2000) “Sexual Risk Behaviors and Alcohol: Higher Base Rates Revealed Using the Unmatched-Count Technique,” Journal of Sex Research, 37:321–326.10.1080/00224490009552054Search in Google Scholar

Lax, J. R., J. H. Phillips and A. F. Stollwerk (2016) “Are Survey Respondents Lying About Their Support for Same-Sex Marriage? Lessons from a List Experiment,” Public Opinion Quarterly, 80:510–533.10.1093/poq/nfv056Search in Google Scholar

Lyall, J., G. Blair and K. Imai (2013) “Explaining Support for Combatants duringWartime: A Survey Experiment in Afghanistan,” American Political Science Review, 107:679–705.10.1017/S0003055413000403Search in Google Scholar

Mellon, J. and C. Prosser (2017) “Missing Nonvoters and Misweighted Samples: Explaining the 2015 Great British Polling Miss,” Public Opinion Quarterly, forthcoming.10.1093/poq/nfx015Search in Google Scholar

Miller, J. (1984) A New Survey Technique for Studying Deviant Behavior, Phd thesis., George Washington University.Search in Google Scholar

Payne, J. G. (2010) “The Bradley Effect: Mediated Reality of Race and Politics in the 2008 US Presidential Election,” American Behavioral Scientist, 54: 417–435.10.1177/0002764210381713Search in Google Scholar

Powell, R. J. (2013) “Social Desirability Bias in Polling on Same-sex Marriage Ballot Measures,” American Politics Research, 41:1052–1070.10.1177/1532673X13484791Search in Google Scholar

Streb, M. J., B. Burrell, B. Frederick and M. A. Genovese (2008) “Social Desirability Effects and Support for a Female American President,” Public Opinion Quarterly, 72:76–89.10.1093/poq/nfm035Search in Google Scholar

Published Online: 2017-6-26
Published in Print: 2017-10-26

©2017 Walter de Gruyter GmbH, Berlin/Boston