Accessible Requires Authentication Published by De Gruyter May 20, 2020

Evaluating Visualisations in Voting Advice Applications

Bastiaan Bruinsma ORCID logo

Abstract

While the design of voting advice applications (VAAs) is witnessing an increasing amount of attention, one aspect has until now been overlooked: its visualisations. This is remarkable, as it are those visualisations that communicate to the user the advice of the VAA. Therefore, this article aims to provide a first look at which visualisations VAAs adopt, why they adopt them, and how users comprehend them. For this, I will look at how design choices, specifically those on matching, influence the type of visualisation VAAs not only do but also have to, use. Second, I will report the results of a small-scale experiment that looked if all users comprehend similar visualisations in the same way. Here, I find that this is often not the case and that the interpretations of the users often differ. These first results suggest that VAA visualisations are wrongly underappreciated and demand closer attention of VAA designers.

Acknowledgements

Parts of this work were first presented at the ECPR General Conference, University of Hamburg (August 22–25, 2018). I would like to thank the reviewers there, as well as the anonymous reviewers of Statistics, Politics and Policy for their helpful comments. Any additional errors remain my responsibility.

Appendix

Overview of Voting Advice Applications:

Name Country Years
Educatieve Stemtest Belgium 2016
Euandi European Union 2014, 2019
iVoter Taiwan 2012, 2016
Kiesbalans The Netherlands 2017
Kieskompas The Netherlands 2006, 2010, 2012, 2017
Mainichi Japan 2009, 2012
ParteieNavi Germany 2013
Smartvote Switzerland 2011, 2015
StemWijzer The Netherlands 2006, 2010, 2012, 2017
Vaalikone Finland 2004, 2006, 2009, 2012
Vote Compass United States 2012, 2016
Wahl-O-Mat Germany 2005, 2009, 2013
Who Shall I Vote For? United Kingdom 2015
Who Should You Vote For? United Kingdom 2015

References

Albo, Y., J. Lanir, P. Bak and S. Rafaeli (2016) “Off the Radar: Comparative Evaluation of Radial Visualization Solutions for Composite Indicators,” IEEE Transactions on Visualization and Computer Graphics, 22:569–578. Search in Google Scholar

Alvarez, R. M., I. Levin, A. H. Trechsel and K. Vassil (2014) “Voting Advice Applications: How Useful and for Whom?” Journal of Information Technology & Politics, 11:82–101. Search in Google Scholar

Baka, A., L. Figgou and V. Triga (2012) “‘Neither Agree, Nor Disagree’: A Critical Analysis of the Middle Answer Category in Voting Advice Applications,” International Journal of Electronic Governance, 5:244–263. Search in Google Scholar

Benoit, K. and M. Laver (2006) Party Policy in Modern Democracies. London: Routledge. Search in Google Scholar

Benoit, K. and M. Laver (2012) “The Dimensionality of Political Space: Epistemological and methodological considerations,” European Union Politics, 13:194–218. Search in Google Scholar

Carpendale, S. (2008) “Evaluating Information Visualizations,” In: (Kerren, A., J. T. Stasko, J.-D. Fekete and C. North, eds.) Information Visualization: Human-Centered Issues and Perspectives. Berlin/Heidelberg: Springer Berlin Heidelberg, pp. 19–45. Search in Google Scholar

Carpenter, P. A. and P. Shah (1998) “A Model of the Perceptual and Conceptual Processes in Graph Comprehension,” Journal of Experimental Psychology: Applied, 4:75–100. Search in Google Scholar

Cleveland, W. S. (1985) The Elements of Graphing Data. Monterey, CA: Wadsworth. Search in Google Scholar

Cleveland, W. S. and R. McGill (1984) “Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods,” Journal of the American Statistical Association, 79:531–554. Search in Google Scholar

Cleveland, W. S. and R. McGill (1985) “Graphical Perception and Graphical Methods for Analyzing Scientific Data,” Science, 229:828–833. Search in Google Scholar

Dinas, E., A. H. Trechsel and K. Vassil (2014) “A Look into the Mirror: Preferences, Representation and Electoral Participation,” Electoral Studies, 36:290–297. Search in Google Scholar

Downs, A. (1957) An Economic Theory of Democracy. New York: Harper and Brothers. Search in Google Scholar

Draper, G. M., Y. Livnat and R. F. Riesenfeld (2009) “A Survey of Radial Methods for Information Visualization,” IEEE Transactions on Visualization and Computer Graphics, 15:759–776. Search in Google Scholar

Enyedi, Z. (2016) “The Influence of Voting Advice Applications on Preferences, Loyalties and Turnout: An Experimental Study,” Political Studies, 64:1000–1015. Search in Google Scholar

Feldman, R. (2013) “Filled Radar Charts Should not be Used to Compare Social Indicators,” Social Indicators Research, 111:709–712. Search in Google Scholar

Freedman, E. G. and P. Shah (2002) “Toward a Model of Knowledge-Based Graph Comprehension,” In: (Hegarty, M., B. Meyer and N. H. Narayanan, eds.) Diagrammatic Representation and Inference. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 18–30. Search in Google Scholar

Gabel, M. and S. Hix (2002) “Defining the EU Political Space: An Empirical Study of the European Elections Manifestos, 1979–1999,” Comparative Political Studies, 35:934–964. Search in Google Scholar

Garcia-Retamero, R., E. T. Cokely, S. Ghazal and A. Joeris (2016) “Measuring Graph Literacy without a Test: A Brief Subjective Assessment,” Medical Decision Making, 36:854–867, pMID: 27353824. Search in Google Scholar

Garry, J., J. Tilley, N. Matthews, F. Mendez and J. Wheatley (2019) “Does receiving advice from Voter Advice Applications (VAAs) affect public opinion in deeply divided societies? Evidence from a field experiment in Northern Ireland,” Party Politics, 25:854–861. Search in Google Scholar

Garzia, D. and S. Marschall (2014) “The Lausanne Declaration on Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 227–228. Search in Google Scholar

Garzia, D. and S. Marschall (2019) “Voting Advice Applications,” In: Oxford Research Encyclopedia of Politics. Retrieved 12 December. 2019, from https://oxfordre.com/politics/view/10.1093/acrefore/9780190228637.001.0001/acrefore-9780190228637-e-620. Search in Google Scholar

Garzia, D., A. de Angelis and J. Pianzola (2014a) “The impact of Voting Advice Applications on Electoral Participation,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 105–114. Search in Google Scholar

Garzia, D., A. H. Trechsel, K. Vassil and E. Dinas (2014b) “Indirect Campaigning: Past, Present and Future of Voting Advice Applications,” In: (Grofman, B., A. H. Trechsel and M. Franklin, eds.) The Internet and Democracy in Global Perspective. Cham: Springer International, pp. 25–41. Search in Google Scholar

Garzia, D., A. H. Trechsel and A. D. Angelis (2017) “Voting Advice Applications and Electoral Participation: A Multi-Method Study,” Political Communication, 34:424–443. Search in Google Scholar

Gemenis, K. (2018) “The Impact of Voting Advice Applications on Electoral Turnout: Evidence from Greece,” Statistics, Politics and Policy, 9:161–179. Search in Google Scholar

Gemenis, K. and C. van Ham (2014) “Comparing Methods for Estimating Parties’ Positions in Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 33–48. Search in Google Scholar

Germann, M. and F. Mendez (2016) “Dynamic Scale Validation Reloaded,” Quality & Quantity, 50:981–1007. Search in Google Scholar

Germann, M. and K. Gemenis (2019) “Getting Out the Vote With Voting Advice Applications,” Political Communication, 36:149–170. Search in Google Scholar

Germann, M., F. Mendez, J. Wheatley and U. Serdült (2014) “Spatial Maps in Voting Advice Applications: The Case for Dynamic Scale Validation,” Acta Politica, 50:1–25. Search in Google Scholar

Glazer, N. (2011) “Challenges with Graph Interpretation: A Review of the Literature,” Studies in Science Education, 47:183–210. Search in Google Scholar

Holleman, B., N. Kamoen, A. Krouwel, J. van de Pol and C. de Vreese (2016) “Positive vs. Negative: The Impact of Question Polarity in Voting Advice Applications,” PLoS One, 11:1–17. Search in Google Scholar

Huff, D. (1954) How to Lie with Statistics. New York, NY: W.W. Norton & Company. Search in Google Scholar

Iliinsky, N. and J. Steele (2011) Designing Data Visualizations. Sebastopol, CA: O’Reilly. Search in Google Scholar

Katakis, I., N. Tsapatsoulis, F. Mendez, V. Triga and C. Djouvas (2014) “Social Voting Advice Applications – Definitions, Challenges, Datasets and Evaluation,” IEEE Transactions on Cybernetics, 44:1039–1052. Search in Google Scholar

Kirk, A. (2016) Data Visualisation – A Handbook for Data Driven Design. London: Sage. Search in Google Scholar

Kleinnijenhuis, J., J. van de Pol, A. van Hoof and A. Krouwel (2017) “VAAs as Sources of Volatility and Fragmentation: Self-selection effects and genuine effects,” Journal of Elections, Public Opinion and Parties, 27:75–96. Search in Google Scholar

Kosslyn, S. M. (1989) “Understanding Charts and Graphs,” Applied Cognitive Psychology, 3:185–225. Search in Google Scholar

Krouwel, A. and A. van Elfrinkhof (2014) “Combining Strengths of Methods of Party Positioning to counter their Weaknesses: The Development of a new Methodology to calibrate Parties on Issues and ideological Dimensions,” Quality & Quantity, 48:1455–1472. Search in Google Scholar

Lam, H., E. Bertini, P. Isenberg, C. Plaisant and S. Carpendale (2012) “Empirical Studies in Information Visualization: Seven Scenarios,” IEEE Transactions on Visualization and Computer Graphics, 18:1520–1536. Search in Google Scholar

Lefevere, J. and S. Walgrave (2014) “A Perfect Match? The Impact of Statement Selection on Voting Advice Applications’ Ability to Match Voters and Parties,” Electoral Studies, 36:252–262. Search in Google Scholar

Louwerse, T. and M. Rosema (2014) “The Design Effects of Voting Advice Applications: Comparing Methods of calculating Results,” Acta Politica, 49:286–312. Search in Google Scholar

MacDonald, S. E., G. Rabinowitz and O. Listhaug (1995) “Political Sophistication and Models of Issue Voting,” British Journal of Political Science, 25:453–483. Search in Google Scholar

Mahéo, V.-A. (2017) “Information Campaigns and (Under)Privileged Citizens: An Experiment on the Differential Effects of a Voting Advice Application,” Political Communication, 34:511–529. Search in Google Scholar

Marschall, S. and C. K. Schmidt (2010) “The Impact of Voting Indicators: The Case of the German Wahl-O-Mat,” In (Cedroni, L. and D. Garzia, eds.) Voting Advice Applications in Europe: The State of the Art. Napoli: ScriptaWeb, pp. 65–90. Search in Google Scholar

Mendez, F. (2012) “Matching Voters with Political Parties and Candidates: An Empirical Test of Four Algorithms,” International Journal of Electronic Governance, 5:264–278. Search in Google Scholar

Mendez, F. (2014) “What’s Behind a Matching Algorithm? A Critical Assessment of how VAAs produce Voting Recommendations,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 49–66. Search in Google Scholar

Munzner, T. (2015) Visualization Analysis & Design. Boca Raton, FI: CRC Press. Search in Google Scholar

Rosema, M. and T. Louwerse (2016) “Response Scales in Voting Advice Applications: Do Different Designs Produce Different Outcomes?” Policy & Internet, 8:431–456. Search in Google Scholar

Schultze, M. (2014) “Effects of Voting Advice Applications (VAAs) on political knowledge about party positions,” Policy & Internet, 6:46–68. Search in Google Scholar

Shah, P. (2002) “Graph Comprehension: The Role of Format, Content and Individual Differences,” In: (Anderson, M., B. Meyer and P. Olivier, eds.) Diagrammatic Representation and Reasoning, London: Springer London, pp. 173–185. Search in Google Scholar

Shah, P. and J. Hoeffner (2002) “Review of Graph Comprehension Research: Implications for Instruction,” Educational Psychology Review, 14:47–69. Search in Google Scholar

Shah, P., E. G. Freedman and I. Vekiri (2005) “The Comprehension of Quantitative Information in Graphical Displays,” In: (Shah, P. and A. Miyake, eds.) The Cambridge Handbook of Visuospatial Thinking. Cambridge Handbooks in Psychology, New York, NY: Cambridge University Press, pp. 426–476. Search in Google Scholar

Shamim, A., V. Balakrishnan and M. Tahir (2015) “Evaluation of Opinion Visualization Techniques,” Information Visualization, 14:339–358. Search in Google Scholar

Stoet, G. (2010) “PsyToolkit: A software package for programming psychological experiments using Linux,” Behavior Research Methods, 42:1096–1104. Search in Google Scholar

Stoet, G. (2017) “PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments,” Teaching of Psychology, 44:24–31. Search in Google Scholar

Tufte, E. R. (2001) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, second edition. Search in Google Scholar

Walgrave, S., M. Nuytemans and K. Pepermans (2009) “Voting Aid Applications and the Effect of Statement Selection,” West European Politics, 32:1161–1180. Search in Google Scholar

Published Online: 2020-05-20
Published in Print: 2020-06-25

©2020 Walter de Gruyter GmbH, Berlin/Boston