Evaluating Visualisations in Voting Advice Applications

  • 1 Goethe University Frankfurt am Main, Faculty of Social Sciences, Theodor-W.-Adorno-Platz 6, 60323 Frankfurt am Main, Germany
Bastiaan BruinsmaORCID iD: https://orcid.org/0000-0002-2556-4940


While the design of voting advice applications (VAAs) is witnessing an increasing amount of attention, one aspect has until now been overlooked: its visualisations. This is remarkable, as it are those visualisations that communicate to the user the advice of the VAA. Therefore, this article aims to provide a first look at which visualisations VAAs adopt, why they adopt them, and how users comprehend them. For this, I will look at how design choices, specifically those on matching, influence the type of visualisation VAAs not only do but also have to, use. Second, I will report the results of a small-scale experiment that looked if all users comprehend similar visualisations in the same way. Here, I find that this is often not the case and that the interpretations of the users often differ. These first results suggest that VAA visualisations are wrongly underappreciated and demand closer attention of VAA designers.

  • Albo, Y., J. Lanir, P. Bak and S. Rafaeli (2016) “Off the Radar: Comparative Evaluation of Radial Visualization Solutions for Composite Indicators,” IEEE Transactions on Visualization and Computer Graphics, 22:569–578.

    • PubMed
    • Export Citation
  • Alvarez, R. M., I. Levin, A. H. Trechsel and K. Vassil (2014) “Voting Advice Applications: How Useful and for Whom?” Journal of Information Technology & Politics, 11:82–101.

  • Baka, A., L. Figgou and V. Triga (2012) “‘Neither Agree, Nor Disagree’: A Critical Analysis of the Middle Answer Category in Voting Advice Applications,” International Journal of Electronic Governance, 5:244–263.

  • Benoit, K. and M. Laver (2006) Party Policy in Modern Democracies. London: Routledge.

  • Benoit, K. and M. Laver (2012) “The Dimensionality of Political Space: Epistemological and methodological considerations,” European Union Politics, 13:194–218.

  • Carpendale, S. (2008) “Evaluating Information Visualizations,” In: (Kerren, A., J. T. Stasko, J.-D. Fekete and C. North, eds.) Information Visualization: Human-Centered Issues and Perspectives. Berlin/Heidelberg: Springer Berlin Heidelberg, pp. 19–45.

  • Carpenter, P. A. and P. Shah (1998) “A Model of the Perceptual and Conceptual Processes in Graph Comprehension,” Journal of Experimental Psychology: Applied, 4:75–100.

  • Cleveland, W. S. (1985) The Elements of Graphing Data. Monterey, CA: Wadsworth.

  • Cleveland, W. S. and R. McGill (1984) “Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods,” Journal of the American Statistical Association, 79:531–554.

  • Cleveland, W. S. and R. McGill (1985) “Graphical Perception and Graphical Methods for Analyzing Scientific Data,” Science, 229:828–833.

    • PubMed
    • Export Citation
  • Dinas, E., A. H. Trechsel and K. Vassil (2014) “A Look into the Mirror: Preferences, Representation and Electoral Participation,” Electoral Studies, 36:290–297.

  • Downs, A. (1957) An Economic Theory of Democracy. New York: Harper and Brothers.

  • Draper, G. M., Y. Livnat and R. F. Riesenfeld (2009) “A Survey of Radial Methods for Information Visualization,” IEEE Transactions on Visualization and Computer Graphics, 15:759–776.

    • PubMed
    • Export Citation
  • Enyedi, Z. (2016) “The Influence of Voting Advice Applications on Preferences, Loyalties and Turnout: An Experimental Study,” Political Studies, 64:1000–1015.

  • Feldman, R. (2013) “Filled Radar Charts Should not be Used to Compare Social Indicators,” Social Indicators Research, 111:709–712.

  • Freedman, E. G. and P. Shah (2002) “Toward a Model of Knowledge-Based Graph Comprehension,” In: (Hegarty, M., B. Meyer and N. H. Narayanan, eds.) Diagrammatic Representation and Inference. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 18–30.

  • Gabel, M. and S. Hix (2002) “Defining the EU Political Space: An Empirical Study of the European Elections Manifestos, 1979–1999,” Comparative Political Studies, 35:934–964.

  • Garcia-Retamero, R., E. T. Cokely, S. Ghazal and A. Joeris (2016) “Measuring Graph Literacy without a Test: A Brief Subjective Assessment,” Medical Decision Making, 36:854–867, pMID: 27353824.

  • Garry, J., J. Tilley, N. Matthews, F. Mendez and J. Wheatley (2019) “Does receiving advice from Voter Advice Applications (VAAs) affect public opinion in deeply divided societies? Evidence from a field experiment in Northern Ireland,” Party Politics, 25:854–861.

  • Garzia, D. and S. Marschall (2014) “The Lausanne Declaration on Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 227–228.

  • Garzia, D. and S. Marschall (2019) “Voting Advice Applications,” In: Oxford Research Encyclopedia of Politics. Retrieved 12 December. 2019, from https://oxfordre.com/politics/view/10.1093/acrefore/9780190228637.001.0001/acrefore-9780190228637-e-620.

  • Garzia, D., A. de Angelis and J. Pianzola (2014a) “The impact of Voting Advice Applications on Electoral Participation,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 105–114.

  • Garzia, D., A. H. Trechsel, K. Vassil and E. Dinas (2014b) “Indirect Campaigning: Past, Present and Future of Voting Advice Applications,” In: (Grofman, B., A. H. Trechsel and M. Franklin, eds.) The Internet and Democracy in Global Perspective. Cham: Springer International, pp. 25–41.

  • Garzia, D., A. H. Trechsel and A. D. Angelis (2017) “Voting Advice Applications and Electoral Participation: A Multi-Method Study,” Political Communication, 34:424–443.

  • Gemenis, K. (2018) “The Impact of Voting Advice Applications on Electoral Turnout: Evidence from Greece,” Statistics, Politics and Policy, 9:161–179.

  • Gemenis, K. and C. van Ham (2014) “Comparing Methods for Estimating Parties’ Positions in Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 33–48.

  • Germann, M. and F. Mendez (2016) “Dynamic Scale Validation Reloaded,” Quality & Quantity, 50:981–1007.

  • Germann, M. and K. Gemenis (2019) “Getting Out the Vote With Voting Advice Applications,” Political Communication, 36:149–170.

  • Germann, M., F. Mendez, J. Wheatley and U. Serdült (2014) “Spatial Maps in Voting Advice Applications: The Case for Dynamic Scale Validation,” Acta Politica, 50:1–25.

  • Glazer, N. (2011) “Challenges with Graph Interpretation: A Review of the Literature,” Studies in Science Education, 47:183–210.

  • Holleman, B., N. Kamoen, A. Krouwel, J. van de Pol and C. de Vreese (2016) “Positive vs. Negative: The Impact of Question Polarity in Voting Advice Applications,” PLoS One, 11:1–17.

  • Huff, D. (1954) How to Lie with Statistics. New York, NY: W.W. Norton & Company.

  • Iliinsky, N. and J. Steele (2011) Designing Data Visualizations. Sebastopol, CA: O’Reilly.

  • Katakis, I., N. Tsapatsoulis, F. Mendez, V. Triga and C. Djouvas (2014) “Social Voting Advice Applications – Definitions, Challenges, Datasets and Evaluation,” IEEE Transactions on Cybernetics, 44:1039–1052.

  • Kirk, A. (2016) Data Visualisation – A Handbook for Data Driven Design. London: Sage.

  • Kleinnijenhuis, J., J. van de Pol, A. van Hoof and A. Krouwel (2017) “VAAs as Sources of Volatility and Fragmentation: Self-selection effects and genuine effects,” Journal of Elections, Public Opinion and Parties, 27:75–96.

  • Kosslyn, S. M. (1989) “Understanding Charts and Graphs,” Applied Cognitive Psychology, 3:185–225.

  • Krouwel, A. and A. van Elfrinkhof (2014) “Combining Strengths of Methods of Party Positioning to counter their Weaknesses: The Development of a new Methodology to calibrate Parties on Issues and ideological Dimensions,” Quality & Quantity, 48:1455–1472.

  • Lam, H., E. Bertini, P. Isenberg, C. Plaisant and S. Carpendale (2012) “Empirical Studies in Information Visualization: Seven Scenarios,” IEEE Transactions on Visualization and Computer Graphics, 18:1520–1536.

    • PubMed
    • Export Citation
  • Lefevere, J. and S. Walgrave (2014) “A Perfect Match? The Impact of Statement Selection on Voting Advice Applications’ Ability to Match Voters and Parties,” Electoral Studies, 36:252–262.

  • Louwerse, T. and M. Rosema (2014) “The Design Effects of Voting Advice Applications: Comparing Methods of calculating Results,” Acta Politica, 49:286–312.

  • MacDonald, S. E., G. Rabinowitz and O. Listhaug (1995) “Political Sophistication and Models of Issue Voting,” British Journal of Political Science, 25:453–483.

  • Mahéo, V.-A. (2017) “Information Campaigns and (Under)Privileged Citizens: An Experiment on the Differential Effects of a Voting Advice Application,” Political Communication, 34:511–529.

  • Marschall, S. and C. K. Schmidt (2010) “The Impact of Voting Indicators: The Case of the German Wahl-O-Mat,” In (Cedroni, L. and D. Garzia, eds.) Voting Advice Applications in Europe: The State of the Art. Napoli: ScriptaWeb, pp. 65–90.

  • Mendez, F. (2012) “Matching Voters with Political Parties and Candidates: An Empirical Test of Four Algorithms,” International Journal of Electronic Governance, 5:264–278.

  • Mendez, F. (2014) “What’s Behind a Matching Algorithm? A Critical Assessment of how VAAs produce Voting Recommendations,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 49–66.

  • Munzner, T. (2015) Visualization Analysis & Design. Boca Raton, FI: CRC Press.

  • Rosema, M. and T. Louwerse (2016) “Response Scales in Voting Advice Applications: Do Different Designs Produce Different Outcomes?” Policy & Internet, 8:431–456.

  • Schultze, M. (2014) “Effects of Voting Advice Applications (VAAs) on political knowledge about party positions,” Policy & Internet, 6:46–68.

  • Shah, P. (2002) “Graph Comprehension: The Role of Format, Content and Individual Differences,” In: (Anderson, M., B. Meyer and P. Olivier, eds.) Diagrammatic Representation and Reasoning, London: Springer London, pp. 173–185.

    • Crossref
    • Export Citation
  • Shah, P. and J. Hoeffner (2002) “Review of Graph Comprehension Research: Implications for Instruction,” Educational Psychology Review, 14:47–69.

  • Shah, P., E. G. Freedman and I. Vekiri (2005) “The Comprehension of Quantitative Information in Graphical Displays,” In: (Shah, P. and A. Miyake, eds.) The Cambridge Handbook of Visuospatial Thinking. Cambridge Handbooks in Psychology, New York, NY: Cambridge University Press, pp. 426–476.

  • Shamim, A., V. Balakrishnan and M. Tahir (2015) “Evaluation of Opinion Visualization Techniques,” Information Visualization, 14:339–358.

  • Stoet, G. (2010) “PsyToolkit: A software package for programming psychological experiments using Linux,” Behavior Research Methods, 42:1096–1104.

    • PubMed
    • Export Citation
  • Stoet, G. (2017) “PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments,” Teaching of Psychology, 44:24–31.

  • Tufte, E. R. (2001) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, second edition.

  • Walgrave, S., M. Nuytemans and K. Pepermans (2009) “Voting Aid Applications and the Effect of Statement Selection,” West European Politics, 32:1161–1180.

Purchase article
Get instant unlimited access to the article.
Log in
Already have access? Please log in.

Log in with your institution

Journal + Issues

Statistics, Politics, and Policy studies the ways that statistical analysis drives public policy decisions, and publishes significant research on the application of statistical ideas to problems that relate to policy implementation.