Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter May 20, 2020

Evaluating Visualisations in Voting Advice Applications

  • Bastiaan Bruinsma ORCID logo EMAIL logo


While the design of voting advice applications (VAAs) is witnessing an increasing amount of attention, one aspect has until now been overlooked: its visualisations. This is remarkable, as it are those visualisations that communicate to the user the advice of the VAA. Therefore, this article aims to provide a first look at which visualisations VAAs adopt, why they adopt them, and how users comprehend them. For this, I will look at how design choices, specifically those on matching, influence the type of visualisation VAAs not only do but also have to, use. Second, I will report the results of a small-scale experiment that looked if all users comprehend similar visualisations in the same way. Here, I find that this is often not the case and that the interpretations of the users often differ. These first results suggest that VAA visualisations are wrongly underappreciated and demand closer attention of VAA designers.


Parts of this work were first presented at the ECPR General Conference, University of Hamburg (August 22–25, 2018). I would like to thank the reviewers there, as well as the anonymous reviewers of Statistics, Politics and Policy for their helpful comments. Any additional errors remain my responsibility.


Overview of Voting Advice Applications:

Name Country Years
Educatieve Stemtest Belgium 2016
Euandi European Union 2014, 2019
iVoter Taiwan 2012, 2016
Kiesbalans The Netherlands 2017
Kieskompas The Netherlands 2006, 2010, 2012, 2017
Mainichi Japan 2009, 2012
ParteieNavi Germany 2013
Smartvote Switzerland 2011, 2015
StemWijzer The Netherlands 2006, 2010, 2012, 2017
Vaalikone Finland 2004, 2006, 2009, 2012
Vote Compass United States 2012, 2016
Wahl-O-Mat Germany 2005, 2009, 2013
Who Shall I Vote For? United Kingdom 2015
Who Should You Vote For? United Kingdom 2015


Albo, Y., J. Lanir, P. Bak and S. Rafaeli (2016) “Off the Radar: Comparative Evaluation of Radial Visualization Solutions for Composite Indicators,” IEEE Transactions on Visualization and Computer Graphics, 22:569–578.10.1109/TVCG.2015.2467322Search in Google Scholar

Alvarez, R. M., I. Levin, A. H. Trechsel and K. Vassil (2014) “Voting Advice Applications: How Useful and for Whom?” Journal of Information Technology & Politics, 11:82–101.10.1080/19331681.2013.873361Search in Google Scholar

Baka, A., L. Figgou and V. Triga (2012) “‘Neither Agree, Nor Disagree’: A Critical Analysis of the Middle Answer Category in Voting Advice Applications,” International Journal of Electronic Governance, 5:244–263.10.1504/IJEG.2012.051306Search in Google Scholar

Benoit, K. and M. Laver (2006) Party Policy in Modern Democracies. London: Routledge.10.4324/9780203028179Search in Google Scholar

Benoit, K. and M. Laver (2012) “The Dimensionality of Political Space: Epistemological and methodological considerations,” European Union Politics, 13:194–218.10.1177/1465116511434618Search in Google Scholar

Carpendale, S. (2008) “Evaluating Information Visualizations,” In: (Kerren, A., J. T. Stasko, J.-D. Fekete and C. North, eds.) Information Visualization: Human-Centered Issues and Perspectives. Berlin/Heidelberg: Springer Berlin Heidelberg, pp. 19–45.10.1007/978-3-540-70956-5_2Search in Google Scholar

Carpenter, P. A. and P. Shah (1998) “A Model of the Perceptual and Conceptual Processes in Graph Comprehension,” Journal of Experimental Psychology: Applied, 4:75–100.10.1037/1076-898X.4.2.75Search in Google Scholar

Cleveland, W. S. (1985) The Elements of Graphing Data. Monterey, CA: Wadsworth.Search in Google Scholar

Cleveland, W. S. and R. McGill (1984) “Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods,” Journal of the American Statistical Association, 79:531–554.10.1080/01621459.1984.10478080Search in Google Scholar

Cleveland, W. S. and R. McGill (1985) “Graphical Perception and Graphical Methods for Analyzing Scientific Data,” Science, 229:828–833.10.1126/science.229.4716.828Search in Google Scholar

Dinas, E., A. H. Trechsel and K. Vassil (2014) “A Look into the Mirror: Preferences, Representation and Electoral Participation,” Electoral Studies, 36:290–297.10.1016/j.electstud.2014.04.011Search in Google Scholar

Downs, A. (1957) An Economic Theory of Democracy. New York: Harper and Brothers.Search in Google Scholar

Draper, G. M., Y. Livnat and R. F. Riesenfeld (2009) “A Survey of Radial Methods for Information Visualization,” IEEE Transactions on Visualization and Computer Graphics, 15:759–776.10.1109/TVCG.2009.23Search in Google Scholar

Enyedi, Z. (2016) “The Influence of Voting Advice Applications on Preferences, Loyalties and Turnout: An Experimental Study,” Political Studies, 64:1000–1015.10.1111/1467-9248.12213Search in Google Scholar

Feldman, R. (2013) “Filled Radar Charts Should not be Used to Compare Social Indicators,” Social Indicators Research, 111:709–712.10.1007/s11205-012-0028-6Search in Google Scholar

Freedman, E. G. and P. Shah (2002) “Toward a Model of Knowledge-Based Graph Comprehension,” In: (Hegarty, M., B. Meyer and N. H. Narayanan, eds.) Diagrammatic Representation and Inference. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 18–30.10.1007/3-540-46037-3_3Search in Google Scholar

Gabel, M. and S. Hix (2002) “Defining the EU Political Space: An Empirical Study of the European Elections Manifestos, 1979–1999,” Comparative Political Studies, 35:934–964.10.1177/001041402236309Search in Google Scholar

Garcia-Retamero, R., E. T. Cokely, S. Ghazal and A. Joeris (2016) “Measuring Graph Literacy without a Test: A Brief Subjective Assessment,” Medical Decision Making, 36:854–867, pMID: 27353824.10.1177/0272989X16655334Search in Google Scholar

Garry, J., J. Tilley, N. Matthews, F. Mendez and J. Wheatley (2019) “Does receiving advice from Voter Advice Applications (VAAs) affect public opinion in deeply divided societies? Evidence from a field experiment in Northern Ireland,” Party Politics, 25:854–861.10.1177/1354068818818789Search in Google Scholar

Garzia, D. and S. Marschall (2014) “The Lausanne Declaration on Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 227–228.Search in Google Scholar

Garzia, D. and S. Marschall (2019) “Voting Advice Applications,” In: Oxford Research Encyclopedia of Politics. Retrieved 12 December. 2019, from in Google Scholar

Garzia, D., A. de Angelis and J. Pianzola (2014a) “The impact of Voting Advice Applications on Electoral Participation,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 105–114.Search in Google Scholar

Garzia, D., A. H. Trechsel, K. Vassil and E. Dinas (2014b) “Indirect Campaigning: Past, Present and Future of Voting Advice Applications,” In: (Grofman, B., A. H. Trechsel and M. Franklin, eds.) The Internet and Democracy in Global Perspective. Cham: Springer International, pp. 25–41.10.1007/978-3-319-04352-4_3Search in Google Scholar

Garzia, D., A. H. Trechsel and A. D. Angelis (2017) “Voting Advice Applications and Electoral Participation: A Multi-Method Study,” Political Communication, 34:424–443.10.1080/10584609.2016.1267053Search in Google Scholar

Gemenis, K. (2018) “The Impact of Voting Advice Applications on Electoral Turnout: Evidence from Greece,” Statistics, Politics and Policy, 9:161–179.10.1515/spp-2018-0011Search in Google Scholar

Gemenis, K. and C. van Ham (2014) “Comparing Methods for Estimating Parties’ Positions in Voting Advice Applications,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 33–48.Search in Google Scholar

Germann, M. and F. Mendez (2016) “Dynamic Scale Validation Reloaded,” Quality & Quantity, 50:981–1007.10.1007/s11135-015-0186-0Search in Google Scholar

Germann, M. and K. Gemenis (2019) “Getting Out the Vote With Voting Advice Applications,” Political Communication, 36:149–170.10.1080/10584609.2018.1526237Search in Google Scholar

Germann, M., F. Mendez, J. Wheatley and U. Serdült (2014) “Spatial Maps in Voting Advice Applications: The Case for Dynamic Scale Validation,” Acta Politica, 50:1–25.10.1057/ap.2014.3Search in Google Scholar

Glazer, N. (2011) “Challenges with Graph Interpretation: A Review of the Literature,” Studies in Science Education, 47:183–210.10.1080/03057267.2011.605307Search in Google Scholar

Holleman, B., N. Kamoen, A. Krouwel, J. van de Pol and C. de Vreese (2016) “Positive vs. Negative: The Impact of Question Polarity in Voting Advice Applications,” PLoS One, 11:1–17.10.1371/journal.pone.0164184Search in Google Scholar

Huff, D. (1954) How to Lie with Statistics. New York, NY: W.W. Norton & Company.Search in Google Scholar

Iliinsky, N. and J. Steele (2011) Designing Data Visualizations. Sebastopol, CA: O’Reilly.Search in Google Scholar

Katakis, I., N. Tsapatsoulis, F. Mendez, V. Triga and C. Djouvas (2014) “Social Voting Advice Applications – Definitions, Challenges, Datasets and Evaluation,” IEEE Transactions on Cybernetics, 44:1039–1052.10.1109/TCYB.2013.2279019Search in Google Scholar

Kirk, A. (2016) Data Visualisation – A Handbook for Data Driven Design. London: Sage.Search in Google Scholar

Kleinnijenhuis, J., J. van de Pol, A. van Hoof and A. Krouwel (2017) “VAAs as Sources of Volatility and Fragmentation: Self-selection effects and genuine effects,” Journal of Elections, Public Opinion and Parties, 27:75–96.10.1080/17457289.2016.1268143Search in Google Scholar

Kosslyn, S. M. (1989) “Understanding Charts and Graphs,” Applied Cognitive Psychology, 3:185–225.10.1002/acp.2350030302Search in Google Scholar

Krouwel, A. and A. van Elfrinkhof (2014) “Combining Strengths of Methods of Party Positioning to counter their Weaknesses: The Development of a new Methodology to calibrate Parties on Issues and ideological Dimensions,” Quality & Quantity, 48:1455–1472.10.1007/s11135-013-9846-0Search in Google Scholar

Lam, H., E. Bertini, P. Isenberg, C. Plaisant and S. Carpendale (2012) “Empirical Studies in Information Visualization: Seven Scenarios,” IEEE Transactions on Visualization and Computer Graphics, 18:1520–1536.10.1109/TVCG.2011.279Search in Google Scholar

Lefevere, J. and S. Walgrave (2014) “A Perfect Match? The Impact of Statement Selection on Voting Advice Applications’ Ability to Match Voters and Parties,” Electoral Studies, 36:252–262.10.1016/j.electstud.2014.04.002Search in Google Scholar

Louwerse, T. and M. Rosema (2014) “The Design Effects of Voting Advice Applications: Comparing Methods of calculating Results,” Acta Politica, 49:286–312.10.1057/ap.2013.30Search in Google Scholar

MacDonald, S. E., G. Rabinowitz and O. Listhaug (1995) “Political Sophistication and Models of Issue Voting,” British Journal of Political Science, 25:453–483.10.1017/S0007123400007316Search in Google Scholar

Mahéo, V.-A. (2017) “Information Campaigns and (Under)Privileged Citizens: An Experiment on the Differential Effects of a Voting Advice Application,” Political Communication, 34:511–529.10.1080/10584609.2017.1282560Search in Google Scholar

Marschall, S. and C. K. Schmidt (2010) “The Impact of Voting Indicators: The Case of the German Wahl-O-Mat,” In (Cedroni, L. and D. Garzia, eds.) Voting Advice Applications in Europe: The State of the Art. Napoli: ScriptaWeb, pp. 65–90.Search in Google Scholar

Mendez, F. (2012) “Matching Voters with Political Parties and Candidates: An Empirical Test of Four Algorithms,” International Journal of Electronic Governance, 5:264–278.10.1504/IJEG.2012.051316Search in Google Scholar

Mendez, F. (2014) “What’s Behind a Matching Algorithm? A Critical Assessment of how VAAs produce Voting Recommendations,” In: (Garzia, D. and S. Marschall, eds.) Matching Voters with Parties and Candidates: Voting Advice Applications in a Comparative Perspective. Colchester, UK: ECPR Press, pp. 49–66.Search in Google Scholar

Munzner, T. (2015) Visualization Analysis & Design. Boca Raton, FI: CRC Press.10.1201/b17511Search in Google Scholar

Rosema, M. and T. Louwerse (2016) “Response Scales in Voting Advice Applications: Do Different Designs Produce Different Outcomes?” Policy & Internet, 8:431–456.10.1002/poi3.139Search in Google Scholar

Schultze, M. (2014) “Effects of Voting Advice Applications (VAAs) on political knowledge about party positions,” Policy & Internet, 6:46–68.10.1002/1944-2866.POI352Search in Google Scholar

Shah, P. (2002) “Graph Comprehension: The Role of Format, Content and Individual Differences,” In: (Anderson, M., B. Meyer and P. Olivier, eds.) Diagrammatic Representation and Reasoning, London: Springer London, pp. 173–185.10.1007/978-1-4471-0109-3_10Search in Google Scholar

Shah, P. and J. Hoeffner (2002) “Review of Graph Comprehension Research: Implications for Instruction,” Educational Psychology Review, 14:47–69.10.1023/A:1013180410169Search in Google Scholar

Shah, P., E. G. Freedman and I. Vekiri (2005) “The Comprehension of Quantitative Information in Graphical Displays,” In: (Shah, P. and A. Miyake, eds.) The Cambridge Handbook of Visuospatial Thinking. Cambridge Handbooks in Psychology, New York, NY: Cambridge University Press, pp. 426–476.10.1017/CBO9780511610448.012Search in Google Scholar

Shamim, A., V. Balakrishnan and M. Tahir (2015) “Evaluation of Opinion Visualization Techniques,” Information Visualization, 14:339–358.10.1177/1473871614550537Search in Google Scholar

Stoet, G. (2010) “PsyToolkit: A software package for programming psychological experiments using Linux,” Behavior Research Methods, 42:1096–1104.10.3758/BRM.42.4.1096Search in Google Scholar

Stoet, G. (2017) “PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments,” Teaching of Psychology, 44:24–31.10.1177/0098628316677643Search in Google Scholar

Tufte, E. R. (2001) The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press, second edition.Search in Google Scholar

Walgrave, S., M. Nuytemans and K. Pepermans (2009) “Voting Aid Applications and the Effect of Statement Selection,” West European Politics, 32:1161–1180.10.1080/01402380903230637Search in Google Scholar

Published Online: 2020-05-20
Published in Print: 2020-06-25

©2020 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 3.12.2023 from
Scroll to top button