Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Official Statistics

The Journal of Statistics Sweden

4 Issues per year

IMPACT FACTOR 2016: 0.411
5-year IMPACT FACTOR: 0.776

CiteScore 2016: 0.63

SCImago Journal Rank (SJR) 2016: 0.710
Source Normalized Impact per Paper (SNIP) 2016: 0.975

Open Access
See all formats and pricing
More options …

Web-Face-to-Face Mixed-Mode Design in a Longitudinal Survey: Effects on Participation Rates, Sample Composition, and Costs

Annamaria Bianchi
  • Department of Management, Economics and Quantitative Methods, University of Bergamo, via dei Caniana 2, 24127 Bergamo, Italy.
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Silvia Biffignandi
  • Department of Management, Economics and Quantitative Methods, University of Bergamo, via dei Caniana 2, 24127 Bergamo, Italy.
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Peter Lynn
  • Institute for Social and Economic Research, University of Essex, Wivenhoe Park, Colchester, Essex C04 3SQ, United Kingdom of Great Britain and Northern Ireland.
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2017-06-12 | DOI: https://doi.org/10.1515/jos-2017-0019


Sequential mixed-mode designs are increasingly considered as an alternative to interviewer-administered data collection, allowing researchers to take advantage of the benefits of each mode. We assess the effects of the introduction of a sequential web-face-to-face mixed-mode design over three waves of a longitudinal survey in which members were previously interviewed face-to-face. Findings are reported from a large-scale randomised experiment carried out on the UK Household Longitudinal Study. No differences are found between the mixed-mode design and face-to-face design in terms of cumulative response rates and only minimal differences in terms of sample composition. On the other hand, potential cost savings are evident.

Keywords: Attrition; total survey error; nonresponse bias; randomised experiment

8. References

  • Bianchi, A. and S. Biffignandi. 2014. “Responsive Design for Economic Data in Mixed-Mode Panels.” In Contribution to Sampling Statistics, edited by F. Mecatti, P.L. Conti, and M.G. Ranalli, 85–102. Springer International Publishing.Google Scholar

  • Bianchi, A. and S. Biffignandi. 2017. “Representativeness in Panel Surveys.” To appear in Mathematical Population Studies.Google Scholar

  • Bianchi, A. and S. Biffignandi. Forthcoming. “Survey Experiments on Interactions: a Case Study of Incentives and Modes.” In Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment, edited by P.J. Lavrakas, E. de Leeuw, A. Holbrook, C. Kennedy, M.W. Traugott, and B.T. West. Hoboken NJ: John Wiley & Sons.Google Scholar

  • Biemer, P.P. 2010. “Total Survey Error: Design, Implementation and Evaluation.” Public Opinion Quarterly 74(5): 817–848. Doi: http://dx.doi.org/10.1093/poq/nfq058.Crossref

  • Biemer, P.P. and L.E. Lyberg. 2003. Introduction to Survey Quality. New York: John Wiley.Google Scholar

  • Bowling, A. 2005. “Mode of Questionnaire Administration Can Have Serious Effects on Data Quality.” Journal of Public Health 27: 281–291.CrossrefGoogle Scholar

  • Buck, N. and S. McFall. 2012. “Understanding Society: Design Overview.” Longitudinal and Life Course Studies 3: 5–17.Google Scholar

  • Calinescu, M. and B. Schouten. 2015. “Adaptive Survey Designs to Minimize Survey Mode Effects – a Case Study on the Dutch Labor Force Survey.” Survey Methodology 41(2) : 403–425.Google Scholar

  • Chng, S., M. White, C. Abraham, and S. Skippon. 2016. “Commuting and Wellbeing in London: the Roles of Commute Mode and Local Public Transport Connectivity.” Preventive Medicine 88: 182–188. Doi: http://dx.doi.org/10.1016/j.ypmed.2016.04.014.Crossref

  • Couper, M. 2011. “The Future of Modes of Data Collection.” Public Opinion Quarterly 75: 889–908. Doi: http://dx.doi.org/10.1093/poq/nfr046.Crossref

  • Evandrou, M., J. Falkingham, Z. Feng, and A. Vlachantoni. 2016. “Ethnic Inequalities in Limiting Health and Self-Reported Health in Later Life Revisited.” Journal of Epidemiology & Community Health 70: 653–662. Doi: http://dx.doi.org/10.1136/jech-2015-206074.Crossref

  • Fan, W. and Z. Yan. 2010. “Factors Affecting Response Rates of the Web Surveys: a Systematic Review.” Computers in Human Behavior 26: 132–139.CrossrefGoogle Scholar

  • Farrant, G. and C. O’Muircheartaigh. 1991. “Components of Nonresponse Bias in the British Election Surveys.” In Understanding Political Change, edited by A. Heath, J. Curtice, R. Jowell, S. Evans, J. Field, and S. Witherspoon, 235–249. London: Pergamon Press.Google Scholar

  • Fong, B. and J. Williams. 2011. “British Crime Survey: Feasibility of Boosting Police Force Area (PFA) Sample Sizes Using Supplementary Recontact Surveys.” Report for the Home Office, TNS-BMRB, London.Google Scholar

  • Gaia, A. 2014. “Does a Mixed-Mode Design Increase Panel Attrition? Evidence from the UKHLS Innovation Panel.” Paper presented at the Internet Survey Methodology Workshop, Bolzano, December 1–3.Google Scholar

  • Göritz, A. 2006. “Incentives in Web Studies: Methodological Issues and a Review.” International Journal of Internet Science 1: 58–70.Google Scholar

  • Göritz, A. 2010. “Using Lotteries, Loyalty Points, and Other Incentives to Increase Participant Response and Completion.” In Advanced methods for conducting online behavioural research, edited by S. Gosling and J. Johnson, 219–233. Washington DC: American Psychological Association. Doi: http://dx.doi.org/10.1037/12076-014.Crossref

  • Göritz, A. 2015. “Incentive Effects.” In Improving Survey Methods: Lessons from Recent Research, edited by U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, and P. Sturgis, 339–350. London: Routledge.Google Scholar

  • Griffin, D., D. Fischer, and M. Morgan. 2001. “Testing an Internet Response Option for the American Community Survey.” Paper presented at the annual conference of the American Association for Public Opinion Research Montreal.” Quebec, Canada, May 17–20.Google Scholar

  • Groves, R. and L. Lyberg. 2010. “Total Survey Error: Past, Present, and Future.” Public Opinion Quarterly 74(5): 849–879. Doi: http://dx.doi.org/10.1093/poq/nfq065.Crossref

  • Groves, R.M. and F. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72: 167–189. Doi: https://doi.org/10.1093/poq/nfn011.Crossref

  • Jäckle, A. 2016. “Mode Effects on Measurement in Understanding Society.” Paper presented at the International Panel Survey Methods Workshop, Berlin, June 20–21. Available at: http://www.diw.de/en/diw_01.c.534396.en/program_psmw2016.html (accessed March 2017).

  • Jäckle, A. and P. Lynn. 2008. “Respondent Incentives in a Multi-Mode Panel Survey: Cumulative Effects on Nonresponse and Bias.” Survey Methodology 34: 105–117.Google Scholar

  • Jäckle, A., P. Lynn, and J. Burton. 2015. “Going Online with a Face-to-Face Household Panel: Effects of a Mixed Mode Design on Item and Unit Non-Response.” Survey Research Methods 9(1): 57–70. Doi: http://dx.doi.org/10.18148/srm/2015.v9i1.5475.Crossref

  • Janssen, B. 2006. “Web Data Collection in a Mixed Mode Approach: An Experiment.” Paper presented at the European Conference on Quality in Official Statistics (Q2006), Cardiff, April 24–26. Available at: webarchive.nationalarchives.gov.uk/20140721132900/http://ons.gov.uk/about/newsroom/events/q2006––european-conference-on-quality-in-survey-statistics-24-26-april-2006/agenda/session-19-wednesday.pdf (accessed March 2017).Google Scholar

  • Klausch, T., J. Hox, and B. Schouten. 2015a. “Selection Error in Single- and Mixed Mode Surveys of the Dutch General Population.” Journal of the Royal Statistical Society, Series A 178(4): 945–961. Doi: http://dx.doi.org/10.1111/rssa.12102.Crossref

  • Klausch, T., B. Schouten, and J.J. Hox. 2015b. “Evaluating Bias of Sequential Mixedmode Designs Against Benchmark Surveys.” Sociological Methods & Research : 1–34. Doi: http://dx.doi.org/10.1177/0049124115585362.Crossref

  • Kreuter, F. 2013. “Facing the Nonresponse Challenge.” The Annals of the American Academy of Political and Social Science 645: 23–35. Doi: https://doi.org/10.1177/0002716212456815.Crossref

  • Krosnick, J.A. and D.F. Alwin. 1987. “An Evaluation of a Cognitive Theory of Response-Order Effects in Survey Measurement.” Public Opinion Quarterly 51: 201–219. Doi: https://doi.org/10.1086/269029.Crossref

  • Lagerstrøm, B. 2008. “Cost Efficiency in a Mixed-Mode Survey – Evidence from the Norwegian Rent Market Survey.” Paper presented at the 19th International Workshop on Household Survey Nonresponse, Ljubljana, September 15. Available at: http://www.nonresponse.org/db/3/558/Biblliography/Cost%20efficiency%20in%20a%20mixed-mode%20survey%20Evidence%20from%20the%20Norwegian%20Rent%20Marked%20Survey/?&p1=308&p2=74&p3=551 (accessed March 2017).

  • Laurie, H. and P. Lynn. 2009. “The Use of Respondent Incentives on Longitudinal Surveys.” In Methodology of Longitudinal Surveys, edited by P. Lynn, 205–233. Chichester: Wiley.Google Scholar

  • Leesti, T. 2010. “Canadian Labour Force Survey Internet Data Collection Pilot Test.” Paper presented at the Fifth Workshop on Labour Force Survey Methodology, Paris, April 15–16.Google Scholar

  • De Leeuw, E.D. 2005. “To Mix or not to Mix Data Collection in Surveys.” Journal of Official Statistics 21: 233–255.Google Scholar

  • Lugtig, P. 2014. “Panel Attrition: Separating Stayers, Fast Attriters, Gradual Attriters, and Lurkers.” Sociological Methods and Research 43(4): 699–723. Doi: http://dx.doi.org/10.1177/0049124113520305.Crossref

  • Lynn, P. 2009. “Sample Design for Understanding Society.” Understanding Society Working Paper 2009-01, ISER, University of Essex, Colchester. Available at: www.understandingsociety.ac.uk/research/publications/working-paper/understandingsociety/2009-01 (accessed 16 March 2017).

  • Lynn, P. 2013. “Alternative Sequential Mixed-Mode Designs: Effects on Attrition Rates, Attrition Bias and Costs.” Journal of Survey Statistics and Methodology 1: 183–205. Doi: http://dx.doi.org/10.1093/jssam/smt015.Crossref

  • Lynn, P. 2014. “Targeted Response Inducement Strategies on Longitudinal Surveys.” In Improving Survey Methods: Lessons from Recent Research, edited by U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, and P. Sturgis. Routledge/Psychology Press.Google Scholar

  • Lynn, P. Forthcoming. “Tackling Panel Attrition.” In The Palgrave Handbook of Survey Research, edited by D.L. Vannette and J.A. Krosnick. Palgrave.Google Scholar

  • Lynn, P. and A. Jäckle. Forthcoming. “Mounting Multiple Experiments on Longitudinal Social Surveys: Design and Implementation Considerations.” In Experimental Methods in Survey Research: Techniques that Combine Random Sampling with Random Assignment, edited by P.J. Lavrakas, E.E. de Leeuw, A. Holbrook, C. Kennedy, M.W. Traugott, and B.T. West. Hoboken NJ: Wiley.Google Scholar

  • Lynn, P. and P. Lugtig. 2017. “Total Survey Error for Longitudinal Surveys.” In Total Survey Error in Practice, edited by Paul Biemer, Edith de Leeuw, Stephanie Eckman, Brad Edwards, Frauke Kreuter, Lars Lyberg, Clyde Tucker, and Brady West, 279–298. Hoboken, New Jersey: Wiley.Google Scholar

  • Lynn, P. and D. Lievesley. 1991. “Drawing General Population Samples in Great Britain.” London: SCPR.Google Scholar

  • Lynn, P. S.C.N. Uhrig, and J. Burton. 2010. “Lessons from a Randomized Experiment with Mixed-Mode Designs for a Household Panel Survey.” Understanding Society, Working Paper Series, 2010-03.Google Scholar

  • Martin, P. and P. Lynn. 2011. “The Effects of Mixed Mode Survey Designs on Simple and Complex Analyses.” ISER Working Paper Series, 2011-28. Colchester: Institute for Social and Economic Research, University of Essex. Available at: https://www.iser.essex.ac.uk/research/publications/working-papers/iser/2011-28 (accessed February 5, 2013).

  • Schoeni, R., F. Stafford, K. McGonagle, and P. Andreski. 2013. “Response Rates in National Panel Surveys.” The Annals of the American Academy of Political and Social Science 645: 60–87. Doi: https://doi.org/10.1177/0002716212456363.Crossref

  • Schwarz, N., F. Strack, H.-J. Hippler, and G. Bishop. 1991. “The Impact of Administration Mode on Response Effects in Survey Measurement.” Applied Cognitive Psychology 5: 193–212.CrossrefGoogle Scholar

  • Souren, M. 2012. “Multi-Mode Surveys at Statistics Netherlands: Implications, Experiences and Open Issues.” Paper presented at Opening Conference of the European Statistical System Network (ESSNet) on Data Collection for Social Surveys using Multiple Modes, Wiesbaden, October 11–12, 2012.Google Scholar

  • Uhrig, S.C.N. 2008. “The Nature and Causes of Attrition in the British Household Panel Study.” Institute for Social and Economic Research Working Paper 2008-05. Available at: https://www.iser.essex.ac.uk/research/publications/working-papers/iser/2008-05 (accessed 16 March 2017).

  • Uhrig, S.C.N. 2011. “Using Experiments to Guide Decision Making in Understanding Society: Introducing the Innovation Panel.” In Understanding Society: Early Findings from the First Wave of the UK’s Household Longitudinal Study, edited by S.L. McFall and C. Garrington. Colchester: University of Essex. Available at: http://research.understandingsociety.org.uk/findings/early-findings (accessed 16 March 2017).

  • Voogt, R. and W. Saris. 2005. “Mixed Mode Designs: Finding the Balance between Nonresponse Bias and Mode Effects.” Journal of Official Statistics 21(3): 367–387.Google Scholar

  • Voorpostel, M. and V.A. Ryser. 2011. “Mixed Mode Data Collection as a Strategy to Decrease Panel Attrition in the Swiss Household Panel.” FORS Working Paper 2_11. Available at: http://ohs-shp.unil.ch/workingpapers/WP2_11.pdf (accessed 28 March 2017).

  • Wallace, S., J. Nazroo, and L. Bécares. 2016. “Cumulative Effect of Racial Discrimination on the Mental Health of Ethnic Minorities in the United Kingdom.” American Journal of Public Health 106(7): 1294–1300. Doi: http://dx.doi.org/10.2105/AJPH.2016.303121.Crossref

  • Watson, N. and M. Wooden. 2014. “Re-Engaging with Survey Non-Respondents: Evidence from Three Household Panels.” Journal of the Royal Statistical Society Series A (Statistics in Society) 177(2): 499–522. Doi: http://dx.doi.org/10.1111/rssa.12024.Crossref

About the article

Received: 2016-02-01

Revised: 2016-12-01

Accepted: 2017-01-01

Published Online: 2017-06-12

Published in Print: 2017-06-01

Citation Information: Journal of Official Statistics, Volume 33, Issue 2, Pages 385–408, ISSN (Online) 2001-7367, DOI: https://doi.org/10.1515/jos-2017-0019.

Export Citation

© 2017 Annamaria Bianchi et al., published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Annamaria Bianchi and Silvia Biffignandi
Social Indicators Research, 2018

Comments (0)

Please log in or register to comment.
Log in