Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Official Statistics

The Journal of Statistics Sweden

4 Issues per year


IMPACT FACTOR 2016: 0.411
5-year IMPACT FACTOR: 0.776

CiteScore 2016: 0.63

SCImago Journal Rank (SJR) 2016: 0.710
Source Normalized Impact per Paper (SNIP) 2016: 0.975

Open Access
Online
ISSN
2001-7367
See all formats and pricing
More options …

“Interviewer” Effects in Face-to-Face Surveys: A Function of Sampling, Measurement Error, or Nonresponse?

Brady T. West
  • Corresponding author
  • Survey Methodology Program (SMP), Survey Research Center (SRC), Institute for Social Research, University of Michigan, 426 Thompson Street, Ann Arbor, MI 48106, U.S.A.
  • Email:
/ Frauke Kreuter
  • Corresponding author
  • Joint Program in Survey Methodology (JPSM), Ludwig Maximilian University of Munich, Institute for Employment Research (IAB), Nuremberg, Germany
  • Email:
/ Ursula Jaenichen
  • Corresponding author
  • Institute for Employment Research (IAB), Nuremberg, Germany
  • Email:
Published Online: 2013-10-03 | DOI: https://doi.org/10.2478/jos-2013-0023

Abstract

Recent research has attempted to examine the proportion of interviewer variance that is due to interviewers systematically varying in their success in obtaining cooperation from respondents with varying characteristics (i.e., nonresponse error variance), rather than variance among interviewers in systematic measurement difficulties (i.e., measurement error variance) - that is, whether correlated responses within interviewers arise due to variance among interviewers in the pools of respondents recruited, or variance in interviewer-specific mean response biases. Unfortunately, work to date has only considered data from a CATI survey, and thus suffers from two limitations: Interviewer effects are commonly much smaller in CATI surveys, and, more importantly, sample units are often contacted by several CATI interviewers before a final outcome (response or final refusal) is achieved. The latter introduces difficulties in assigning nonrespondents to interviewers, and thus interviewer variance components are only estimable under strong assumptions. This study aims to replicate this initial work, analyzing data from a national CAPI survey in Germany where CAPI interviewers were responsible for working a fixed subset of cases.

Keywords: Interviewer variance; nonresponse error variance; measurement error variance; face-to-face data collection; multilevel modeling; PASS study

  • Bauer, T.K., Fertig, M., and Vorrell, M. (2011a). Neighborhood Effects and Individual Unemployment. SOEP paper 409, DIW Berlin.Google Scholar

  • Bauer, T.K., Flake, R., and Sinning, M.G. (2011b). Labor Market Effects of Immigration: Evidence from Neighborhood Data. Ruhr Economic Papers #257, Bochum, Dortmund, Duisburg, Essen.Google Scholar

  • Beland, Y. and St-Pierre, M. (2008). Mode Effects in the Canadian Community Health Survey: A Comparison of CATI and CAPI. Advances in Telephone Survey Methodology, Chapter 14, J.M. Lepkowski, C. Tucker, J.M. Brick, E.D. de Leeuw, L. Japec, P.J. Lavrakas, M.W. Link, R.L. Sangsler (eds). New York: Wiley.Google Scholar

  • Bender, S. and Heining, J. (2011). The Research-Data-Centre in Research-Data-Centre Approach: A First Step Towards Decentralised International Data Sharing. FDZ Methodenreport, 07/2011, Nürnberg.Google Scholar

  • Biemer, P.P. (1980). A Survey Error Model which Includes Edit and Imputation Error. Proceedings of the American Statistical Association, Section on Survey Research Methods, 616-621.Google Scholar

  • Biemer, P.P. (2010). Total Survey Error: Design, Implementation, and Evaluation. Public Opinion Quarterly, 74, 817-848.Web of ScienceGoogle Scholar

  • Biemer, P.P. and Stokes, S.L. (1991). Approaches to the Modeling of Measurement Error. Measurement Errors in Surveys, P.P. Biemer, R.M. Groves, L. Lyberg, N.A. Mathiowetz, and S. Sudman (eds). New York: Wiley. Biemer, P.P. and Trewin, D. (1997). A Review of Measurement Error Effects on the Analysis of Survey Data. In Survey Measurement and Process Quality, L. Lyberg, P.P.Google Scholar

  • Biemer, M. Collins, E.D. de Leeuw, C. Dippo, N. Schwarz, and D. Trewin (eds). New York: Wiley-Interscience.Google Scholar

  • Brunton-Smith, I., Sturgis, P., and Williams, J. (2012). Is Success in Obtaining Contact and Cooperation Correlated with the Magnitude of Interviewer Variance? Public Opinion Quarterly, 76, 265-286.Web of ScienceGoogle Scholar

  • Büngeler, K., Gensicke, M., Hartmann, J., Jäckle, R., and Tschersich, N. (2010). IABHaushaltspanel im Niedrigeinkommensbereich Welle 3 (2008/2009). Methoden- und Feldbericht. FDZ Methodenreport, 10/2010, Nu¨rnberg.Google Scholar

  • Campanelli, P. and O’Muircheartaigh, C. (1999). Interviewers, Interviewer Continuity, and Panel Survey Nonresponse. Journal of Official Statistics, 2, 303-314.Google Scholar

  • Christoph, B., Müller, G., Gebhardt, D., Wenzig, C., Trappmann, M., Achatz, J., Tisch, A., and Gayer, C. (2008). Codebook and Documentation of the Panel Study “Labour Market and Social Security” (PASS). Volume 1: Introduction and Overview, Wave 1 (2006/2007). FDZ Datenreport, 05/2008, Nürnberg.Google Scholar

  • Collins, M. and Butcher, B. (1982). Interviewer and Clustering Effects in an Attitude Survey. Journal of the Market Research Society, 25, 39-58.Google Scholar

  • Davis, P. and Scott, A. (1995). The Effect of Interviewer Variance on Domain Comparisons. Survey Methodology, 21, 99-106.Google Scholar

  • Durrant, G.B., Groves, R.M., Staetsky, L., and Steele, F. (2010). Effects of Interviewer Attitudes and Behaviors on Refusal in Household Surveys. Public Opinion Quarterly, 74, 1-36.Web of ScienceGoogle Scholar

  • Fellegi, I.P. (1964). Response Variance and its Estimation. Journal of the American Statistical Association, 59, 1016-1041.Web of ScienceGoogle Scholar

  • Freeman, J. and Butler, E.W. (1976). Some Sources of Interviewer Variance in Surveys. Public Opinion Quarterly, 40, 79-91.Google Scholar

  • Gebhardt, D., Müller, G., Bethmann, A., Trappmann, M., Christoph, B., Gayer, C., Müller, B., Tisch, A., Siflinger, B., Kiesl, H., Huyer-May, B., Achatz, J., Wenzig, C., Rudolph, H., Graf, T., and Biedermann, A. (2009). Codebook and Documentation of the Panel Study “Labour Market and Social Security” (PASS). Volume 1: Introduction and Overview, Wave 2 (2007/2008). FDZ Datenreport, 06/2009, Nürnberg.Google Scholar

  • Groves, R.M. (2004). The Interviewer as a Source of Survey Measurement Error. Survey Errors and Survey Costs, (Second Edition). New York: Wiley-Interscience.Google Scholar

  • Groves, R.M. and Lyberg, L. (2010). Total Survey Error: Past, Present, and Future. Public Opinion Quarterly, 74, 849-879.Web of ScienceGoogle Scholar

  • Groves, R.M. and Magilavy, L.J. (1984). An Experimental Measurement of Total Survey Error. Proceedings of the Joint Statistical Meetings of the American Statistical Association, Section on Survey Research Methods, 698-703.Google Scholar

  • Groves, R.M. and Magilavy, L.J. (1986). Measuring and Explaining Interviewer Effects in Centralized Telephone Surveys. Public Opinion Quarterly, 50, 251-266.Google Scholar

  • Groves, R.M. and Peytcheva, E. (2008). The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis. Public Opinion Quarterly, 72, 167-189.Web of ScienceGoogle Scholar

  • Hansen, M.H., Hurwitz, W.N., and Bershad, M.A. (1960). Measurement Errors in Censuses and Surveys. Bulletin of the International Statistical Institute, 32nd Session, Vol. 38, Part 2, 359-374.Google Scholar

  • Holbrook, A.L., Green, M.C., and Krosnick, J.A. (2003). Telephone Versus Face-to-Face Interviewing of National Probability Samples with Long Questionnaires: Comparisons of Respondent Satisficing and Social Desirability Bias. Public Opinion Quarterly, 67, 79-125.Google Scholar

  • Hox, J.J. (1998). Multilevel Modeling: When and Why. In Classification, Data Analysis, and Data Highways, I. Balderjahn, R. Mathar, and M. Schader (eds). New York: Springer.Google Scholar

  • Hox, J.J. and de Leeuw, E.D. (2002). The Influence of Interviewers’ Attitude and Behavior on Household Survey Nonresponse: An International Comparison. Survey Nonresponse, R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little (eds). New York: Wiley.Google Scholar

  • Kish, L. (1962). Studies of Interviewer Variance for Attitudinal Variables. Journal of the American Statistical Association, 57, 92-115.Google Scholar

  • Mangione, T.W., Fowler, F.J., and Louis, T.A. (1992). Question Characteristics and Interviewer Effects. Journal of Official Statistics, 8, 293-307.Google Scholar

  • Morton-Williams, J. (1993). Interviewer Approaches. Aldershot: Dartmouth Publishing Company Limited.Google Scholar

  • O’Muircheartaigh, C. and Campanelli, P. (1998). The Relative Impact of Interviewer Effects and Sample Design Effects on Survey Precision. Journal of the Royal Statistical Society, Series A, 161, 63-77. Google Scholar

  • O’Muircheartaigh, C. and Campanelli, P. (1999). A Multilevel Exploration of the Role of Interviewers in Survey Non-Response. Journal of the Royal Statistical Society, Series A, 162(Part 3), 437-446.Google Scholar

  • Patterson, H.D. and Thompson, R. (1971). Recovery of Inter-Block Information when Block Sizes are Unequal. Biometrika, 58, 545-554.Google Scholar

  • Pickery, J. and Loosveldt, G. (2002). A Multilevel Multinomial Analysis of Interviewer Effects on Various Components of Unit Nonresponse. Quality and Quantity, 36, 427-437.Google Scholar

  • Rudolph, H. and Trappmann, M. (2007). Design und Stichprobe des Panels “Arbeitsmarkt und Soziale Sicherung” (PASS). In Neue Daten für die Sozialstaatsforschung. Zur Konzeption der IAB-Panelerhebung “Arbeitsmarkt und Soziale Sicherung”, (ed.) M. Promberger, IAB Forschungsbericht, 12/2007, Nürnberg.Google Scholar

  • Schaeffer, N.C., Dykema, J., and Maynard, D.W. (2010). Interviewers and Interviewing. In Handbook of Survey Research, J.D. Wright and P.V. Marsden (eds). (Second Edition). Bingley, U.K. Emerald Group Publishing Limited.Google Scholar

  • Schnell, R. (1997). Nonresponse in Bevölkerungsumfragen: Ausmaß, Entwicklung und Ursachen. Opladen: Leske þ Budrich.Google Scholar

  • Schnell, R. (2012). Survey-Interviews. Standardisierte Befragungen in den Sozialwissenschaften. Wiesbaden: VS-Verlag.Google Scholar

  • Schnell, R. and Kreuter, F. (2005). Separating Interviewer and Sampling-Point Effects. Journal of Official Statistics, 21, 389-410.Google Scholar

  • Snijkers, G., Hox, J.J., and de Leeuw, E.D. (1999). Interviewers’ Tactics for Fighting Survey Nonresponse. Journal of Official Statistics, 15, 185-198.Google Scholar

  • Trappmann, M., Gundert, S., Wenzig, C., and Gebhardt, D. (2010). PASS: a Household Panel Survey for Research on Unemployment and Poverty. Proceedings of the American Statistical Association, Section on Survey Research Methods, 130, 609-622.Google Scholar

  • West, B.T. and Olson, K. (2010). How Much of Interviewer Variance is Really Nonresponse Error Variance? Public Opinion Quarterly, 74, 1004-1026.Web of ScienceGoogle Scholar

  • Wiggins, R.D., Longford, N.T., and O’Muircheartaigh, C.A. (1992). A Variance Components Approach to Interviewer Effects. In Survey and Statistical Computing, A. Westlake, R. Banks, C. Payne, and T. Orchard (eds). Amsterdam: North-Holland.Google Scholar

  • Zhang, D. and Lin, X. (2008). Variance Component Testing in Generalized Linear Mixed Models for Longitudinal/Clustered Data and Other Related Topics. Random Effect and Latent Variable Model Selection, D.B. Dunson (ed.)., Springer Lecture Notes in Statistics, 192. Google Scholar

About the article

Published Online: 2013-10-03

Published in Print: 2013-09-01


Citation Information: Journal of Official Statistics, ISSN (Online) 2001-7367, DOI: https://doi.org/10.2478/jos-2013-0023.

Export Citation

This content is open access.

Comments (0)

Please log in or register to comment.
Log in