Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Official Statistics

The Journal of Statistics Sweden

4 Issues per year

IMPACT FACTOR 2016: 0.411
5-year IMPACT FACTOR: 0.776

CiteScore 2016: 0.63

SCImago Journal Rank (SJR) 2016: 0.710
Source Normalized Impact per Paper (SNIP) 2016: 0.975

Open Access
See all formats and pricing
More options …

Interviewer Effects on a Network-Size Filter Question

Michael Josten / Mark Trappmann
  • Institute for Employment Research, Regensburger Str. 104, 90478 Nuremberg, Germany and University of Bamberg, Germany.
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2016-05-28 | DOI: https://doi.org/10.1515/jos-2016-0020


There is evidence that survey interviewers may be tempted to manipulate answers to filter questions in a way that minimizes the number of follow-up questions. This becomes relevant when ego-centered network data are collected. The reported network size has a huge impact on interview duration if multiple questions on each alter are triggered. We analyze interviewer effects on a network-size question in the mixed-mode survey “Panel Study ‘Labour Market and Social Security’” (PASS), where interviewers could skip up to 15 follow-up questions by generating small networks. Applying multilevel models, we find almost no interviewer effects in CATI mode, where interviewers are paid by the hour and frequently supervised. In CAPI, however, where interviewers are paid by case and no close supervision is possible, we find strong interviewer effects on network size. As the area-specific network size is known from telephone mode, where allocation to interviewers is random, interviewer and area effects can be separated. Furthermore, a difference-in-difference analysis reveals the negative effect of introducing the follow-up questions in Wave 3 on CAPI network size. Attempting to explain interviewer effects we neither find significant main effects of experience within a wave, nor significantly different slopes between interviewers.

Keywords: Partial falsification; network generator; filter questions; interviewer cheating

7. References

  • American Association for Public Opinion Research (AAPOR). 2003. Interviewer Falsification in Survey Research: Current Best Methods for Prevention, Detection, and Repair of its Effects. Available at: https://www.aapor.org/AAPOR_Main/media/MainSiteFiles/falsification.pdf (accessed April 6th, 2016).

  • Bethmann, A. and D. Gebhardt. 2011. User Guide “Panel Study Labor Market and Social Security” (PASS) * Wave 3. FDZ Datenreport, 04/2011 (en), Nuremberg. Available at: http://doku.iab.de/fdz/reporte/2011/DR_04-11_EN.pdf (accessed April 6th, 2016).

  • Biemer, P.P. and S.L. Stokes. 1989. “The Optimal Design of Quality Control Samples to Detect Interviewer Cheating.” Journal of Official Statistics 5: 23–39.Google Scholar

  • Biemer, P.P. 2010. “Overview of Design Issues: Total Survey Error.” In Handbook of Survey Research, 2nd ed., edited by P.V. Marsden and J.D. Wright, 27–58. Bingley: Emerald.Google Scholar

  • Blasius, J. and J. Friedrichs. 2013. “Faked Interviews.” In Methods, Theories, and Empirical Applications in the Social Sciences, edited by S. Salzborn, E. Davidov, and J. Reinecke, 49–56. Wiesbaden: VS Verlag für Sozialwissenschaften.Google Scholar

  • Brashears, M.E. 2011. “Small Networks and High Isolation? A Reexamination of American Discussion Networks.” Social Networks 33: 331–341. Doi: http://dx.doi.org/10.1016/j.socnet.2011.10.003.Crossref

  • Bredl, S., P. Winker, and K. Koetschau. 2012. “A Statistical Approach to Detect Interviewer Falsification of Survey Data.” Survey Methodology 38: 1–10.Google Scholar

  • Brüderl, J., B. Huyer-May, and C. Schmiedeberg. 2013. “Interviewer Behavior and the Quality of Social Network Data.” In Interviewers’ Deviations in Surveys. Impact, Reasons, Detection and Prevention, edited by P. Winkler, R. Porst, and N. Menold, 147–160. Frankfurt: Peter Lang.Google Scholar

  • Büngeler, K., M. Gensicke, J. Hartmann, R. Jäckle, and N. Tschersich. 2010. IAB-Haushaltspanel im Niedrigeinkommensbereich Welle 3 (2008/09): Methoden- und Feldbericht. FDZ Methodenreport, 10/2010 (de), Nuremberg. Available at: http://doku.iab.de/fdz/reporte/2010/MR_10-10.pdf (accessed April 6th, 2016).

  • Campanelli, P. and C. O’Muircheartaigh. 1999. “Interviewers, Interviewer Continuity, and Panel Survey Nonresponse.” Quality & Quantity 33: 59–76. Doi: http://dx.doi.org/10.1023/A:1004357711258.Crossref

  • Cleveland, W.S. 1979. “Robust Locally Weighted Regression and Smoothing Scatterplot.” Journal of the American Statistical Association 74: 829–836.CrossrefGoogle Scholar

  • Cragg, J.G. 1971. “Some Statistical Models for Limited Dependent Variables with Application to the Demand for Durable Goods.” Econometrica 39: 829–844. Doi: http://dx.doi.org/10.2307/1909582.Crossref

  • Crespi, L.P. 1945. “The Cheater Problem in Polling.” Public Opinion Quarterly 9: 431–445.CrossrefGoogle Scholar

  • Eckman, S., F. Kreuter, A. Jäckle, A. Kirchner, S. Presser, and R. Tourangeau. 2014. “Assessing the Mechanisms of Misreporting to Filter Questions in Surveys.” Public Opinion Quarterly 78: 721–733. Doi: http://dx.doi.org/10.1093/poq/nfu030.Crossref

  • Fischer, C.S. 2009. “The 2004 GSS Finding of Shrunken Social Networks: An Artifact?” American Sociological Review 74: 657–669. Doi: http://dx.doi.org/10.1177/000312240907400408.Crossref

  • Freeman, J. and E.W. Butler. 1976. “Some Sources of Interviewer Variance in Surveys.” Public Opinion Quarterly 40: 79–91. Doi: http://dx.doi.org/10.1086/268-269.Crossref

  • Goldstein, H. 1986. “Multilevel Mixed Linear Model Analysis Using Iterative Generalized Least Squares.” Biometrika 73: 43–56. Doi: http://dx.doi.org/10.1093/biomet/73.1.43.Crossref

  • Groves, R.M. and N.H. Fultz. 1985. “Gender Effects among Telephone Interviewers in a Survey of Economic Attitudes.” Sociological Methods Research 14: 31–52. Doi: http://dx.doi.org/10.1177/0049124185014001002.Crossref

  • Groves, R.M. 1989. Survey Errors and Survey Costs. New York: Wiley.Google Scholar

  • Groves, R.M., F.J. Fowler, M.P. Couper, J.M. Lepkowski, E. Singer, and R. Tourangeau. 2004. Survey Methodology. Hoboken, NJ: Wiley.Google Scholar

  • Guterbrock, T.M. 2008. “Falsifications.” In Handbook of Survey Research, edited by P.J. Lavrakas, 267–270. Los Angeles: Sage.Google Scholar

  • Huddy, L., J. Billig, J. Bracciodieta, L. Hoeffler, P.J. Moynihan, and P. Pugliani. 1997. “The Effect of Interviewer Gender on the Survey Response.” Political Behavior 19: 197–220. http://dx.doi.org/10.1023/A:1024882714254.Crossref

  • Hughes, A., J. Chromy, K. Giacoletti, and D. Odom. 2002. “Impact of Interviewer Experience on Respondent Reports of Substance Use.” In Redesigning an Ongoing National Household Survey, edited by J. Gfroerer, J. Eyerman, and J. Chromy, 161–184. Washington: Substance Abuse and Mental Health Services Administration.Google Scholar

  • Kosyakova, Y., J. Skopek, and S. Eckman. 2015. “Do Interviewers Juggle Filter Questions? Evidence from a Multilevel Approach”. International Journal of Public Opinion Research 27: 417–431. Doi: http://dx.doi.org/10.1093/ijpor/edu027.Crossref

  • Kreuter, F., S. McCulloch, S. Presser, and R. Tourangeau. 2011. “The Effects of Asking Filter Questions in Interleafed Versus Grouped Format.” Sociological Methods & Research 40: 88–104. Doi: http://dx.doi.org/10.1177/0049124110392342.Crossref

  • Lechner, M. 2011. “The Estimation of Causal Effects by Difference-In-Difference Methods.” Foundations and Trends in Econometrics 4: 165–224.Google Scholar

  • Longford, N.T. 1993. Random Coefficient Models. Oxford: Oxford University Press.Google Scholar

  • Mangione, T.W., F.J. Fowler, and T.A. Louis. 1992. “Question Characteristics and Interviewer Effects.” Journal of Official Statistics 8: 293–307.Google Scholar

  • Marsden, P.V. 2003. “Interviewer Effects in Measuring Network Size Using a Single Name-Generator.” Social Networks 25: 1–16. Doi: http://dx.doi.org/10.1016/S0378-8733(02)00009-6.Crossref

  • Matschinger, H., S. Bernert, and M.C. Angermeyer. 2005. “An Analysis of Interviewer Effects on Screening Questions in a Computer Assisted Personal Mental Health Interview.” Journal of Official Statistics 21: 657–674.Google Scholar

  • McPherson, M., L. Smith-Lovin, and M.E. Brashears. 2006. “Social Isolation in America: Changes in Core Discussion Networks over Two Decades.” American Sociological Review 71: 353–375. Doi: http://dx.doi.org/10.1177/000312240607100301.Crossref

  • Nübling, M., H.H. Andersen, A. Mühlbacher, J. Schupp, and G.G. Wagner. 2007. “Computation of Standard Values for Physical and Mental Health Scale Scores Using the SOEP Version of SF12v2.” Schmollers Jahrbuch: Journal of Applied Social Science Studies 127: 171–182. Available at: https://www.researchgate.net/publication/23645941_Computation_of_Standard_Values_for_Physical_and_Mental_Health_Scale_Scores_Using_the_SOEP_Version_of_SF12v2 (accessed April 6th, 2016).

  • O’Connell, A.A. 2006. Logistic Regression Models for Ordinal Response Variables. Thousand Oaks, CA: Sage.Google Scholar

  • O’Muircheartaigh, C. and P. Campanelli. 1998. “The Relative Impact of Interviewer Effects and Sample Design Effects on Survey Precision.” Journal of the Royal Statistical Society. Series A (Statistics in Society) 161: 63–77. Doi: http://dx.doi.org/10.1111/1467-985X.00090.Crossref

  • Paik, A. and K. Sanchagrin. 2013. “Social Isolation in America: An Artifact.” American Sociological Review 78: 339–360. Doi: http://dx.doi.org/10.1177/0003122413482919.Crossref

  • Pinheiro, J.C. and D.M. Bates. 1995. “Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model.” Journal of Computational and Graphical Statistics 4: 12–35. Doi: http://dx.doi.org/10.1080/10618600.1995.10474663.Crossref

  • Pinheiro, J.C. and E.C. Chao. 2006. “Efficient Laplacian and Adaptive Gaussian Quadrature Algorithms for Multilevel Generalized Linear Mixed Models.” Journal of Computational and Graphical Statistics 15: 58–81. Doi: http://dx.doi.org/10.1198/106186006X96962.Crossref

  • Putnam, R.D. 1995. “Bowling Alone: America’s Declining Social Capital.” Journal of Democracy 6: 65–78. Doi: http://dx.doi.org/10.1353/jod.1995.0002.Crossref

  • Rabe-Hesketh, S. and A. Skrondal. 2012. Multilevel and Longitudinal Modeling Using Stata. Volume II:Categorical Responses, Counts, and Survival. College Station, TX: Stata Press.Google Scholar

  • Rasbash, J. and H. Goldstein. 1994. “Efficient Analysis of Mixed Hierarchical and Cross-Classified Random Structures Using a Multilevel Model.” Journal of Educational and Behavioral Statistics 19: 337–350. Doi: http://dx.doi.org/10.3102/10769986019004337.Crossref

  • Schnell, R. 1991. “Der Einfluß gefälschter Interviews auf Survey-Ergebnisse.” Zeitschrift für Soziologie 20: 25–35.Google Scholar

  • Schnell, R. 2012. Survey-Interviews: Methoden standardisierter Befragungen. Wiesbaden: VS Verlag.Google Scholar

  • Schnell, R. and F. Kreuter. 2000. “Untersuchungen zur Ursache unterschiedlicher Ergebnisse sehr ähnlicher Viktimisierungssurveys.” Kölner Zeitschrift für Soziologie und Sozialpsychologie 52: 96–117. Doi: http://dx.doi.org/10.1007/s11577-000-0005-y.Crossref

  • Schnell, R. and F. Kreuter. 2005. “Separating Interviewer and Sampling-Point Effects.” Journal of Official Statistics 21: 389–410.Google Scholar

  • Schraepler, J.P. and G.G. Wagner. 2005. “Characteristics and Impact of Faked Interviews in Surveys – An Analysis of Genuine Fakes in the Raw Data of SOEP.” Allgemeines Statistisches Archiv 89: 7–20. Doi: http://dx.doi.org/10.1007/s101820500188.Crossref

  • Schuman, H. and J. Converse. 1971. “The Effects of Black and White Interviewers on Black Responses in 1968.” Public Opinion Quarterly 35: 44–68. Doi: http://dx.doi.org/10.1086/267866.

  • Snijders, T.A.B. and R. Bosker. 2000. Multilevel Analysis. London: Sage.Google Scholar

  • StataCorp. 2011. Stata. Longitudinal-Data/Panel-Data Reference Manual. Release 12. College Station, TX: StataCorp.Google Scholar

  • Swamy, P.A.V.B. 1971. Statistical Inference in a Random Coefficient Model. New York: Springer.Google Scholar

  • Tourangeau, R. and T. Yan. 2007. “Sensitive Questions in Surveys.” Psychological Bulletin 133: 859–883.CrossrefGoogle Scholar

  • Tourangeau, R., F. Kreuter, and S. Eckman. 2012. “Motivated Underreporting in Screening Interviews.” Public Opinion Quarterly 76: 453–469. Doi: http://dx.doi.org/10.1093/poq/nfs033.Crossref

  • Tourangeau, R., F. Kreuter, and S. Eckman. 2013. Motivated Misreporting: Shaping Answers to Reduce Survey Burden. In Survey Measurement: Techniques and Findings from Recent Research, edited by U. Engel, 24–41, Frankfurt: Campus.Google Scholar

  • Trappmann, M., S. Gundert, C. Wenzig, and D. Gebhardt. 2010. “PASS: a Household Panel Survey for Research on Unemployment and Poverty.” Schmollers Jahrbuch. Journal of Applied Social Science Studies 130: 609–622 Doi: http://dx.doi.org/10.3790/schm.130.4.609.Crossref

  • Trappmann, M., J. Beste, A. Bethmann, and G. Mü ller. 2013. “The PASS Panel Survey After Six Waves.” Journal for Labour Market Research 46: 275–281. Doi: http://dx.doi.org/10.1007/s12651-013-0150-1.Crossref

  • van der Zouwen, J., W. Dijkstra, and J.H. Smit. 2004. “Studying Respondent-Interviewer Interaction: The Relationship Between Interviewing Style, Interviewer Behavior, and Response Behavior.” In Measurement Errors in Surveys, edited by P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowetz, and S. Sudman, 419–437. New York: Wiley.Google Scholar

  • van Tilburg, T.G. 1998. “Interviewer Effects in the Measurement of Personal Network Size. A Non-Experimental Study.” Sociological Methods and Research 26: 300–328. Doi: http://dx.doi.org/10.1177/0049124198026003002.Crossref

  • Vassallo, R., G.B. Durrant, and P.W.F. Smith. 2016. Separating Interviewer and Area Effects Using a Cross-Classified Multilevel Logistic Model: Simulation Findings and Implications for Survey Designs. Submitted manuscript (available from the author on request: ).Google Scholar

  • West, B.T., F. Kreuter, and U. Jaenichen. 2013. “Interviewer Effects in Face-to-Face Surveys: A Function of Sampling, Measurement Error, or Nonresponse?” Journal of Official Statistics 29: 277–297. Doi: http://dx.doi.org/10.2478/jos-2013-0023.Crossref

About the article

Received: 2014-08-28

Revised: 2015-06-27

Accepted: 2015-09-08

Published Online: 2016-05-28

Published in Print: 2016-06-01

Citation Information: Journal of Official Statistics, Volume 32, Issue 2, Pages 349–373, ISSN (Online) 2001-7367, DOI: https://doi.org/10.1515/jos-2016-0020.

Export Citation

© 2016 Michael Josten et al., published by De Gruyter Open. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Thomas W. Valente, Leanne Dougherty, and Emily Stammer
Field Methods, 2017, Page 1525822X1770371
Brady T. West and Annelies G. Blom
Journal of Survey Statistics and Methodology, 2016, Page smw024

Comments (0)

Please log in or register to comment.
Log in