Jump to ContentJump to Main Navigation
Show Summary Details
More options …

ICAME Journal

1 Issue per year

Open Access
See all formats and pricing
More options …

Assessing the inter-coder reliability of the Body Type Dictionary (BTD)

Laura A. Cariola
Published Online: 2014-04-28 | DOI: https://doi.org/10.2478/icame-2014-0001


Computer-assisted content analysis has many advantages compared to a manual scoring system, provided that computerised dictionaries represent valid and reliable measures. This study aimed to assess the inter-coder reliability, alternate- form reliability and scoring consistency of the Body Type Dictionary (BTD) (Wilson 2006) based on Fisher and Cleveland’s (1956, 1958) manual body boundary scoring scheme. The results indicated an acceptable inter-coder agreement with barrier and penetration imagery in the sub-sample (N = 53) of manually coded Rorschach responses. Additionally manually coded scores showed an acceptable correlation with the computerised frequency counts, and thus indicating an alternate-form reliability. In the full data set (N = 526), barrier imagery in the Rorschach responses only correlated with the picture response test, showing low scoring consistency, which might disconfirm the notion of body boundary awareness representing a stable personality trait but instead it might be dependent on the level of cognitive dedifferentiation.


  • Ackerman, Steven J., Mark J. Hilsenroth, Amanda J. Clemence, Robin Weatherill and I. Christopher Fowler. 2001. Convergent validity of Rorschach and TAT scales of object relations. Journal of Personality Assessment 77: 295-306.CrossrefGoogle Scholar

  • Anzieu, Didier. 1989. The skin ego. New Haven: Yale University Press.Google Scholar

  • Argamon, Shlomo and Shlomo Levitan. 2005. Measuring the usefulness of function words for authorship attribution. Proceedings of the 2005 ACH/ ALLC conference, June 2005. Victoria, Canada.Google Scholar

  • Artstein, Ron and Massimo Poesio. 2007. Inter-coder agreement for computational linguistics. Computational Linguistics 34: 555-596.Google Scholar

  • Bick, Esther. 1968. Experience of the skin in early object relations. International Journal of Psycho-Analysis 49: 484-486.Google Scholar

  • Buchanan, Tom. 2002. One assessment: Desirable or dangerous? Professional Psychology: Research and Practice 33: 148-154.CrossrefGoogle Scholar

  • Buck, Lucien A. and Michael Barden. 1971. Body image scores and varieties of consciousness. Journal of Personality Assessment 35: 309-314.CrossrefGoogle Scholar

  • Burris, Christopher T. and John K. Rempel. 2004. ‘It’s the end of the world as we know it’: Threat and the spatial-symbolic self. Journal of Personality and Social Psychology 86: 19-42.CrossrefGoogle Scholar

  • Cariola, Laura A. 2012. A case study of primary process language and body boundary imagery in discourses of religious-mystical and psychotic altered states of consciousness. Empirical Text and Cultural Research - ETC 5: 36-61.Google Scholar

  • Chang, Cindy and James W. Pennebaker. 2007. The psychology of function words. In K. Fiedler (ed.). Social communication, 343-359. New York: Psychology Press.Google Scholar

  • Fisher, Seymour and Sidney E. Cleveland. 1956. Body-image boundaries and style of life. Journal of Abnormal and Social Psychology 52: 373-379.CrossrefGoogle Scholar

  • Fisher, Seymour and Sidney E. Cleveland. 1958. Body image and personality. New York: Dover Publications.Google Scholar

  • Fisher, Seymour. 1986. Development and structure of the body image. Hillsdale: Lawrence Erlbaum.Google Scholar

  • Fleiss, Joseph L. 1981. Statistical methods for rates and proportions. New York: John Wiley.Google Scholar

  • Fox, John. 2005. The R commander: A basic statistics graphical user interface to R. Journal of Statistical Software 14: 1-42.Google Scholar

  • Friedman, Milton. 1937. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32: 675-701.CrossrefGoogle Scholar

  • Garmer, Matthias, Jim Lemon, Ian Fellows and Suspendra Singh. 2012. Various coefficients of interrater reliability and agreement. Available at http://www.cran.r-project.org/web/packages/irr/irr.pdf. Last accessed on 19 January, 2014.Google Scholar

  • Gigerenzer, Gerd. 2004. Mindless statistics. The Journal of Socio-Economics 33: 587-606.Google Scholar

  • Guimon, Jose. 1997. Corporality and psychoses. In J. Guimon (ed.). The body in psychotherapy, 63-72. Basel: Karger. Google Scholar

  • Hogenraad, Robert, Claude Daubies, Yves Bestgen and Pierre Mahau. 2003. Une théorie et une méthode générale d’analyse textuelle assistée par ordinateur: Le systéme PROTAN (PROTocol ANalyzer). 32-bits version of November 10, 2003 by Pierre Mahau. Louvain-la-Neuve: Psychology Department, Catholic University of Louvain.Google Scholar

  • Jackson, Sherri. 2011. Research methods and statistics: A critical thinking approach. Belmont: Wadsworth.Google Scholar

  • Kolbe, Richard H. and Melissa S. Burnett. 1991. Content-analysis research: An examination of applications with directives for improving research reliability and objectivity. Journal of Consumer Research 18: 243-250.CrossrefGoogle Scholar

  • Krippendorff, Klaus. 2004. Content analysis: An introduction to its methodology. Thousand Oaks: Sage. (First published 1980).Google Scholar

  • Lacy, Stephen and Daniel Riffle. 1996. Sampling error and selecting intercoder reliability samples for nominal content categories: Sins of omission and commission in mass communication quantitative research. Journalism & Mass Communication Quarterly 73: 969-973.Google Scholar

  • Lombard, Matthew, Jennifer Snyder-Duch and Cheryl Campanella Bracken. 2002. Content analysis in mass communication: Assessment and reporting of inter-coder reliability. Human Communication Research 28: 587-604.CrossrefGoogle Scholar

  • Lombard, Matthew, Jennifer Snyder-Duch and Cheryl Bracken. 2010. Practical resources for assessing and reporting intercoder reliability in content analysis research projects. Available at http://matthewlombard.com/reliability/. Last accessed on 29 January, 2014.Google Scholar

  • Michalczyk, Alan E. and Lloyd A. Lewis. 1980. Significance alone is not enough. Journal of Medical Education 55: 835-838.Google Scholar

  • Morgan, Christina D. and Henry A. Murray. 1935. A method of investigating fantasies: The Thematic Apperception Test. Archives of Neurology and Psychiatry 34: 289-306.CrossrefGoogle Scholar

  • Neuendorf, Kimberly A. 2002. The content analysis guidebook. London: Sage Publications.Google Scholar

  • Neundorf, Kimberly A. and Paul D. Skalsi. 2010. Extending the utility of content analysis via the scientific method. Manuscript in support of presentation to the Social Science Computing Workshop, University of Hawaii, Honolulu. Available at http://www.manoa.hawaii.edu/ccpv/workshops/KimberlyNeuendorf.pdf.Google Scholar

  • Ogden, Thomas H. 1989. The primitive edge of experience. Northvale: Jason Aronson. O’Neill, Regina M. 2005. Body image, body boundary and the Barrier and Penetration Rorschach scoring system. In R. F. Bornstein and J. M. Masling (eds.). Scoring the Rorschach: Seven validated systems, 159-189. London: Lawrence Erlbaum.Google Scholar

  • Orne, Martin T. 1962. On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist 17: 776-783.CrossrefGoogle Scholar

  • Passonneau, Rebecca. 2006. Measuring agreement on set-valued items (MASI) for semantic and pragmatic annotation. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). Genoa, Italy.Google Scholar

  • Potter, W. James and Deborah Levine-Donnerstein. 1999. Rethinking validity and reliability in content analysis. Journal of Applied Communication Research 27: 258-284.CrossrefGoogle Scholar

  • R Development Core Team. 2011. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Available at http://www.R-project.org. Last accessed on 19 January, 2014.Google Scholar

  • Rorschach, Hermann 1921. Psychodiagnostik. Leipzig: Ernst Bircher Verlag.Google Scholar

  • Rosenthal, Robert and Ralph L. Rosnow. 1984. Essentials of behavioural research: Methods and data analysis. London: McGraw-Hill.Google Scholar

  • Rourke, Liam, Terry Anderson, D. R. Garrison and Walter Archer. 2000. Methodology issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education 11: 8-22.Google Scholar

  • Saraceni, Carlo, Giuseppe Ruggeri and D. Filocamo. 1980. Studio sperimentale con il test di Rorschach sulle modificazioni dell’immagine corporea in ipnosi. Archivio di Psicologia, Neurologia e Psichiatria 41: 50-64.Google Scholar

  • Schmeidler, Gertrude and Lawrence LeShan. 1970. An aspect of body image related to ESP scores. Journal of the American Society for Psychical Research 64: 211-218.Google Scholar

  • Schmitt, Norbert and Bruce Dunham. 1999. Exploring native and non-native intuitions of Word frequency. Second Language Research 15: 389-411.Google Scholar

  • Schnurr, Paula P., Stanley D. Rosenberg, Thomas E. Oxman and Gary J. Tucker. 1986. A methodological note on content analysis: Estimates of reliability. Journal of Personality Assessment 50: 601-609.CrossrefGoogle Scholar

  • Singletary, Michael W. 1994. Mass communication research: Contemporary methods and applications. New York: Longman. Spearman, Charles. 1904. The proof and measurement of association between two things. American Journal of Psychology 15: 72-101.Google Scholar

  • Tinsley, Howard. E. A. and David J. Weiss. 1975. Interrater reliability and agreement of subjective judgements. Journal of Counseling Psychology 22: 358-376.CrossrefGoogle Scholar

  • Tustin, Frances. 1981. Autistic states in children. London: Routledge.Google Scholar

  • Weber, Robert P. 1990. Basic content analysis. London: Sage.Google Scholar

  • West, Alan N. 1991. Primary process content in the King James Bible: The five stages of Christian mysticism. Computers and the Humanities 25: 227-238.Google Scholar

  • Wilson, Andrew. 2006. The development and application of a content analysis dictionary for body boundary research. Literary and Linguistic Computing 21: 105-110.Google Scholar

  • Wilson, Andrew. 2008. Psychosomatic cycles and the liturgical year: A case study and framework for research. Gottingen: Cuvillier Verlag.Google Scholar

  • Wilson, Andrew. 2009. Barrier and penetration imagery in altered states of consciousness discourse: Replicating the five-stage model of Christian mysticism in the Bible. In W. Oleksy and P. Stalmaszczyk (eds.). Cognitive approaches to language and linguistic data: Studies in honor of Barbara Lewandowska-Tomaszczyk (Polish Studies in English Language and Literature 27), 357-372. Frankfurt am Main: Peter Lang.Google Scholar

  • Wilson, Andrew. 2011. The regressive imagery dictionary: A test of its concurrent validity in English, German, Latin and Portuguese. Literary and Linguistic Computing 26: 125-135. Google Scholar

About the article

Published Online: 2014-04-28

Citation Information: ICAME Journal, Volume 38, Issue 1, Pages 5–33, ISSN (Online) 1502-5462, DOI: https://doi.org/10.2478/icame-2014-0001.

Export Citation

This article is distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. BY-NC-ND 3.0

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Laura A. Cariola
Imagination, Cognition and Personality, 2014, Volume 34, Number 2, Page 133

Comments (0)

Please log in or register to comment.
Log in