Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter Mouton April 19, 2016

Smiling synchronicity and gaze patterns in dyadic humorous conversations

  • Elisa Gironzetti

    Elisa Gironzetti received her PhD in Spanish Language Teaching from the University of Alicante (Spain) with a dissertation on humorous and non-humorous communication through single-panel cartoons. She teaches in the Spanish undergraduate and graduate programs at Texas A&M University–Commerce. Her research focuses on humor and Spanish language teaching, and she is currently performing eye-tracking experiments to study the interaction of humor, gaze, and smiling in conversation.

    EMAIL logo
    , Lucy Pickering

    Lucy Pickering is Associate Professor and director of the Applied Linguistics Laboratory at Texas A&M University–Commerce. She received her Ph.D. in Applied Linguistics in 1999 from the University of Florida and has taught in University of Alabama, Georgetown University, and Georgia State University. Her research program is focused on spoken discourse. She has done considerable work with Brazil’s model of Discourse Intonation and its application to second language classroom discourse.

    , Meichan Huang

    Meichan Huang received her M.Sc. in TESOL (Teaching English to the Speakers of Other Languages) from the University of Edinburgh, and B.A. in English Translation from Sichuan International Studies University in China. Her major research interest is humor in second language classrooms, with a focus on the use of verbal humor as teaching material. She is also interested in second language acquisition and language pedagogy.

    , Ying Zhang

    Ying Zhang holds a PhD in English Linguistics and Literature from Shanghai International Studies University. She is now a lecturer in the School of Foreign Languages at Shanghai University. She worked as a visiting scholar sponsored by China Scholarship Council in the Department of Literature and Languages at Texas A&M University-Commerce from April 2014 to April 2015. Her research programs focus on conversational humor. Her major research interests are humor, pragmatics, sociolinguistics and applied linguistics.

    , Shigehito Menjo

    Shigehito Menjo received his M.A in Japanese Language and Pedagogy, and B.A in Linguistics from University of Oregon. He also received a TESOL certificate from Texas A&M University–Commerce. He has several years of experience in teaching Japanese, first year composition, and TESOL methods classes at universities in the United States. His major research interest is the acquisition of prosody in second language and the use of smiling and prosody in the spoken discourse of humor/comedy.

    and Salvatore Attardo

    Salvatore Attardo is Dean of the College of Humanities, Social Sciences and Arts at Texas A&M University-Commerce, where he is also Full Professor of Linguistics. He hold degrees from The Catholic University of Milan (1986) and Purdue University (PhD 1991). He has published extensively in humor, pragmatics, and semantics, primarily on issues relating to implicatures, irony, rationality and more generally on Neo-Gricean Pragmatics. His other areas of interest are in sociolinguistics, cognitive linguistics, and computational semantics.

From the journal HUMOR

Abstract

This present article is part of a larger study on speaker-hearer allocation of attentional resources in face-to-face interactions. The goal of the paper is twofold: first, we present results concerning the degree of correlation, in computer-mediated conversation, between speaker’s timing and intensity of smiling when humor is either present or absent in the conversation. The results were obtained from the analysis of five dyadic interactions between English speakers that were video and audio recorded, transcribed, and analyzed to establish a baseline of synchronicity of smiling among participants. From the study it emerged that conversational partners engaged in humorous conversations not only reciprocate each other’s smiling, but also match each other’s smiling intensity. Our data led to the identification of different smiling and non-smiling synchronic behaviors that point to the existence of a synchronous multimodal relationship between humorous events and smiling intensity for conversational partners. Second, in the last part of the paper, we argue for the need of a multimodal conversational corpus in humor studies and present the corpus that is being collected, annotated, and analyzed at Texas A&M University–Commerce. The corpus consists of humorous interactions among dyads of native speakers of English, Spanish, and Chinese for which video, audio, and eye-tracking data have been recorded. As part of this section of the paper, we also present some preliminary results based on the analysis of one English conversation, and some exploratory analysis of Chinese data, that show that greater attention is paid to facial areas involved in smiling when humor is present. This study sheds light on the role of smiling as a discourse marker (Attardo, S., L. Pickering, F. Lomotey & S. Menjo. 2013. Multimodality in Conversational Humor. Review of Cognitive Linguistics 11(2). 400–414.), and therefore as a meaningful device in verbal communication.

About the authors

Elisa Gironzetti

Elisa Gironzetti received her PhD in Spanish Language Teaching from the University of Alicante (Spain) with a dissertation on humorous and non-humorous communication through single-panel cartoons. She teaches in the Spanish undergraduate and graduate programs at Texas A&M University–Commerce. Her research focuses on humor and Spanish language teaching, and she is currently performing eye-tracking experiments to study the interaction of humor, gaze, and smiling in conversation.

Lucy Pickering

Lucy Pickering is Associate Professor and director of the Applied Linguistics Laboratory at Texas A&M University–Commerce. She received her Ph.D. in Applied Linguistics in 1999 from the University of Florida and has taught in University of Alabama, Georgetown University, and Georgia State University. Her research program is focused on spoken discourse. She has done considerable work with Brazil’s model of Discourse Intonation and its application to second language classroom discourse.

Meichan Huang

Meichan Huang received her M.Sc. in TESOL (Teaching English to the Speakers of Other Languages) from the University of Edinburgh, and B.A. in English Translation from Sichuan International Studies University in China. Her major research interest is humor in second language classrooms, with a focus on the use of verbal humor as teaching material. She is also interested in second language acquisition and language pedagogy.

Ying Zhang

Ying Zhang holds a PhD in English Linguistics and Literature from Shanghai International Studies University. She is now a lecturer in the School of Foreign Languages at Shanghai University. She worked as a visiting scholar sponsored by China Scholarship Council in the Department of Literature and Languages at Texas A&M University-Commerce from April 2014 to April 2015. Her research programs focus on conversational humor. Her major research interests are humor, pragmatics, sociolinguistics and applied linguistics.

Shigehito Menjo

Shigehito Menjo received his M.A in Japanese Language and Pedagogy, and B.A in Linguistics from University of Oregon. He also received a TESOL certificate from Texas A&M University–Commerce. He has several years of experience in teaching Japanese, first year composition, and TESOL methods classes at universities in the United States. His major research interest is the acquisition of prosody in second language and the use of smiling and prosody in the spoken discourse of humor/comedy.

Salvatore Attardo

Salvatore Attardo is Dean of the College of Humanities, Social Sciences and Arts at Texas A&M University-Commerce, where he is also Full Professor of Linguistics. He hold degrees from The Catholic University of Milan (1986) and Purdue University (PhD 1991). He has published extensively in humor, pragmatics, and semantics, primarily on issues relating to implicatures, irony, rationality and more generally on Neo-Gricean Pragmatics. His other areas of interest are in sociolinguistics, cognitive linguistics, and computational semantics.

References

Anolli, L. & L. Lambiase. 1990. “Giochi di sguardo” nella conversazione. Giornale Italiano di psicologia 17(1). 27–58.Search in Google Scholar

Attardo, S. 2001. Humorous texts. Berlin: Mouton de Gruyter.10.1515/9783110887969Search in Google Scholar

Attardo, S. 2012. Smiling, laughter, and humor. In Paolo Santangelo (ed.), Laughing in Chinese, 421–436. Rome: Aracne.Search in Google Scholar

Attardo, S, L. Pickering & A. Baker. 2011. Prosodic and multimodal markers of humor in conversation. Pragmatics and Cognition 19(2). 224–247.10.1075/bct.55.03attSearch in Google Scholar

Attardo, S., L. Pickering, F. Lomotey & S. Menjo. 2013. Multimodality in conversational humor. Review of Cognitive Linguistics 11(2). 400–414.10.1075/bct.78.12attSearch in Google Scholar

Beattie, G. W. 1978. Floor apportionment and gaze in conversational dyads. British Journal of Social and Clinical Psychology 17(1). 7–15.10.1111/j.2044-8260.1978.tb00889.xSearch in Google Scholar

Brône, G. & B. Oben. 2015. InSight Interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation 49(1). 195–214.10.1007/s10579-014-9283-2Search in Google Scholar

Broz, F., H. Lehmann, C. L. Nehaniv & K. Dautenhahn. 2012. Mutual gaze, personality, and familiarity: Dual eye-tracking during conversation. IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, 858–864.10.1109/ROMAN.2012.6343859Search in Google Scholar

Calvo, Manuel G., Andrés Fernández-Martín & Lauri Nummenmaa. 2013a. A smile biases the recognition of eye expressions: Configural projection from a salient mouth. The Quarterly Journal of Experimental Psychology 66(6). 1159–1181, DOI: 10.1080/17470218.2012.732586.Search in Google Scholar

Calvo, Manuel G., Aida Gutiérrez-García, Pedro Avero & Daniel Lundqvist. 2013b. Attentional mechanisms in judging genuine and fake smiles: Eye-movement patterns. Emotion 13(4). 792–802. DOI: 10.1037/a0032317.Search in Google Scholar

Cappella, J. N. 1997. Behavioral and judged coordination in adult informal social interactions: Vocal and Kinesic Indicators. Journal of Personality and Social Psychology 72(1). 119–131. http://dx.doi.org/10.1037/0022-3514.72.1.119.10.1037/0022-3514.72.1.119Search in Google Scholar

Cary, M. 1978. The role of gaze in the initiation of conversation. Social Psychology 41. 269–271.10.2307/3033565Search in Google Scholar

Chafe, W. 1994. Discourse, consciousness, and time. The flow and displacement of conscious experience in speaking and writing. Chicago: University of Chicago Press.Search in Google Scholar

Chartrand, T. L. & J. A. Bargh. 1999. The chameleon effect: The perception-behavior link and social interaction. Journal of Personality and Social Psychology 76(6). 893–910. doi:10.1037/0022-3514.76.6.893.Search in Google Scholar

Dale, R., R. Fusaroli, N. D. Duran & D. C. Richardson. 2013. The self-organization of human interaction. Psychology of Learning and Motivation 59. 43–96.10.1016/B978-0-12-407187-2.00002-2Search in Google Scholar

Dale, R., R. Fusaroli, D. D. Håkonsson, P. Healey, D. Mønster, J. J. McGraw, P. Mitkidis & K. Tylén. 2013. Beyond synchrony: Complementarity and asynchrony in joint action. Proceedings of CogSci 2013.Search in Google Scholar

Delaherche, E. & M. Chetouani. 2010. Multimodal Coordination: Exploring Relevant Features and Measures. In Proceedings of ACM Multimedia Second International Workshop. Social Signal Processing. Florence, Italy.10.1145/1878116.1878131Search in Google Scholar

Ekman, P. & W. V. Friesen. 1978. Facial action coding system. Palo Alto, California: Consulting Psychologists Press.10.1037/t27734-000Search in Google Scholar

Ekman, P., E. R. Sorenson & W. V. Friesen. 1969. Pan-cultural elements in facial displays of emotion. Science 164. 86–88.10.1126/science.164.3875.86Search in Google Scholar

Elfenbein, H. A. & N. Ambady. 2002. On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychology Bulletin 128. 203–235.10.1037/0033-2909.128.2.203Search in Google Scholar

Fernández-Martín, A. & M. Gutiérrez-Calvo. 2012. Curso temporal en la discriminación de la sonrisa: medidas de latencia sacádica. Psicológica 33. 319–343.Search in Google Scholar

Fernández-Martín, A., A. Gutiérrez-García & M. G. Calvo. 2013. A smile radiates outwards and biases the eye expression. Spanish Journal of Psychology 16(53). 1–11.10.1017/sjp.2013.68Search in Google Scholar

Fusaroli, R., J. Rączaszek-Leonardi & K. Tylén. 2014. Dialog as interpersonal synergy. New Ideas in Psychology 32. 147–157. doi:10.1016/j.newideapsych.2013.03.005.Search in Google Scholar

Fusaroli, R. & K. Tylén. 2012. Carving language for social interaction: a dynamic approach. Interaction Studies 13(1). 103–123.10.1075/is.13.1.07fusSearch in Google Scholar

Gervais, M. & D. S. Wilson. 2005. The evolution and functions of laughter and humor: A synthetic approach. Quarterly Review of Biology 80(4). 395–430.10.1086/498281Search in Google Scholar

Gironzetti, E., S. Attardo & L. Pickering. 2016. Smiling, gaze, and humor in conversation: A Pilot Study. In L. Ruiz Gurillo (ed.), Metapragmatics of humor: Current research trends. Amsterdam: John Benjamins.10.1075/ivitra.14.12girSearch in Google Scholar

Gironzetti, E. & S. Menjo. 2014. Smiling as a Discourse Marker of Humor. Paper presented at the 2nd Conference of the American Pragmatics Association (AMPRA), October 17–19, 2014. University of California, Los Angeles.Search in Google Scholar

Hay, J. 2001. The pragmatics of humor support: Humor – International Journal of Humor Research. Humor: International Journal of Humor Research 14(1). 55–82. http://doi.org/10.1515/humr.14.1.55.http://doi.org/10.1515/humr.14.1.55Search in Google Scholar

Heerey, E. A. & H. M. Crossley. 2013. Predictive and reactive mechanisms in smile reciprocity. Psychological Science 24(8). 1446–1455.10.1177/0956797612472203Search in Google Scholar

Heerey E. A. & A. M. Kring. 2007. Interpersonal consequences of social anxiety. Journal of Abnormal Psychology 116. 125–134.10.1037/0021-843X.116.1.125Search in Google Scholar

Hess U. & P. Bourgeois. 2010. You smile – I smile: Emotion expression in social interaction. Biological Psychology 84. 514–520.10.1016/j.biopsycho.2009.11.001Search in Google Scholar

Holt, E. & P. J. Glenn. 2013. Studies of Laughter in Interaction London, UK: Bloomsbury AcademicSearch in Google Scholar

Jefferson, G. 1979. “A technique for inviting laughter and its subsequent acceptance declination.” In George Psathas (ed.), Everyday language: Studies in ethnomethodology, 79–96. New York: Irvington.Search in Google Scholar

Kendon, A. 1967. Some functions of gaze-direction in social interaction. Acta Psychologica 26. 22–63.10.1016/0001-6918(67)90005-4Search in Google Scholar

Kleinke, C. L. 1986. Gaze and eye contact: A research review. PsyChological Bulletin 100(1). 78–100.10.1037/0033-2909.100.1.78Search in Google Scholar

Knackstedt, G. & C. Kleinke. 1991. Eye contact, gender, and personality judgments. The Journal of Social Psychology, 131. 303–304.10.1080/00224545.1991.9713855Search in Google Scholar

Lakin, J. L. & T. L. Chartrand. 2003. Using nonconscious behavioral mimicry to create affiliation and rapport. Psychological Science 14(4). 334–339. doi:10.1111/1467-9280.14481.Search in Google Scholar

Levine, M. H. & P. Sutton-Smith. 1973. Effects of age, sex and task on visual behavior during dyadic interaction. Developmental Psychology 9. 400–405.10.1037/h0034929Search in Google Scholar

Lumsden, J., K. M. Lynden, M. J. Richardson, C. A. Smith & C. N. Macrae. 2012. “Who Syncs? Social motives and interpersonal coordination.” Journal of Experimental Social Psychology 48(3). 746–751. Retrieved December 2, 2014 (http://www.sciencedirect.com/science/article/pii/S0022103111002976).10.1016/j.jesp.2011.12.007Search in Google Scholar

Macrae, C. N., O. K. Duffy, L. K. Miles & J. Lawrence. 2008. A case of hand waving: Action synchrony and person perception. Cognition 109. 152–156.10.1016/j.cognition.2008.07.007Search in Google Scholar

Miles, L. K., J. L. Griffiths, M. J. Richardson & C. N. Macrae. 2010. Too late to coordinate: Contextual influences on behavioral synchrony. European Journal of Social Psychology 40. 52–60.10.1002/ejsp.721Search in Google Scholar

Miles, L. K., L. K. Nind & C. N. Macrae. 2009. The rhythm of rapport: Interpersonal synchrony and social perception. Journal of Experimental Social Psychology 45(3). 585–589. http://doi.org/10.1016/j.jesp.2009.02.002.10.1016/j.jesp.2009.02.002Search in Google Scholar

Norrick, N. R. 1993. Conversational joking: Humor in everyday talk. Bloomington, Indiana: Indiana University Press.Search in Google Scholar

O‘Donnell-Trujillo, N. & K. Adams. 1983. Heheh in conversation: Some coordinating accomplishments of laughter. Western Journal of Speech Communication 47. 175–191.10.1080/10570318309374114Search in Google Scholar

Paxton, A. & R. Dale. 2013. Multimodal networks for interpersonal interaction and conversational contexts. Proceedings of the 35th Annual Meeting of the Cognitive Science Society. Retrieved December 2, 2014 (http://alexandrapaxton.com/wp-content/uploads/2013/05/Paxton-Dale-2013-CogSci.pdf).Search in Google Scholar

Pickering, L., M. Corduas, J. Eisterhold, B. Seifried, A. Eggleston & S. Attardo. 2009. Prosodic markers of saliency in humorous narratives. Discourse Processes 46. 517–540.10.1080/01638530902959604Search in Google Scholar

Provine, R. R. 2004. Laughing, tickling, and the evolution of speech and self. Current Directions in Psychological Science 13(6). 215–218. http://doi.org/10.1111/j.0963-7214.2004.00311.x10.1111/j.0963-7214.2004.00311.xSearch in Google Scholar

Raidt, S., G. Bailly & F. Elisei. 2007. Analyzing and modeling gaze during face-to-face interactions. 7th International Conference on Intelligent Virtual Agents (IVA), Paris, France, 100–101.Search in Google Scholar

Raskin, V. 1985. Semantic mechanisms of humor. Dodrecht, the Netherlands: D. Reidel.10.1007/978-94-009-6472-3Search in Google Scholar

Rosengrant, D., D. Hearrington, K. Alvarado, & D. Keeble. 2012. Following Student Gaze Patterns in Physical Science Lectures. AIP Conference Proceedings, 1413. 323–326. doi: http://dx.doi.org/10.1063/1.368006010.1063/1.3680060Search in Google Scholar

Sacks, H. 1974 [1989]. An analysis of the course of a joke’s telling in conversation. In Bauman, R. & Joel Sherzer (eds.) Explorations in the ethnography of speaking, 2nd edn, 337–353. Cambridge: Cambridge University Press.10.1017/CBO9780511611810.022Search in Google Scholar

Sauter, D. A., F. Eisner, P. Ekman & S. K. Scott. 2010. Cross-cultural recognition of basic emotions through nonverbal emotional vocalizations. Proceedings of the National Academy of Sciences 107(6). 2408–2412. http://doi.org/10.1073/pnas.0908239106.http://doi.org/10.1073/pnas.0908239106Search in Google Scholar

Schegloff, E. A., G. Jefferson & H. Sacks. 1977. The preference for self-correction in the organization of repair in conversation. Language 53. 361–382.10.1353/lan.1977.0041Search in Google Scholar

Scott, S. K., N. Lavan, S. Chen & C. McGettigan. 2014. The social life of laughter. Trends in Cognitive Sciences 18(12). 618–620. http://doi.org/10.1016/j.tics.2014.09.002.http://doi.org/10.1016/j.tics.2014.09.002Search in Google Scholar

Tschacher, W., G. M. Rees & F. Ramseyer. 2014. Nonverbal synchrony and affect in dyadic interactions. Frontiers in Psychology 5. 1323. doi:10.3389/fpsyg.2014.01323.Search in Google Scholar

Valdesolo, P., J. Ouyang & D. A. DeSteno. 2010. The rhythm of joint action: Synchrony promotes cooperative ability. Journal of Experimental Social Psychology 46. 693–695.10.1016/j.jesp.2010.03.004Search in Google Scholar

van Hooff, J. 1972. A comparative approach to the phylogeny of laughter and smiling. In R. A. Hinde (ed.), Nonverbal communication. Cambridge: Cambridge University Press.Search in Google Scholar

Vertegaal, R., R. Slagter, G. van der Veer & A. Nijholt. 2001. Eye gaze patterns in conversation: There is more to conversational agents than meets the eyes. CHI 3(1). 301–308.10.1145/365024.365119Search in Google Scholar

Wild B., M. Erb, M. Eyb, M. Bartels & W. Grodd. 2003. Why are smiles contagious? An fMRI study of the interaction between perception of facial affect and facial movements. Psychiatry Research 123. 17–36.10.1016/S0925-4927(03)00006-4Search in Google Scholar

Williams, J. A., E. L. Burns & E. A. Harmon. 2009. Insincere utterances and gaze: Eye contact during sarcastic statements. Perceptual and Motor Skills 108. 565–572.10.2466/pms.108.2.565-572Search in Google Scholar

Appendix A. Smiling Intensity Scale

The five levels of this Smiling Intensity Scale (SIS) are descriptive of different smiling behaviors:

  1. Level 0: Neutral. No smile, no flexing of the zygomaticus (no AU12), may show dimpling (AU14) or squinting of the eyes (caused by AU6 or AU7), but no raised side of the mouth (no AU 12), the mouth may be closed or open (AU25 or AU26).

  2. Level 1: Closed mouth smile. Shows flexing of the zygomaticus (AU12), may show dimpling (AU14) and may show flexing of the orbicularis oculi (caused by AU6 or AU7).

  3. Level 2: Open mouth smile. Showing upper teeth (AU25), flexing of the zygomaticus (AU12), may show dimpling (AU14), may show flexing of the orbicularis oculi (caused by AU6 or AU7).

  4. Level 3: Wide open mouth smile. Shows flexing of the zygomaticus (AU12), flexing of the orbicularis oculi (caused by AU6 or AU7), and may show dimpling (AU14). 3A: showing lower and upper teeth (AU25), or 3B: showing a gap between upper and lower teeth (AU25 and AU26).

  5. Level 4: Laughing smile. The jaw is dropped (AU25 and AU26 or AU27), showing lower and upper teeth, flexing zygomaticus (AU12), flexing of the orbicularis oculi (AU6 or AU7), dimpling (AU14)

Published Online: 2016-4-19
Published in Print: 2016-5-1

©2016 by De Gruyter Mouton

Downloaded on 28.3.2024 from https://www.degruyter.com/document/doi/10.1515/humor-2016-0005/pdf
Scroll to top button