Skip to content
Licensed Unlicensed Requires Authentication Published online by De Gruyter Mouton September 21, 2020

“Do you understand (me)?” negotiating mutual understanding by using gaze and environmentally coupled gestures between two deaf signing participants

Nina Sivunen and Elina Tapio

Abstract

In this paper we explore the use of multimodal and multilingual semiotic resources in interactions between two deaf signing participants, a researcher and an asylum seeker. The focus is on the use of gaze and environmentally coupled gestures. Drawing on multimodal analysis and linguistic ethnography, we demonstrate how gaze and environmentally coupled gestures are effective semiotic resources for reaching mutual understanding. The study provides insight into the challenges and opportunities (deaf) asylum seekers, researchers, and employees of reception centres or the state may encounter because of the asymmetrical language competencies. Our concern is that such asymmetrical situations may be created and maintained by ignoring visual and embodied resources in interaction and, in the case of deaf asylum seekers, by unrealistic expectations towards conventionalized forms of international sign.


Corresponding author: Nina Sivunen, University of Jyväskylä, Centre of Applied Language Studies, Jyväskylä, Finland, E-mail:

Acknowledgments

We want to express our thanks to Sari Pöyhönen, Mia Halonen, Lindsay Ferrara and Teppo Jakonen for their valuable comments and insights on previous versions of this article. We would also like to thank the anonymous reviewers whose detailed comments and suggestions helped to improve the manuscript. Nina Sivunen is grateful for the support from the Centre of Applied Language Studies (CALS) of the University of Jyväskylä.

  1. Research funding: This research was funded by the University of Jyväskylä, Faculty of Humanities and Social Sciences, Humak University of Applied Sciences and by the EU Home Affairs Funds (SMDno:2016-1561) through the PICCORE project.

Appendix

[

beginning of overlapping sign

]

end of overlapping sign

(0.5)

pauses timed in tenths of a second (approximately)

capitals

gloss for a manual sign and a gesture

cd:xx

constructed dialogue

pnt-2

pointing to another interlocutor

pnt-right/left

lexical pointing direction

pnt-table/door

pointing to an object in the environment

tch-table

touch

-h

hold

hs

head shakings

hn

head noddings

shdr dwn

shoulders down

xxxx

eyegaze direction, head and body activities, non-manual elements (e.g. raised eyebrows)

fig. x

The location of the picture (figure x) in the partiture

References

Bateman, John, Janina Wildfeuer & Tuomo Hiippala. 2017. Multimodality: Foundations, research and analysis a problem-oriented introduction. Berlin: De Gruyter.10.1515/9783110479898Search in Google Scholar

Bezemer, Jeff & Gunther Kress. 2014. Touch: A resource for making meaning. Australian Journal of Language and Literacy 37(2). 77–85.Search in Google Scholar

Blommaert, Jan, James Collins & Stef Slembrouck. 2005. Spaces of multilingualism. Multilingualism and Diasporic Populations: Special Issue of Language and Communication. 25(2). 197–215. https://doi.org/10.1016/j.langcom.2005.05.002.Search in Google Scholar

Copland, Fiona & Angela Creese. 2015. Linguistic ethnography: Collecting, analysing and presenting data. London: Sage.10.4135/9781473910607Search in Google Scholar

Cormier, Kearsy, Sandra Smith & Zed Sevcikova-Sehyr. 2015. Rethinking constructed action. Sign Language and Linguistics 18(2). 167–204. https://doi.org/10.1075/sll.18.2.01cor.Search in Google Scholar

Crasborn, Onno & Anja Hiddinga. 2015. The paradox of international sign: The importance of deaf-hearing encounters for deaf-deaf communication across sign language borders. In Michele Friedner & Annelies Kusters (eds.), It’s a Small World: International deaf spaces and encounters, 59–69. Washington, DC: Gallaudet University Press.Search in Google Scholar

Crasborn, Onno & Han Sloetjes. 2008. Enhanced ELAN functionality for sign language corpora. In: Proceedings of LREC 2008, sixth international conference on language resources and evaluation.Search in Google Scholar

Drew, Paul. 1997. ‘Open’ class repair initiators in response to sequential sources of troubles in conversation. Journal of Pragmatics 28(1). 69–101. https://doi.org/10.1016/s0378-2166(97)89759-7.Search in Google Scholar

Engberg-Pedersen, Elisabeth. 2013. From pointing to reference and predication: Pointing signs, eyegaze, and head and body orientation in Danish sign language. In Sotaro Kita (ed.), Pointing: Where language, culture, and cognition meet, 269–292. Mahwah, NJ: Erlbaum.Search in Google Scholar

Goodwin, Marjorie & Charles Goodwin. 1986. Gesture and co-participation in the activity of searching for a word. Semiotica 62(1-2). 51–75 https://doi.org/10.1515/semi.1986.62.1-2.51.Search in Google Scholar

Goodwin, Charles. 2000. Action and embodiment within situated human interaction. Journal of Pragmatics 32(10). 1489–1522 https://doi.org/10.1016/s0378-2166(99)00096-x.Search in Google Scholar

Goodwin, Charles. 2003. Pointing as situated practice. In Sotaro Kita (ed.), Pointing: Where language, culture and cognition meet, 217–241. Mahwah, NJ: Erlbaum.Search in Google Scholar

Goodwin, Charles. 2007. Environmentally coupled gestures. In Susan D. Duncan, Justine Cassel & Elena T. Levy (eds.), Gesture and the dynamic dimension of language, 195–212. Amsterdam: John Benjamins Publishing Company.10.1075/gs.1.18gooSearch in Google Scholar

Green, E. Mara. 2015. One language, or maybe two: Direct communication, understanding, and informal interpreting in international deaf encounters. In. Michele Friedner & Annelies Kusters (eds.), It’s a small world: International deaf spaces and encounters, 70–82. Washington, DC: Gallaudet University Press.Search in Google Scholar

Groeber, Simone & Evelyne Pochon-Berger. 2014. Turns and turn-taking in sign language interaction: A study of turn-final holds. Journal of Pragmatics 65. 121–136. https://doi.org/10.1016/j.pragma.2013.08.012.Search in Google Scholar

Heritage, John & Steven Clayman. 2010. Talk in action: Interactions, identities, and institutions. Chichester: Wiley-Blackwell.10.1002/9781444318135Search in Google Scholar

Hiddinga, Anja & Onno Crasborn. 2011. Signed languages and globalization. Language in Society 40(4). 483–505 https://doi.org/10.1017/s0047404511000480.Search in Google Scholar

Hutchins, Edwin & Saeko Nomura. 2011. Collaborative construction of multimodal utterances. In Jürgen Streeck, Charles Goodwin & Curtis, LeBaron (eds.), Embodied interaction: Language and body in the material world, 29–42. Cambridge: Cambridge University Press.Search in Google Scholar

Jefferson, Gail. 2004. Glossary of transcript symbols with an introduction. In Gene H. Lerner (ed.), Conversation analysis. Studies from the first generation, 13–31. Amsterdam: John Benjamins.10.1075/pbns.125.02jefSearch in Google Scholar

Jewitt, Carey. 2004. Multimodality and new communication technologies. In Philip LeVine & Ron Scollon (eds.), Discourse & technology: Multimodal discourse analysis, 184–195. Washington, D.C.: Georgetown University Press.Search in Google Scholar

Jewitt, Carey. 2008. Multimodality and literacy in school classrooms. Review of Research in Education 32(1). 241–267. https://doi.org/10.3102/0091732x07310586.Search in Google Scholar

Johnston, Trevor. 2016. Auslan corpus annotation guidelines. Melbourne: Macquarie University (Sydney) & La Trobe University. Available at: https://media.auslan.org.au/attachments/Auslan_Corpus_Annotation_Guidelines_November2016.pdf (accessed 25 October 2019).Search in Google Scholar

Kendon, Adam. 2014. Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society of London 369. 20130293. 1–13. https://doi.org/10.1098/rstb.2013.0293.Search in Google Scholar

Kramsch, Claire (ed.). 2002. Language acquisition and language socialization: Ecological perspectives. London: Continuum.Search in Google Scholar

Kress, Gunther & Theo van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. Oxford: Oxford University Press.Search in Google Scholar

Kusters, Annelies. 2017. Gesture-based customer interactions: Deaf and hearing Mumbaikars’ multimodal and metrolingual practices. International Journal of Multilingualism 14(3). 283–302. https://doi.org/10.1080/14790718.2017.1315811.Search in Google Scholar

Laakso, Minna. 2011. Kun sanat eivät riitä kertomaan: Eleet afasiakeskusteluissa ja lasten vuorovaikutustilanteissa [when words are not enough to tell: Gestures in aphasia conversation and children’s interaction]. In Pentti Haddington & Leila Kääntä (eds.), Kieli, keho ja vuorovaikutus. Multimodaalinen näkökulma sosiaaliseen toimintaan, 152–173. Helsinki: Suomalaisen Kirjallisuuden Seura.Search in Google Scholar

Leskelä, Leealaura & Camilla Lindholm. 2012. Haavoittuva keskustelu: Keskustelunanalyyttisia tutkimuksia kielellisesti epäsymmetrisestä vuorovaikutuksesta [Vulnerable conversation: Conversation analytic studies of linguistically asymmetric interaction]. Helsinki: Kehitysvammaliitto.Search in Google Scholar

Liddell, Scott K. 2003. Grammar, gesture, and meaning in American sign language. Cambridge: Cambridge University Press.10.1017/CBO9780511615054Search in Google Scholar

McKee, Rachel Locker & Jemina Napier. 2002. Interpreting into international sign pidgin: An analysis. Sign Language and Linguistics 5(1). 27–54. https://doi.org/10.1075/sll.5.1.04mck.Search in Google Scholar

McNeill, David. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Search in Google Scholar

Messina Dahlberg, Giulia & Sangeeta Bagga-Gupta. 2019. On the quest to “go beyond” a bounded view of language. Research in the intersections of the educational sciences, language studies and deaf studies domains 1997–2018. Deafness and Educational International 21(2-3). 74–98. https://doi.org/10.1080/14643154.2018.1561782.Search in Google Scholar

Mondada, Lorenza. 2014. The local constitution of multimodal resources in social interaction. Journal of Pragmatics 65. 137–156. https://doi.org/10.1016/j.pragma.2014.04.004.Search in Google Scholar

Mondada, Lorenza. 2019. Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics 145. 47–62. https://doi.org/10.1016/j.pragma.2019.01.016.Search in Google Scholar

Määttä, Simo K. 2015. Interpreting the discourse of reporting: The case of screening interviews with asylum seekers and police interviews in Finland. Translation & Interpreting 7(3). 21–35.Search in Google Scholar

Norris, Sigrid. 2004. Analyzing multimodal interaction: A methodological framework. London: Routledge.10.4324/9780203379493Search in Google Scholar

Ortega, Lourdes. 2019. SLA and the study of equitable multilingualism. The Modern Language Journal 103. 23–38. https://doi.org/10.1111/modl.12525.Search in Google Scholar

Paananen, Jenny. 2015. Kuinka eleet helpottavat yhteisymmärrystä? Ikoniset ja deiktiset eleet monikulttuurisilla lääkärin vastaanotolla. [How does gesturing facilitate mutual understanding? Iconic and deictic gestures in multicultural general practice consultations]. Puhe ja Kieli 35(2). 73–95.Search in Google Scholar

Puumala, Eeva, Riitta Ylikomi & Hanna-Leena Ristimäki. 2017. Giving an account of persecution: The dynamic formation of asylum narratives. Journal of Refugee Studies, 31(2). 197–215. https://doi.org/10.1093/jrs/fex024.Search in Google Scholar

Puupponen, Anna, Tuija Wainio, Birgitta Burger & Tommi Jantunen. 2015. Head movements in Finnish Sign Language on the basis of motion capture data: A study of the form and function of nods, nodding, head thrusts, and head pulls. Sign Language and Linguistics 18. 41–89. https://doi.org/10.1075/sll.18.1.02puu.Search in Google Scholar

Rosenstock, Rachel & Jemina Napier. 2016. International sign: Linguistic, usage, and status issues. Washington DC: Gallaudet University Press.Search in Google Scholar

Sacks, Harvey, Emanuel A. Schegloff & Gail Jefferson. 1974. A simplest systematic for the organization of turn-taking for conversation. Language 50(4). 696–735. https://doi.org/10.2307/412243.Search in Google Scholar

Schegloff, Emanuel A., Gail Jefferson & Harvey, Sacks. 1977. The preference for self-correction in the organization of repair in conversation. Language 53(2). 361–382. https://doi.org/10.2307/413107.Search in Google Scholar

Shaw, Sara, Fiona Copland & Julia Snell. 2015. An introduction to linguistic ethnography: Interdisciplinary explorations. In Julia Snell, Sara Shaw & Fiona Copland (eds.), Linguistic ethnography: Interdisciplinary explorations, 1–13. Basingstoke: Palgrave Macmillan.10.1057/9781137035035_1Search in Google Scholar

Sivunen, Nina. 2019. An ethnographic study of deaf refugees seeking asylum in Finland. Societies 9(1). 2.10.3390/soc9010002Search in Google Scholar

Stevanovic, Melisa. 2016. Sosiaaliset rakenteet [Social structures]. In Melisa Stevanovic & Camilla Lindholm (eds.), Keskustelunanalyysi: Kuinka tutkia sosiaalista toimintaa ja vuorovaikutusta [Conversation analysis: How to study social action and interaction], 200–221. Tampere: Vastapaino.Search in Google Scholar

Streeck, Jürgen, Charles Goodwin & Curtis LeBaron. 2011. Embodied interaction in the material world: An introduction. In Jürgen Streeck, Charles Goodwin & Curtis, LeBaron (eds.), Embodied interaction: Language and body in the material world, 1–26. Cambridge: Cambridge University Press.Search in Google Scholar

Tapio, Elina. 2013. A nexus analysis of English in the everyday life of FinSL signers: A multimodal view on interaction. Oulu: University of Oulu dissertation.Search in Google Scholar

Tapio, Elina. 2014. The marginalisation of finely tuned semiotic practices and misunderstandings in relation to (signed) languages and deafness. Multimodal Communication 3(2). 131.10.1515/mc-2014-0010Search in Google Scholar

Tapio, Elina. 2019. Developing picture communication for interactional situations at the beginning of the asylum process; mapping interactional practices. Apples Journal of Applied Language Studies 13(3). 131–138.10.17011/apples/urn.201910224569Search in Google Scholar

Whynot, Lori. 2016. Understanding international sign. A sociolinguistic study. Washington, DC: Gallaudet University Press.Search in Google Scholar

Published Online: 2020-09-21

© 2020 Walter de Gruyter GmbH, Berlin/Boston