Abstract
In this paper we explore the use of multimodal and multilingual semiotic resources in interactions between two deaf signing participants, a researcher and an asylum seeker. The focus is on the use of gaze and environmentally coupled gestures. Drawing on multimodal analysis and linguistic ethnography, we demonstrate how gaze and environmentally coupled gestures are effective semiotic resources for reaching mutual understanding. The study provides insight into the challenges and opportunities (deaf) asylum seekers, researchers, and employees of reception centres or the state may encounter because of the asymmetrical language competencies. Our concern is that such asymmetrical situations may be created and maintained by ignoring visual and embodied resources in interaction and, in the case of deaf asylum seekers, by unrealistic expectations towards conventionalized forms of international sign.
Acknowledgments
We want to express our thanks to Sari Pöyhönen, Mia Halonen, Lindsay Ferrara and Teppo Jakonen for their valuable comments and insights on previous versions of this article. We would also like to thank the anonymous reviewers whose detailed comments and suggestions helped to improve the manuscript. Nina Sivunen is grateful for the support from the Centre of Applied Language Studies (CALS) of the University of Jyväskylä.
-
Research funding: This research was funded by the University of Jyväskylä, Faculty of Humanities and Social Sciences, Humak University of Applied Sciences and by the EU Home Affairs Funds (SMDno:2016-1561) through the PICCORE project.
Appendix
- [
-
beginning of overlapping sign
- ]
-
end of overlapping sign
- (0.5)
-
pauses timed in tenths of a second (approximately)
- capitals
-
gloss for a manual sign and a gesture
- cd:”xx”
-
constructed dialogue
- pnt-2
-
pointing to another interlocutor
- pnt-right/left
-
lexical pointing direction
- pnt-table/door
-
pointing to an object in the environment
- tch-table
-
touch
- -h
-
hold
- hs
-
head shakings
- hn
-
head noddings
- shdr dwn
-
shoulders down
- xxxx
-
eyegaze direction, head and body activities, non-manual elements (e.g. raised eyebrows)
- fig. x
-
The location of the picture (figure x) in the partiture
References
Bateman, John, Janina Wildfeuer & Tuomo Hiippala. 2017. Multimodality: Foundations, research and analysis a problem-oriented introduction. Berlin: De Gruyter.10.1515/9783110479898Search in Google Scholar
Bezemer, Jeff & Gunther Kress. 2014. Touch: A resource for making meaning. Australian Journal of Language and Literacy 37(2). 77–85.Search in Google Scholar
Blommaert, Jan, James Collins & Stef Slembrouck. 2005. Spaces of multilingualism. Multilingualism and Diasporic Populations: Special Issue of Language and Communication. 25(2). 197–215. https://doi.org/10.1016/j.langcom.2005.05.002.Search in Google Scholar
Copland, Fiona & Angela Creese. 2015. Linguistic ethnography: Collecting, analysing and presenting data. London: Sage.10.4135/9781473910607Search in Google Scholar
Cormier, Kearsy, Sandra Smith & Zed Sevcikova-Sehyr. 2015. Rethinking constructed action. Sign Language and Linguistics 18(2). 167–204. https://doi.org/10.1075/sll.18.2.01cor.Search in Google Scholar
Crasborn, Onno & Anja Hiddinga. 2015. The paradox of international sign: The importance of deaf-hearing encounters for deaf-deaf communication across sign language borders. In Michele Friedner & Annelies Kusters (eds.), It’s a Small World: International deaf spaces and encounters, 59–69. Washington, DC: Gallaudet University Press.10.2307/j.ctv2rcnnjs.9Search in Google Scholar
Crasborn, Onno & Han Sloetjes. 2008. Enhanced ELAN functionality for sign language corpora. In: Proceedings of LREC 2008, sixth international conference on language resources and evaluation.Search in Google Scholar
Drew, Paul. 1997. ‘Open’ class repair initiators in response to sequential sources of troubles in conversation. Journal of Pragmatics 28(1). 69–101. https://doi.org/10.1016/s0378-2166(97)89759-7.Search in Google Scholar
Engberg-Pedersen, Elisabeth. 2013. From pointing to reference and predication: Pointing signs, eyegaze, and head and body orientation in Danish sign language. In Sotaro Kita (ed.), Pointing: Where language, culture, and cognition meet, 269–292. Mahwah, NJ: Erlbaum.Search in Google Scholar
Goodwin, Marjorie & Charles Goodwin. 1986. Gesture and co-participation in the activity of searching for a word. Semiotica 62(1-2). 51–75 https://doi.org/10.1515/semi.1986.62.1-2.51.Search in Google Scholar
Goodwin, Charles. 2000. Action and embodiment within situated human interaction. Journal of Pragmatics 32(10). 1489–1522 https://doi.org/10.1016/s0378-2166(99)00096-x.Search in Google Scholar
Goodwin, Charles. 2003. Pointing as situated practice. In Sotaro Kita (ed.), Pointing: Where language, culture and cognition meet, 217–241. Mahwah, NJ: Erlbaum.Search in Google Scholar
Goodwin, Charles. 2007. Environmentally coupled gestures. In Susan D. Duncan, Justine Cassel & Elena T. Levy (eds.), Gesture and the dynamic dimension of language, 195–212. Amsterdam: John Benjamins Publishing Company.10.1075/gs.1.18gooSearch in Google Scholar
Green, E. Mara. 2015. One language, or maybe two: Direct communication, understanding, and informal interpreting in international deaf encounters. In. Michele Friedner & Annelies Kusters (eds.), It’s a small world: International deaf spaces and encounters, 70–82. Washington, DC: Gallaudet University Press.10.2307/j.ctv2rcnnjs.10Search in Google Scholar
Groeber, Simone & Evelyne Pochon-Berger. 2014. Turns and turn-taking in sign language interaction: A study of turn-final holds. Journal of Pragmatics 65. 121–136. https://doi.org/10.1016/j.pragma.2013.08.012.Search in Google Scholar
Heritage, John & Steven Clayman. 2010. Talk in action: Interactions, identities, and institutions. Chichester: Wiley-Blackwell.10.1002/9781444318135Search in Google Scholar
Hiddinga, Anja & Onno Crasborn. 2011. Signed languages and globalization. Language in Society 40(4). 483–505 https://doi.org/10.1017/s0047404511000480.Search in Google Scholar
Hutchins, Edwin & Saeko Nomura. 2011. Collaborative construction of multimodal utterances. In Jürgen Streeck, Charles Goodwin & Curtis, LeBaron (eds.), Embodied interaction: Language and body in the material world, 29–42. Cambridge: Cambridge University Press.Search in Google Scholar
Jefferson, Gail. 2004. Glossary of transcript symbols with an introduction. In Gene H. Lerner (ed.), Conversation analysis. Studies from the first generation, 13–31. Amsterdam: John Benjamins.10.1075/pbns.125.02jefSearch in Google Scholar
Jewitt, Carey. 2004. Multimodality and new communication technologies. In Philip LeVine & Ron Scollon (eds.), Discourse & technology: Multimodal discourse analysis, 184–195. Washington, D.C.: Georgetown University Press.Search in Google Scholar
Jewitt, Carey. 2008. Multimodality and literacy in school classrooms. Review of Research in Education 32(1). 241–267. https://doi.org/10.3102/0091732x07310586.Search in Google Scholar
Johnston, Trevor. 2016. Auslan corpus annotation guidelines. Melbourne: Macquarie University (Sydney) & La Trobe University. Available at: https://media.auslan.org.au/attachments/Auslan_Corpus_Annotation_Guidelines_November2016.pdf (accessed 25 October 2019).Search in Google Scholar
Kendon, Adam. 2014. Semiotic diversity in utterance production and the concept of ‘language’. Philosophical Transactions of the Royal Society of London 369. 20130293. 1–13. https://doi.org/10.1098/rstb.2013.0293.Search in Google Scholar
Kramsch, Claire (ed.). 2002. Language acquisition and language socialization: Ecological perspectives. London: Continuum.Search in Google Scholar
Kress, Gunther & Theo van Leeuwen. 2001. Multimodal discourse: The modes and media of contemporary communication. Oxford: Oxford University Press.Search in Google Scholar
Kusters, Annelies. 2017. Gesture-based customer interactions: Deaf and hearing Mumbaikars’ multimodal and metrolingual practices. International Journal of Multilingualism 14(3). 283–302. https://doi.org/10.1080/14790718.2017.1315811.Search in Google Scholar
Laakso, Minna. 2011. Kun sanat eivät riitä kertomaan: Eleet afasiakeskusteluissa ja lasten vuorovaikutustilanteissa [when words are not enough to tell: Gestures in aphasia conversation and children’s interaction]. In Pentti Haddington & Leila Kääntä (eds.), Kieli, keho ja vuorovaikutus. Multimodaalinen näkökulma sosiaaliseen toimintaan, 152–173. Helsinki: Suomalaisen Kirjallisuuden Seura.Search in Google Scholar
Leskelä, Leealaura & Camilla Lindholm. 2012. Haavoittuva keskustelu: Keskustelunanalyyttisia tutkimuksia kielellisesti epäsymmetrisestä vuorovaikutuksesta [Vulnerable conversation: Conversation analytic studies of linguistically asymmetric interaction]. Helsinki: Kehitysvammaliitto.Search in Google Scholar
Liddell, Scott K. 2003. Grammar, gesture, and meaning in American sign language. Cambridge: Cambridge University Press.10.1017/CBO9780511615054Search in Google Scholar
McKee, Rachel Locker & Jemina Napier. 2002. Interpreting into international sign pidgin: An analysis. Sign Language and Linguistics 5(1). 27–54. https://doi.org/10.1075/sll.5.1.04mck.Search in Google Scholar
McNeill, David. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Search in Google Scholar
Messina Dahlberg, Giulia & Sangeeta Bagga-Gupta. 2019. On the quest to “go beyond” a bounded view of language. Research in the intersections of the educational sciences, language studies and deaf studies domains 1997–2018. Deafness and Educational International 21(2-3). 74–98. https://doi.org/10.1080/14643154.2018.1561782.Search in Google Scholar
Mondada, Lorenza. 2014. The local constitution of multimodal resources in social interaction. Journal of Pragmatics 65. 137–156. https://doi.org/10.1016/j.pragma.2014.04.004.Search in Google Scholar
Mondada, Lorenza. 2019. Contemporary issues in conversation analysis: Embodiment and materiality, multimodality and multisensoriality in social interaction. Journal of Pragmatics 145. 47–62. https://doi.org/10.1016/j.pragma.2019.01.016.Search in Google Scholar
Määttä, Simo K. 2015. Interpreting the discourse of reporting: The case of screening interviews with asylum seekers and police interviews in Finland. Translation & Interpreting 7(3). 21–35.Search in Google Scholar
Norris, Sigrid. 2004. Analyzing multimodal interaction: A methodological framework. London: Routledge.10.4324/9780203379493Search in Google Scholar
Ortega, Lourdes. 2019. SLA and the study of equitable multilingualism. The Modern Language Journal 103. 23–38. https://doi.org/10.1111/modl.12525.Search in Google Scholar
Paananen, Jenny. 2015. Kuinka eleet helpottavat yhteisymmärrystä? Ikoniset ja deiktiset eleet monikulttuurisilla lääkärin vastaanotolla. [How does gesturing facilitate mutual understanding? Iconic and deictic gestures in multicultural general practice consultations]. Puhe ja Kieli 35(2). 73–95.Search in Google Scholar
Puumala, Eeva, Riitta Ylikomi & Hanna-Leena Ristimäki. 2017. Giving an account of persecution: The dynamic formation of asylum narratives. Journal of Refugee Studies, 31(2). 197–215. https://doi.org/10.1093/jrs/fex024.Search in Google Scholar
Puupponen, Anna, Tuija Wainio, Birgitta Burger & Tommi Jantunen. 2015. Head movements in Finnish Sign Language on the basis of motion capture data: A study of the form and function of nods, nodding, head thrusts, and head pulls. Sign Language and Linguistics 18. 41–89. https://doi.org/10.1075/sll.18.1.02puu.Search in Google Scholar
Rosenstock, Rachel & Jemina Napier. 2016. International sign: Linguistic, usage, and status issues. Washington DC: Gallaudet University Press.10.2307/j.ctv2t5xgp9Search in Google Scholar
Sacks, Harvey, Emanuel A. Schegloff & Gail Jefferson. 1974. A simplest systematic for the organization of turn-taking for conversation. Language 50(4). 696–735. https://doi.org/10.2307/412243.Search in Google Scholar
Schegloff, Emanuel A., Gail Jefferson & Harvey, Sacks. 1977. The preference for self-correction in the organization of repair in conversation. Language 53(2). 361–382. https://doi.org/10.2307/413107.Search in Google Scholar
Shaw, Sara, Fiona Copland & Julia Snell. 2015. An introduction to linguistic ethnography: Interdisciplinary explorations. In Julia Snell, Sara Shaw & Fiona Copland (eds.), Linguistic ethnography: Interdisciplinary explorations, 1–13. Basingstoke: Palgrave Macmillan.10.1057/9781137035035_1Search in Google Scholar
Sivunen, Nina. 2019. An ethnographic study of deaf refugees seeking asylum in Finland. Societies 9(1). 2.10.3390/soc9010002Search in Google Scholar
Stevanovic, Melisa. 2016. Sosiaaliset rakenteet [Social structures]. In Melisa Stevanovic & Camilla Lindholm (eds.), Keskustelunanalyysi: Kuinka tutkia sosiaalista toimintaa ja vuorovaikutusta [Conversation analysis: How to study social action and interaction], 200–221. Tampere: Vastapaino.Search in Google Scholar
Streeck, Jürgen, Charles Goodwin & Curtis LeBaron. 2011. Embodied interaction in the material world: An introduction. In Jürgen Streeck, Charles Goodwin & Curtis, LeBaron (eds.), Embodied interaction: Language and body in the material world, 1–26. Cambridge: Cambridge University Press.Search in Google Scholar
Tapio, Elina. 2013. A nexus analysis of English in the everyday life of FinSL signers: A multimodal view on interaction. Oulu: University of Oulu dissertation.Search in Google Scholar
Tapio, Elina. 2014. The marginalisation of finely tuned semiotic practices and misunderstandings in relation to (signed) languages and deafness. Multimodal Communication 3(2). 131.10.1515/mc-2014-0010Search in Google Scholar
Tapio, Elina. 2019. Developing picture communication for interactional situations at the beginning of the asylum process; mapping interactional practices. Apples Journal of Applied Language Studies 13(3). 131–138.10.17011/apples/urn.201910224569Search in Google Scholar
Whynot, Lori. 2016. Understanding international sign. A sociolinguistic study. Washington, DC: Gallaudet University Press.10.2307/j.ctv2rr3fntSearch in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
