Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Cognitive Linguistics

Editor-in-Chief: Newman, John

4 Issues per year


IMPACT FACTOR 2016: 2.135

CiteScore 2016: 1.29

SCImago Journal Rank (SJR) 2016: 1.247
Source Normalized Impact per Paper (SNIP) 2016: 1.485

Online
ISSN
1613-3641
See all formats and pricing
More options …
Volume 28, Issue 3

Issues

Experiencing and construing spatial artifacts from within: Simulated artifact immersion as a multimodal viewpoint strategy

Irene Mittelberg
Published Online: 2017-08-08 | DOI: https://doi.org/10.1515/cog-2016-0124

Abstract

Viewpoint has been shown to be a powerful construal mechanism in multimodal spoken and signed discourse, as well as in various other modalities and genres. This paper investigates embodied viewpoint strategies that have been observed when speakers combine speech, gestures, postures, gaze, and simulated action to describe their interaction with spatial artifacts such as gallery buildings, virtual architectural models, and paintings. Simulated artifact immersion is introduced as a multimodal viewpoint strategy whereby speakers submerge into their mental representation of an artifact by perceiving and experiencing it from an internal vantage point. It is argued that this viewpoint strategy tends to be employed when there is no narrative structure for the speakers to fall back on. The paper’s aim is twofold: (a) to show that when speakers talk about their own experiences with spatial artifacts, distinguishing between immersed and non-immersed experiential viewpoint strategies may be more fitting than distinguishing between character and observer viewpoint; and (b) to discuss how considering the interaction of iconic, indexical, and metonymic principles in gesture may elucidate viewpoint phenomena in general.

Keywords: viewpoint; gesture; indexicality; metonymy; immersion

References

  • Arnheim, R. 1969. Visual thinking. Berkeley, CA: University of California Press.Google Scholar

  • Bergen, Benjamin K. 2005. Mental simulation in literal and figurative language understanding. In Seana Coulson & Barbara Lewandowska-Tomaszczyk (eds.), The literal and the nonliteral in language and thought, 255–278. Frankfurt a. M: Peter Lang.Google Scholar

  • Bergen, Benjamin K. 2012. Louder than words: The new science of how the mind makes meaning. New York: Basic Books.Google Scholar

  • Bouvet, Danielle. 1997. Le corps et la métaphore dans les langues gestuelles: A la recherche des modes de production des signes. Paris: L’Harmattan.Google Scholar

  • Bredekamp, Horst. 2010. Theorie des Bildakts. Frankfurt a.m.: Suhrkamp.Google Scholar

  • Brône, Geert, Bert Oben, Annelies Jehoul, Jelena Vranjes & Kurt Feyaerts. 2017. Eye gaze and viewpoint in multimodal interaction management. DOI:.Crossref

  • Calbris, Geneviève. 1990. The semiotics of French gestures. Cambridge: Cambridge University Press.Google Scholar

  • Cienki, Alan. 2013. Cognitive linguistics: Spoken language and gesture as expressions of conceptualization. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.1), 182–201. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Cienki, Alan. 2015. Spoken language usage events. Language and Cognition 7(4). 499–514.Google Scholar

  • Cienki, Alan & Irene Mittelberg. 2013. Creativity in the forms and functions of gestures with speech. In Tony Veale, Kurt Feyaerts & Charles Forceville (eds.), Creativity and the agile mind: A multi-disciplinary study of a multi-faceted phenomenon, 231–252. Berlin & New York: Mouton de Gruyter.Google Scholar

  • Clark, Herbert H. 2016. Depicting as a method of communication. Psychological Review 123(3). 324–347.Google Scholar

  • Coulson, Seana. 2001. Semantic leaps. Cambridge: Cambridge University Press.Google Scholar

  • Croft, William & Alan Cruse. 2004. Cognitive linguistics. Cambridge: Cambridge University Press.Google Scholar

  • Dancygier, Barbara. 2012. The language of stories: A cognitive approach. Cambridge: Cambridge University Press.Google Scholar

  • Dancygier, Barbara & Eve Sweetser (eds.) 2012. Viewpoint in language: A multimodal perspective. Cambridge: Cambridge University Press.Google Scholar

  • Dancygier, Barbara & Eve Sweetser (eds.) 2014. Figurative language. Cambridge: Cambridge University Press.Google Scholar

  • Dancygier, Barbara, Lu Wei-Lun & Arie Verhagen (eds.) 2016. Viewpoint and the fabric of meaning: Form and use of viewpoint tools across languages and modalities. Berlin: Mouton de Gruyter.Google Scholar

  • Debras, Camille. 2013. Multimodal stancetaking in a videotaped corpus of discussions about the environment in British English. Paris: Sorbonne Nouvelle – Paris 3 University dissertation.Google Scholar

  • Deliema, David & Eve Sweetser. 2016. Rethinking gestural viewpoint as multidimensional rather than a dichotomy. Conference talk at the International Society for Gesture Studies Conference (ISGS7), Paris, France, 18–22 July.Google Scholar

  • Dudis, Paul. 2004. Body partitioning and real-space blends. Cognitive Linguistics 15(2). 223–238.Google Scholar

  • Duncan, Susan D., Justine Cassell & Elena T. Levy (eds.) 2007. Gesture and the dynamic dimension of language. Amsterdam & Philadelphia: John Benjamins.Google Scholar

  • Ehmer, Oliver. 2011. Imagination und Animation. Die Herstellung mentaler Räume durch animierte Rede. Berlin & New York: Mouton de Gruyter.Google Scholar

  • Emmorey, Karen, Barbara Tversky & Holly A. Taylor. 2001. Using space to describe space: Perspective in speech, sign, and gesture. Spatial Cognition and Computation 2. 157–180.Google Scholar

  • Fauconnier, Gilles & Mark Turner. 2002. The way we think. New York: Basic Books.Google Scholar

  • Fillmore, Charles J. 1982. Frame semantics. In The Linguistic Society of Korea (eds.), Linguistics in the morning calm, 111–137. Seoul: Hanshin.Google Scholar

  • Gibbs, Raymond W., Jr. 2006. Embodiment and cognitive science. Cambridge: Cambridge University Press.Google Scholar

  • Gibbs, Raymond W., Jr. & Teenie Matlock. 2008. Metaphor, imagination, and simulation: Psycholinguistic evidence. In Raymond W. Gibbs Jr. (ed.), The Cambridge handbook of metaphor and thought, 161–176. Cambridge: Cambridge University Press.Google Scholar

  • Hostetter, Autumn B. & Martha Alibali. 2008. Visible embodiment: Gestures as simulated action. Psychonomic Bulletin and Review 15(3). 495–514.Google Scholar

  • Hutchins, Edwin. 2005. Material anchors for conceptual blends. Journal of Pragmatics 37. 1555–1577.CrossrefGoogle Scholar

  • Jakobson, Roman. 1971. Shifters, verbal categories, and the Russian verb. In Roman Jakobson (ed.), Selected wrtings, Vol. II: Words and language, 130–147. The Hague & Paris: Mouton.Google Scholar

  • Jakobson, Roman & Krystyna Pomorska. 1983. Dialogues. Cambridge, MA: MIT Press.Google Scholar

  • Janzen, Terry. 2012. Two ways of conceptualizing space: Motivating the use of static and rotated vantage point space in ASL. In Barbara Dancygier & Eve E. Sweetser (eds.), Viewpoint in language: A multimodal perspective, 156–176. Cambridge: Cambridge University Press.Google Scholar

  • Johnson, Mark. 1987. The body in the mind: The bodily basis of meaning, imagination and reasoning. Chicago: Chicago University Press.Google Scholar

  • Johnson, Mark. 2005. The philosophical significance of image schemas. In Beate Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 15–33. Berlin: Mouton de Gruyter.Google Scholar

  • Johnson, Mark. 2007. The meaning of the body: Aesthetics of human understanding. Chicago: University of Chicago Press.Google Scholar

  • Krois, John M. 2011. Körperbilder und Bildschemata. Aufsätze zur Verkörperungstheorie ikonischer Formen (Actus et imago 2), hrsg. v. Horst Bredekamp & Marion Lauschke. Berlin: Akademie Verlag.Google Scholar

  • Langacker, Ronald W. 1987. Foundations of cognitive grammar, Vol.1, Theoretical perspectives. Stanford: Stanford University Press.Google Scholar

  • LeBaron, Curtis & Jürgen Streeck. 2000. Gestures, knowledge and the world. In David McNeill (ed.), Language and gesture, 118–138. Cambridge: Cambridge University Press.Google Scholar

  • Liddell, Scott. 2003. Grammar, gesture and meaning in American Sign Language. Cambridge: Cambridge University Press.Google Scholar

  • MacWhinney, Brian. 2005. The emergence of grammar from perspective. In Diane Pecher & Rolf Zwaan (eds.), Grounding cognition, 198–223. Cambridge: Cambridge University Press.Google Scholar

  • Marghetis, Tyler & Benjamin K. Bergen. 2014. Embodied meaning inside and out: The coupling of gestures and mental simulation. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.2), 2000–2007. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Matlock, Teenie. 2004. Fictive motion as cognitive simulation. Memory & Cognition 32. 1389–1400.CrossrefGoogle Scholar

  • Matlock, Teenie. 2006. Depicting fictive motion in drawings. In June Luchjenbroers (ed.), Cognitive linguistic investigations: Across languages, fields, and philosophical boundaries, 67–85. Amsterdam & Philadelphia: John Benjamins.Google Scholar

  • Matlock, Teenie, David Sparks, Justin L. Matthews, Jeremy Hunter & Stephanie Huette. 2012. Smashing new results on aspectual framing: How people talk about car accidents. Studies in Language 36(3). 699–720.Google Scholar

  • McNeill, David. 1992. Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.Google Scholar

  • McNeill, David (ed.) 2000. Language and gesture. Cambridge: Cambridge University Press.Google Scholar

  • Mittelberg, Irene. 2006. Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar. Ann Arbor, MI: UMI Cornell University dissertation.Google Scholar

  • Mittelberg, Irene. 2013a. The exbodied mind: Cognitive-semiotic principles as motivating forces in gesture. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.1), 750–779. Berlin & Boston: de Gruyter Mouton.Google Scholar

  • Mittelberg, Irene. 2013b. Balancing acts: Image schemas and force dynamics as experiential essence in pictures by Paul Klee and their gestural enactments. In Mike Borkent, Barbara Dancygier & Jennifer Hinnell (eds.), Language and the creative mind, 325–346. Stanford: CSLI Publications.Google Scholar

  • Mittelberg, Irene. 2014. Gestures and iconicity. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.2), 1712–1732. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Mittelberg, Irene. 2017. Embodied frames, metonymy and pragmatic inferencing in gesture. Gesture 16(2).Google Scholar

  • Mittelberg, Irene, Thomas H. Schmitz & Hannah Groninger. 2017. Operative manufacts: Gestures as embodied sketches in early stages of the design process. In Sabine Ammon & Remei Capdevila Werning (eds.), The active image: Architecture and engineering in the age of modeling, 99–131. Berlin: Springer.Google Scholar

  • Mittelberg, Irene & Linda R. Waugh. 2009. Metonymy first, metaphor second: A cognitive-semiotic approach to multimodal figures of thought in co-speech gesture. In Charles Forceville and Eduardo Urios-Aparisi (eds.), Multimodal metaphor, 329–356. Berlin & New York: Mouton de Gruyter.Google Scholar

  • Mittelberg, Irene & Linda R. Waugh. 2014. Gestures and metonymy. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.2), 1747–1766. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Müller, Cornelia. 1998. Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag Spitz.Google Scholar

  • Müller, Cornelia. 2014. Gestural modes of representation as techniques of depiction. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Jana Bressem (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.2), 1687–1702. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Müller, Cornelia & Silva H. Ladewig. 2013. Metaphors for sensorimotor experiences: Gestures as embodied and dynamic conceptualizations of balance in dance lessons. In Mike Borkent, Barbara Dancygier & Jennifer Hinnell (eds.), Language and the creative mind, 295–324. Stanford: CSLI Publications.Google Scholar

  • Murphy, Keith M. 2005. Collaborative imagining: The interactive use of gestures, talk, and graphic representation in architectural practice. Semiotica 156. 113–145.Google Scholar

  • Parrill, Fey. 2009. Dual viewpoint gestures. Gesture 9(3). 271–289.CrossrefGoogle Scholar

  • Parrill, Fey. 2010. Viewpoint in speech–gesture integration: Linguistic structure, discourse structure, and event structure. Language and Cognitive Processes 25(5). 650–668.Google Scholar

  • Parrill, Fey. 2012. Interactions between discourse status and viewpoint in co-speech gesture. In Barbara Dancygier & Eve E. Sweetser (eds.), Viewpoint in language: A multimodal perspective, 97–112. Cambridge: Cambridge University Press.Google Scholar

  • Peirce, Charles S. 1960. In Charles Hartshorne & Weiss. Paul (eds.), Collected papers of Charles Sanders Peirce (1931–1958). Vol. I.: Principles of philosophy, Vol. II: Elements of logic. Cambridge: The Belknap Press of Harvard University Press.Google Scholar

  • Perlman, Markus & Raymond W. Gibbs Jr. 2013. Sensorimotor simulation in speaking, gesturing, and understanding. In Cornelia Müller, Alan Cienki, Ellen Fricke, Silva H. Ladewig, David McNeill & Sedinha Teßendorf (eds.), Body – Language – Communication. An international handbook on multimodality in human interaction (Handbooks of Linguistics and Communication Science 38.1), 512–533. Berlin & Boston: De Gruyter Mouton.Google Scholar

  • Saussure, Ferdinand de. 1986. In par Charles Bally, Albert Sechehaye & T. De Mauro. (eds.), Cours de linguistique générale de Ferndinand de Saussure. Paris: Payot.Google Scholar

  • Schoonjans, Steven. 2014. Modalpartikel als multimodale Konstruktionen. Eine korpusbasierte Kookkurrenzanalyse von Modalpartikeln und Gestik im Deutschen. Leuven: KU Leuven unpublished dissertation.Google Scholar

  • Stec, Kashmiri. 2012. Meaningful shifts: A review of viewpoint markers in co-speech gesture and sign language. Gesture 12(3). 327–360.CrossrefGoogle Scholar

  • Streeck, Jürgen. 2009. Gesturecraft: The manu-facture of meaning. Amsterdam & Philadelphia: John Benjamins.Google Scholar

  • Sweetser, Eve. 2007. Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics. In Monica Gonzalez-Marquez, Irene Mittelberg, Seana Coulson & Michael Spivey (eds.), Methods in cognitive linguistics, 201–224. Amsterdam & Philadelphia: John Benjamins.Google Scholar

  • Sweetser, Eve. 2012. Introduction: Viewpoint and perspective in language and gesture, from the ground down. In Barbara Dancygier & Eve E. Sweetser (eds.), Viewpoint in language: A multimodal perspective, 1–22. Cambridge: Cambridge University Press.Google Scholar

  • Sweetser, Eve. 2013. Creativity across modalities in viewpoint construction. In Mike Borkent, Barbara Dancygier & Jennifer Hinnell (eds.), Language and the creative mind, 239–254. Stanford, CA: CSLI Publications.Google Scholar

  • Sweetser, Eve & Kashmiri Stec. 2016. Maintaining multiple viewpoints with gaze. In Barbara Dancygier, Lu Wei-Lun & Arie Verhagen (eds.), Viewpoint and the fabric of meaning: Form and use of viewpoint tools across languages and modalities, 237–257. Berlin: Mouton de Gruyter.Google Scholar

  • Talmy, Leonard. 2000. Towards a cognitive semantics. Cambridge, MA: MIT Press.Google Scholar

  • Taub, Sarah. 2001. Language from the body. Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press.Google Scholar

  • Turner, Mark (ed.) 2006. The artful mind. Oxford: Oxford University Press.Google Scholar

  • Vandelanotte, Lieven. 2009. Speech and thought representation in English: A cognitive-functional approach. Berlin: Mouton de Gruyter.Google Scholar

  • Verhagen, Arie. 2007. Construal and perspectivation. In Dirk Geeraerts & Hubert Cuyckens (eds.), The Oxford handbook of cognitive linguistics, 48–81. Oxford: Oxford University Press.Google Scholar

About the article

Received: 2016-10-25

Accepted: 2017-04-15

Revised: 2017-03-30

Published Online: 2017-08-08

Published in Print: 2017-08-28


Excellence Initiative of the German Federal and State Governments.


Citation Information: Cognitive Linguistics, Volume 28, Issue 3, Pages 381–415, ISSN (Online) 1613-3641, ISSN (Print) 0936-5907, DOI: https://doi.org/10.1515/cog-2016-0124.

Export Citation

© 2017 Walter de Gruyter GmbH, Berlin/Boston. Copyright Clearance Center

Comments (0)

Please log in or register to comment.
Log in