Published by De Gruyter Mouton

Volume 28 Issue 3 - Special Issue: Viewpoint Phenomena in Multimodal Communication, Guest Editors: Barbara Dancygier and Lieven Vandelanotte

Issue of Cognitive Linguistics

Submit manuscript

Contents
Journal Overview

Publicly Available August 9, 2017

Frontmatter

Page range: i-v

Download PDF

Research Articles

July 27, 2017

Viewpoint phenomena in multimodal communication

Barbara Dancygier, Lieven Vandelanotte

Page range: 371-380

Abstract

In this introduction to the special issue on viewpoint phenomena in multimodal communication, we highlight central questions concerning the nature of multimodality and of conceptual viewpoint, which the issue as a whole expands and clarifies. We argue that multimodality needs to be rethought as a varied but cohesive phenomenon, and we briefly illustrate both embodied multimodal interaction (in an example from stand-up comedy) and the meaning emergence in artifacts relying on both text and image (in an example of a poster with an environmental message). Correspondingly, the category of multimodal constructions already recognized for embodied interactions should be expanded to cover conventionalized image/text combinations. Finally, we stress that viewpoint is the pivotal concept that elucidates how communicators use the various modalities for cohesive communicative purposes across the wide range of artifacts and multimodal forms discussed in the special issue, including gesture in political speeches, viewpoint in comics, grammatical forms in Internet memes, ASL, stance expressions, eye gaze, and embodied responses to objects and architectural artifacts.

August 8, 2017

Experiencing and construing spatial artifacts from within: Simulated artifact immersion as a multimodal viewpoint strategy

Irene Mittelberg

Page range: 381-415

Abstract

Viewpoint has been shown to be a powerful construal mechanism in multimodal spoken and signed discourse, as well as in various other modalities and genres. This paper investigates embodied viewpoint strategies that have been observed when speakers combine speech, gestures, postures, gaze, and simulated action to describe their interaction with spatial artifacts such as gallery buildings, virtual architectural models, and paintings. Simulated artifact immersion is introduced as a multimodal viewpoint strategy whereby speakers submerge into their mental representation of an artifact by perceiving and experiencing it from an internal vantage point. It is argued that this viewpoint strategy tends to be employed when there is no narrative structure for the speakers to fall back on. The paper’s aim is twofold: (a) to show that when speakers talk about their own experiences with spatial artifacts, distinguishing between immersed and non-immersed experiential viewpoint strategies may be more fitting than distinguishing between character and observer viewpoint; and (b) to discuss how considering the interaction of iconic, indexical, and metonymic principles in gesture may elucidate viewpoint phenomena in general.

August 4, 2017

How politicians express different viewpoints in gesture and speech simultaneously

Douglas Guilbeault

Page range: 417-447

Abstract

Political speeches are a prime example of how discourse often requires speakers to convey multiple competing viewpoints, both their own and others’. Cognitive linguists have shown how, in speech, speakers express viewpoint through individual choices at the lexical and grammatical level. Recently, cognitive linguists have also shown that speakers express viewpoint using speech-accompanying gestures. To date, the study of viewpoint expression has focused on cases where speakers deliver the same viewpoint across modalities. By examining the persuasive uses of gesture in Obama’s A More Perfect Union speech, I show how speakers can communicate multiple different viewpoints across gesture and speech, simultaneously. There are moments when Obama expresses his opponents’ viewpoint in speech, while framing them in terms of his own viewpoint in gesture, and vice versa. I discuss how the deviation of viewpoints across modalities provides key insights into multimodal cognition, with respect to working memory, metaphor, and persuasion. Specifically, I argue that, as an implicit medium, gesture allows speakers to inject viewpoint into the uptake of speech, below the conscious radar of recipients, and I discuss how this rhetorical capacity is evolving as a result of communication technologies.

July 5, 2017

Eye gaze and viewpoint in multimodal interaction management

Geert Brône, Bert Oben, Annelies Jehoul, Jelena Vranjes, Kurt Feyaerts

Page range: 449-483

Abstract

In this paper, we present an embodiment perspective on viewpoint by exploring the role of eye gaze in face-to-face conversation, in relation to and interaction with other expressive modalities. More specifically, we look into gaze patterns, as well as gaze synchronization with speech, as instruments in the negotiation of participant roles in interaction. In order to obtain fine-grained information on the different modalities under scrutiny, we used the InSight Interaction Corpus (Brône, Geert & Bert Oben. 2015. Insight Interaction: A multimodal and multifocal dialogue corpus. Language Resources and Evaluation 49, 195–214.). This multimodal video corpus consists of two- and three-party interactions (in Dutch), with head-mounted scene cameras and eye-trackers tracking all participants’ visual behavior, providing a unique ‘speaker-internal’ perspective on the conversation. The analysis of interactional sequences from the corpus (dyads and triads) reveals specific patterns of gaze distribution related to the temporal organization of viewpoint in dialogue. Different dialogue acts typically display specific gaze events at crucial points in time, as, e.g., in the case of brief gaze aversion associated with turn-holding, and shared gaze between interlocutors at the critical point of turn-taking. In addition, the data show a strong correlation and temporal synchronization between eye gaze and speech in the realization of specific dialogue acts, as shown by means of a series of cross-recurrence analyses for specific turn-holding mechanisms (e.g., verbal fillers co-occurring with brief moments of gaze aversion).

July 14, 2017

Alignment and empathy as viewpoint phenomena: The case of amplifiers and comical hypotheticals

Kurt Feyaerts, Bert Oben, Helmut Karl Lackner, Ilona Papousek

Page range: 485-509

Abstract

This contribution focuses on verbal amplifiers and comical hypotheticals in a corpus of face-to-face interactions. Both phenomena qualify as markers of a mental viewpoint expressing an (inter)subjective construal of a certain experience. Whereas amplifiers offer a straightforward view onto a speaker’s evaluative stance, comical hypotheticals provide an intersubjective account of a viewpoint construal. As part of their meaning, their use reveals a speaker’s assumption about the interlocutor willing to allow or participate in a particular type of interactional humor. Our research interest for these phenomena concerns their occurrence as well as their interactional alignment in terms of mimicry behavior. In order to capture the impact of both linguistic and psychological variables in the use of these items, we adopt a differentiated methodological approach, which allows to correlate findings from our corpus linguistic analysis with the values obtained for interpersonal difference variables. As our data consists of male dyads of which the participants never met before the beginning of their conversation, we expected to witness an increase, along with the growing familiarity among the interlocutors, in both the use and alignment of these viewpoint phenomena. Indeed, results show a clear increase in the use of both verbal amplifiers and comical hypotheticals over the course of the interaction and independently from the also observed overall increase of communicativeness. However, with respect to the alignment of both viewpoint phenomena, our study reveals a differentiated result. Participants aligned their use of verbal amplifiers with that of their partners over the course of the interaction, but they did not do so for comical hypotheticals. Yet, within the broader discussion of the experiment’s design, this unexpected result may still seem plausible with respect to our general hypothesis. Beyond the limits of this study, the set-up and results of our study nicely connect to recent research on empathy-related behavior in social neuroscience.

July 5, 2017

Composite utterances in a signed language: Topic constructions and perspective-taking in ASL

Terry Janzen

Page range: 511-538

Abstract

Composite utterances are utterances that are built from multiple signs of multiple types, meaning that in any conversational “move” speech, gestures, eye-gaze, intonation patterns, physical stance, etc. all participate in the utterance, and the meaning derived from it is constructed by the composite of these participant types. likewise considers utterances as multimodal ensembles. The present study investigates how the notion of composite utterance plays out in a signed language such as ASL. Articulated in the same modality as are gestures, the distinction between language and gesture has seemed less clear, leading some to ask whether signers even gesture at all and some to suggest that gestures and formal signed language are substantively different systems. On the other hand, others have posited a continuity approach to gesture and signed language especially in light of grammaticalization studies. Here I examine topic-comment constructions and perspectivized clauses in ASL through the lens of Enfield’s composite utterances proposal, looking at component parts and how they function to ground elements in the discourse and guide the interlocutor through the textual structure. I use Enfield’s conventional versus non-conventional type categories in examining lexical and prosodic elements in topic and perspective-taking constructions.

August 3, 2017

Mediated characters: Multimodal viewpoint construction in comics

Mike Borkent

Page range: 539-563

Abstract

I analyze multimodal viewpoint construction in comics to engage with how modalities function within the medium as a specific discourse context with distinct conventions and material qualities. I show how comics employ established storytelling practices with character, narrator, and narrative viewpoint levels, while building up and interweaving these through strategic uses of the modalities of the medium. I mobilize the cognitive theories of embodiment, domains, mental simulation, and mental space blending as an analytical framework. I examine the asynchronicity of viewpoint elements between modalities and their synthesis into composite character viewpoints in several examples. I show how modalities can be prioritized and their different qualities and functions strategically manipulated for viewpoint construal. These brief examples show the complexity inherent in multimodal communication and interpretation and the usefulness of encouraging the medium-specific and interdisciplinary analyses of cultural works from a cognitive linguistic perspective.

August 3, 2017

Internet memes as multimodal constructions

Barbara Dancygier, Lieven Vandelanotte

Page range: 565-598

Abstract

This paper considers a range of so-called image macro Internet memes and describes them as emerging multimodal constructions relying as much on image as on text, and apportioning roles to images much like constructional slots, for instance to fill in a subject role in a subjectless clause, or even to provide the main clause content to a textually given when -clause. In addition to existing or partially altered linguistic constructions, many examples also rely on specific top text/bottom text division of labor, and crucially depend on frame metonymy, with limited formal means quickly cueing richly detailed frames (for instance by using iconic images). The popularity of memes, forming series and cycles of iterations and remixes, and their role in establishing and maintaining discourse communities seems to be driven by a need to express and reconstrue viewpoints, often starting from ideas, affects or stereotypes assumed to be intersubjectively shared with viewers, whose responses they solicit. This paper argues that a proper description of Internet memes of the type considered requires a construction grammar approach, complemented by an understanding of viewpoint dynamics in terms of a Discourse Viewpoint Space regulating the network of spaces and viewpoints.

About this journal

.....

Call for Editor in Chief

Deadline: 15 September 2023

Cognitive Linguistics is the official journal of the International Cognitive Linguistics Association and the flagship journal of the discipline. The current Editor-in-Chief, Professor Dagmar Divjak, is stepping down in June 2024 after 6 years in the role and we are looking to appoint a new Editor-in-Chief to lead the journal and the editorial team.

Cognitive Linguistics is keen to increase the diversity on its editorial team. We welcome applications from cognitive linguists in any region of the world, provided that the prospective post-holder is fluent in English. We particularly welcome candidates who are invested in increasing equality, diversity, and inclusion, and in supporting Open Science.

Read the full Call for an Editor here.

.....

Open Access

As part of De Gruyter‘s Subscribe to Open program, we plan to transform Cognitive Linguistics to Gold Open Access in 2024. For details on S2O, please see here.

Objective
Cognitive Linguistics presents a forum for linguistic research of all kinds on the interaction between language and cognition. The journal focuses on language as an instrument for organizing, processing and conveying information. Cognitive Linguistics is a peer-reviewed journal of international scope and seeks to publish only works that represent a significant advancement to the theory or methods of cognitive linguistics, or that present an unknown or understudied phenomenon.

Topics

the structural characteristics of natural language categorization (such as prototypicality, cognitive models, metaphor, and imagery);
the functional principles of linguistic organization, as illustrated by iconicity;
the conceptual interface between syntax and semantics;
the experiential background of language-in-use, including the cultural background;
the relationship between language and thought, including matters of universality and language specificity

Article formats

Research Articles - these are substantive pieces presening original research, totalling between 8,000 and 10,000 words on first submission
Registered report - each year we have space for one registered report that tackles a core question in cognitive linguistics. See the registered reports guidelines for more information.
Review paper - each year we have space for one paper that reviews progress in a core area of cognitive linguistics.
Commentaries & Replies to Research Articles - with prior consent of the Editor-in-Chief only.
Special Issue proposals: We are currently committed for 2022; the next round of review is expected to take place in Autumn 2021, with publication (in print) expected early 2023.

Please contact cogling@degruyter.com with topics for a Review Paper, Commentary or Reply.
Note that we no longer accept Book Reviews.