Embodied image schemas have been attributed a prominent role not only in structuring our perceptual experience and bodily being-in-the-world but also in motivating both literal and figurative expressions in language and other modalities (e.g., Gibbs 2006; Hampe 2005, 2017; Krois et al. 2007). Image schemas are generally understood as dynamic, embodied patterns of recurrent experiences such as visual perception, sensorimotor routines, and interactions with the physical and social world (e.g., Gibbs and Colston 1995; Johnson 1987, 2007; Lakoff 1987; Lakoff and Johnson 1999). According to Johnson (2007: 144–145).
Image schemas constitute a preverbal and mostly nonconscious, emergent level of meaning. […] Although they are preverbal, they play a major role in the syntax, semantics, and pragmatics of natural language. They lie at the heart of meaning, and they underlie language, abstract reasoning, and all forms of symbolic interaction.
The primary aim of this paper is to demonstrate some of the ways in which spontaneous gestures may be said to share some of the essential characteristics and semantic import of image schemas and force gestalts. Gestures may also, generally speaking, “bind body and mind” (ibid.: 145), not only in momentary instances of situated meaning-making but also across time, contexts, and individuals. More specifically, the present proposal starts from the observation that there seems to be a certain structural similarity between certain image and force schemas and certain gestural forms and movements, which partly seems to stem from their shared experiential bases, in particular, from bodily, object-oriented actions, interpersonal interaction, and movement through space (e.g., Mittelberg and Joue 2017). However, it should be pointed out right from the start that it is not maintained here that all gestures instantiate image schemas or force gestalts. Nor can one assume a one-to-one correspondence between such embodied schematic patterns and their gestural manifestations. In some cases, we might be able to see a certain phenomenological, structural similarity (e.g., Merleau-Ponty 1962; Mittelberg 2013a; 2013b). This similarity seems to reside in their common schematic nature, which may be said to retain essential structures and qualities of basic kinds of experience. When looking at “the phenomenology of the human body in action” (Gibbs 2005: 116) and notably, as it is done here, at gesture as a genuine phenomenon of human communicative behavior, a crucial factor is the degree to which we may perceive and produce gestures more consciously at certain moments of meaning-making than at others. In fact, it is precisely the less conscious aspects of gestures that make them a valuable source of insights into the less monitored functioning of cognition and language during communication (e.g., Sweetser 2007). Since gestures are essentially activations of the human body, they can be viewed as more immediate expressions than spoken or written discourse. Hence, they enable us to empirically study the origins and nature of the deeply embodied aspects of the human mind to which image schemas (and metaphor) clearly belong (e.g., Calbris 2008; Cienki 1998a, 1998b; McNeill 1992; Mittelberg 2010, 2013a; Müller 2008, 2017b; Streeck 2009; Sweetser 1998; Zlatev 2005; see Cienki 2013; Zlatev 2014 for recent overviews).
Gestures here are understood as discourse-embedded, kinetic actions that are performed with the head, hands, arms, torso, or entire body and have some communicative function(s) (e.g., Calbris 2011; Kendon 2004; McNeill 1992; Müller 1998). While gestures may consist of full-fledged, iconic imitations of human postures and actions, they are often rather schematic gestalts that metonymically evoke locally and/or prototypically salient aspects of a certain entity, scenario, action routine, or thought process (Mittelberg and Waugh 2014). Gestures typically consist of evanescent hand configurations, motion onsets, or virtual movement traces from which one may pragmatically infer, for instance, the idea of CONTAINMENT via a cupped hand, an OBJECT by seemingly holding it between two hands or pointing at it, or a PATH by tracing its progression in the air.1 Alternatively, a NEAR-FAR spatial relation may be established by locating a point close to one’s body and another one further away – that is, by exploiting the depth of gesture space as well as interpersonal gesture spaces (e.g., Calbris 2003, 2011; Cienki 1998b; 2005; Mittelberg 2006, 2008, 2010; Sweetser and Sizemore 2008; Williams 2008). In a similar fashion, force-dynamic gestalts (Talmy 1988) may be minimally enacted, or simulated (Gibbs 2005), via communicative postures and movements. For instance, subtle downward shifts in body posture may momentarily evoke the sensation of physical forces acting upon the speaker’s body, thus recruiting the BALANCE and GRAVITY schemas, which may metaphorically construe the emotional state of feeling down (e.g., Mittelberg 2013b). What exactly such typically polysemous articulations locally enact or portray is specified via the concurrent speech and other contextual factors (e.g., Calbris 2011; Kendon 2004; McNeill 1992, 2005; Müller 1998, 2010, 2017a).
The flexible correspondences between gestures and the image schemas of interest here seem to be anchored in, and regularly reinforced through, the mediating role that the human body plays not only in how human beings derive patterns from repeated multisensory experience, thus building up conceptual structures, but also in how they express or enact embodied semantic primitives (Lakoff 2006) and experiential gestalts (Gibbs 2005) when communicating via speech, gesture, and other modalities (Mittelberg 2008). Due to their strongly grounded nature, image and force schemas have already been shown to be enacted both on-the-fly and across similar gestural phenomena, thus engendering a certain regularity in gestural forms and functions within certain cultural communities or discourse contexts (e.g., Bressem and Müller 2014; Ladewig 2011; Mittelberg 2006, 2010; in press). It is thus assumed here that image and force schemas may build the structural essence of individual gestures as well as underpin the emergence of gestural patterns across speakers and contexts of use.
In line with Johnson (1987, 2005, 2007) and (Gibbs 2003, 2005, 2006), my intent is to present gestural evidence that image schemas and force gestalts are not “merely skeletons or abstractions” (Johnson 2005: 31) but should be seen instead as dynamic gestalts of experiential essence that not only operate in cognitive and linguistic processes but also in speakers’ pragmatically minded gestural actions. A central premise of the perspective taken here is that gestures may provide insights into the “semiotic reality” (Danaher 1998: 190; (Mittelberg 2006, 2008, 2013b) of pervasive embodied schemas. Semiotic reality here pertains to both cognitive and expressive dimensions of semiosis and is understood to encompass cognitive, linguistic, physical, and culturally shaped aspects of observable patterns of communicative action. It is further maintained that image schemas and force gestalts also partake in how gestures convey “felt qualities of our experience, meaning and thought” (Johnson 2005: 31). Producing and interpreting multimodal utterances always involves, at least to a certain degree, felt sensations of the subjective and affective qualities in one’s own articulations as well as in the communicative behavior of our conversational partners (e.g., Mittelberg 2013a; Müller 2017b).
With these tenets in mind, this paper presents a first attempt at a gesture-based, body-centered account of image schemas and force gestalts that may be observed to operate in gestural behavior. Inspired by Gibbs’ (2005) dynamic systems account of image schemas, such flexible and typically interacting tendencies in gestural image-schema enactments are seen as evolving broadly along the following lines: dynamic embodied patterns that motivate gestures a) from within the body and in response to forces acting on the body (body-inherent and acting on body); b) that simulate organism-environment interactions (environment-oriented); c) are inscribed in virtually pre-structured gesture spaces conceived of from the perspective of the speaker (pre-structuring gesture space); or d) function as organizing principles of communicative interactions and thus factor into achieving intersubjective understanding among conversational partners (interlocutor-oriented). To back up this proposal, I will draw on gestural examples taken from American-English and German multimodal discourse. Some of the recordings include motion-capture data that allow one to precisely measure the temporal and spatial dimensions of gestural articulations as well as to visualize the path, direction, and manner of movement excursions.
2 Inherently meaningful image schemas and force dynamics across modalities
Understanding communicative gestures as mediators between experiential grounding and conceptual schematicity naturally puts the focus on how multimodal communicative behavior is motivated and structured by embodied patterns stemming from habits of perception and organism-environment interactions. The present section serves to lay out some fundamental aspects of image schemas and force dynamics as well as to show how gesture, being a physical medium, may provide additional evidence for their embodied origins and functions. Since limitations of space do not allow for a comprehensive overview of the substantial, multidisciplinary body of research on image schemas (e.g., Gibbs 2006; Hampe 2005, 2017), special attention will be paid to those aspects that are particularly relevant with regard to communicative bodily actions in general and to the present proposal in particular.
The guiding idea is that such deeply embodied patterns are inherently meaningful (e.g., Gibbs 2005; Lakoff 2006) in the sense that they instill schematic meaning in certain gestural behaviors. This is assumed to function similarly to the way in which basic semantic tendencies reside in, for instance, certain grammatical structures (e.g., prepositional phrases expressing abstract spatial relations) or primary metaphors that draw on correlations of embodied experiences and subjective meaning (e.g., INTIMACY IS CLOSENESS, Grady 1997). In language, these basic patterns have been shown to factor into rich inferences and structures of higher complexity, thus underpinning more complex, semiotically and culturally embedded meaning (e.g., Dancygier and Sweetser 2014; Johnson 2007). In a similar vein, it is suggested here that when gestural behavior is motivated by embodied image schemas and/or force gestalts, its core consists of inherently meaningful structures. A central, underlying assumption is that this kind of semantic essence (or “immanent meaning” Johnson 2006) should emerge from the corresponding gestures to a certain degree, even without considering the speech content (e.g., Mittelberg in press; Wolf et al. 2017). While more empirical research is clearly needed to substantiate this claim, some indications to this effect will be presented throughout the paper.
2.1 Image schemas
According to Johnson’s (1987: xiv) original definition, image schemas are “recurring, dynamic patterns of our perceptual interactions and motor programs that give coherence and structure to our experience.” Basic embodied schemas include OBJECT, CONTAINER, PATH, BOUNDARY, LINK, CENTER-PERIPHERY, and BALANCE, as well as spatial schemas such as OVER, UNDER, and IN-OUT. Most of these have been shown to structure source domains underpinning conceptual metaphor, especially spatial and primary metaphors (e.g., Gibbs 1994, 2005; Grady 1997; Johnson 1987; Johnson 2007; Lakoff 1987; Lakoff and Johnson 1999; Langacker 1987; Mandler 1996; Mandler 2005; Sweetser 1998).
An example of a fundamental schema is the PATH schema, which consists of a beginning (SOURCE), a vector constituting a PATH, and an end point (GOAL). Figure 1 shows a static visual representation. This image-schematic structure may be instantiated across different domains of experience, for example, by traveling from one city to another, scoring a goal in a soccer match, climbing up a winding trail all the way to the top of a mountain, or gesturally drawing a diagrammatic itinerary of the same hiking experience in the air. Due to the schema’s internal structure, language users may profile one of the constitutive parts of a given event. For instance, in John left England, the beginning of the journey is focused upon; yet we may metonymically infer that the trip involved a PATH (i.e., the itinerary) and a GOAL (i.e., the destination or some other end point). In John traveled through the Chunnel to Paris, PATH and GOAL are profiled (Evans and Green 2006: 185).
Image-schematic patterns may also structure the understanding of domains that are more detached from physical experience, such as the passing of time, beliefs, mental states, and abstract thought, among others. The PATH schema also underpins, for instance, the metaphors LIFE IS A JOURNEY and LOVE IS A JOURNEY (Lakoff and Johnson 1980), thus projecting its internal structure, consisting of points of departure and arrival as well as a route linking them, onto the target domains LIFE or JOURNEY, respectively. Here the image schema provides the basis, or backbone, of more complex semantic structures, such as frames (Fillmore 1982); for instance, the related TRAVEL frame is mapped onto the LOVE frame and thus opens access to all the possible, implied, culturally shaped elements, roles, and processes (e.g., Dancygier and Sweetser 2014: 23–24).
The PATH schema has also been shown to lend structure to form constitution and meaning-making in visuo-spatial modalities and thus also in gesture (e.g., Forceville and Jeulik 2011; Forceville and Urios-Aparisi 2009; Hampe 2017). Consider the following gestural manifestation of the PATH schema that was spontaneously created and integrated in a multimodal discourse sequence in American English.2 When talking about her habit of watching a particular sitcom, the speaker in Figure 2 specifies a certain time period within a given season: “from the point of where I was till like the end of the season.” Here the PATH schema underpins the portrayal of a bounded temporal phase (BOUNDEDNESS) reaching out from one point (SOURCE) to a subsequent point in time (GOAL); in this way, the metaphor TIME IS SPACE is manifested as a HORIZONTAL fairly STRAIGHT movement extension evolving from LEFT to RIGHT (seen from the speaker’s perspective). Here, the gesturally evoked, spatial construal of the speaker’s viewing experience is quite specific as it profiles all three parts of the schema (which are also mentioned in the concurrent verbal utterance): the SOURCE (“from where I was”), the PATH through time (“till like”), and the GOAL (“the end of the season”). In doing so, she designates the point of departure with her left, almost vertically held, open hand and the end point with her open right hand, which is likewise held vertically, thus reinforcing the idea of a bounded space. Hence, the latter configuration can also be said to allude to the CONTAINER schema (see also Calbris 2008 on gesturally evoked timelines). This multimodal explanation evokes several embodied patterns in a particularly precise fashion. However, neither in gesture nor in other modalities are image schemas always so fully instantiated; rather, only certain parts and gestalt aspects may be metonymically profiled, thus alluding to the full schema or gestalt.
In one of the first studies on image schemas in gesture, (Cienki 1998a, 1998b) offers detailed discussions of different instantiations of the schemas STRAIGHT and PATH and their metaphoric extensions in language and gesture. In conversations about cheating during exams, North-American college students were found to conceptualize morally right behavior through their gestures, for instance, as movement along a STRAIGHT line. Instantiations of the PATH schema may generally include both linear and nonlinear variants. Additional insights come from a study on dynamic gestural representations of grammar and linguistic theory in teaching contexts (Mittelberg 2006). For example, alluding to a linear model of a sentence, a linguist drew a sketchy horizontal line in the air. An alternative technique evoked the PATH schema by lining up three points in gesture space that stood for subject, verb, and object respectively and thus built a simple syntactic structure with a beginning, an extension, and an end (Mittelberg 2008). When explaining the “nonlinearity” that is typically involved in second language acquisition processes, a linguist made a gestural movement evoking a nonlinear path (Mittelberg 2010). As Williams (2008) has observed, in the process of teaching children how to tell the time, the PATH schema may also be reflected in a teacher’s circling hand movements around a clock face that functions as a material anchor (Hutchins 2005). Núñez (2008) further stresses the central function that image schemas and fictive motion play in teaching mathematical concepts.3 Furthermore, Cienki (2005) has demonstrated that the PATH and other basic images schemas (e.g., OBJECT, CONTAINER, CYCLE, and FORCE) may also be used as descriptors in gesture interpretation. Another crucial observation is that gestures may provide interlocutors with semantic cues that are not necessarily identifiable in the concurrent speech.
2.2 Force gestalts
Force dynamics (Talmy 1988, 2000), too, assume a constitutive role in the way that we structure and understand experiences which tend to cut across physical, cognitive, social, emotional, kinesthetic, and aesthetic domains (e.g., Gibbs and Colston 1995; Johnson 1987, 2007; Krois 2011; Turner 2006). Dynamic patterns of this sort include sensing forces within one’s own body or acting on it, for instance, when being pushed sideward by the wind while walking (DIVERSION), trying to keep an upright posture while riding a bike (BALANCE), pushing a bike uphill (GRAVITY), or not being able to PUSH a heavy door open (RESISTANCE, BLOCKAGE). Johnson (1987: 42; italics in the original) reminds us that:
[…] we easily forget that our bodies are clusters of forces and that every event of which we are part consists, minimally, of forces in interaction […]. We do notice such forces when they are extraordinarily strong, or when they are not balanced off by other forces.
While the force-dynamic system is rooted in our bodily morphology, affordances, and experience, that is, kinesthesia (i.e., a sense of movement through muscular effort), somesthesia (sensing pressure or pain), and proprioception (i.e., a sense of the position of body parts and bodily movement in space, including a sense of gravitational orientation; Sheets-Johnstone 1999/2011: 73), it may also relate to psychological and social forces through metaphorical construal and certain grammatical constructions (e.g., Dancygier and Sweetser 2014; Evans and Green 2006: 199; Gibbs 2005: 119).
While force dynamics have not yet received as much attention as image schemas in gesture research (e.g., Cienki 2005), the spatiotemporal properties of gestures seem to offer great potential for gaining empirical insights into deeply embodied dimensions of force gestalts and how they interact with image-schematic structures. Furthermore, metaphorical extensions of force gestalts may not only portray social pressure or emotional burden but also drive gestural portrayals of the speaker’s subjective stance towards what s/he is talking about, for instance, by pushing away (BLOCKAGE, RESISTENCE) an idea put forth by an interlocutor (e.g., Bressem and Müller 2014; see also Calbris 2011 on the semantics of physical refusal via the recruitment of reflexes and Wehling 2017 on discourse management gestures).
In the following example (adapted from Mittelberg 2010), a linguistics professor is explaining the theory of emergent grammar in the context of second language acquisition. She holds that the boundaries between grammar, seen as a dynamic system, and language learning become fused due to discourse-pragmatic forces: “it sees the use and grammar together… grammar coming out of the use… you learn by doing rather than you learn first the rules and then you do them… [it blurs the boundary] between… learning and doing.” As shown in Figure 3, the speaker produces a relatively forceful gestural action consisting of suddenly pushing both hands, initially held more than shoulder-width apart, towards each other. Through the comparably high level of energy and dynamicity exhibited in this gestural portrayal, the ideas expressed in speech are vividly enacted and draw on the force and image schemas PUSH, MERGE, PROCESS, and BOUNDARY.
2.3 Mimetic schemas
While this paper focuses on how image and force schemas may be said to motivate and structure spontaneous coverbal gestures in adult speakers, a link needs to be made to mimetic schemas, as introduced by Zlatev (2005). Mimetic schemas – such as GRASP, JUMP, KICK, PUSH, and HIT – were originally defined as “dynamic, concrete and preverbal representations, involving the body image, accessible to consciousness and pre-reflectively shared in a community” (ibid.: 334). Crucially, mimetic schemas are “preverbal, body-based, representational, fairly specific and culturally shared” (Zlatev 2014: 2), and they “help explain most literally the ‘grounding’ of both communication and thought through action and imitation” (ibid.: 4). Exhibiting intermediate degrees of abstraction and schematicity, they underpin the communicative use of bodily actions and physical objects and how these communicative acts are understood. Being more specific and probably also enacted more consciously than image schemas, mimetic schemas are implicated in mimetic gestural actions made by children and play a constitutive role in cognitive and linguistic development (see also Andrén’s 2010 notion of “action gestalt” and “action-based gestures”).
Mimetic schemas may also be recruited in later developmental stages when young and adult speakers gesturally portray basic physical actions. They seem to interact with, or factor into, emerging image schemas (regarding language development, see also Mandler 1996; Mandler 2005; Mandler and Pagán Cánovas 2014). For instance, the PUSH force gestalt enacted in the gesture shown in Figure 3 can also be said to draw on the mimetic schema PUSH. Whether mimetic schemas may be projected metaphorically is an interesting aspect to take into consideration (see also Cienki 2013; Mittelberg and Joue 2017). In the case of the gesture portraying the idea of blurring boundaries between learning and doing grammar (Figure 3), the discourse context favors a metaphorical understanding of the forceful gestural action.4
2.4 Flexibly interconnecting schemas
Some scholars emphasize that image and force schemas tend to combine (e.g., Johnson 1987) to create blends (Krois 2011: 248) or form “situated, compound image schemas” (Kimmel 2005). Considering the multisensory bases of image schemas, such as vision, touch, hearing, and movement/balance, Evans and Green (2006: 186) describe them as malleable multimodal cognitive primitives that may cluster in networks of compatible schemas. Importantly, such manifestations of interconnected patterns may also incorporate force-dynamic dimensions such as COMPULSION, BLOCKAGE, or REMOVAL OF RESTRAINT and thus highlight certain profiling mechanisms. There is a contrast, for instance, between “the ball was rolling along the beach,” which is neutral in force-dynamic terms, and “the ball kept rolling along the beach” where the keep V-ing construction puts the focus on the fact that the ball seems to keep moving due to some external force acting on it (Evans and Green 2006: 199; Gärdenfors 2007; Talmy 2000: 412). Due to their deeply embodied motivation, such patterns allow us to assess situations we ourselves or other people experience, how objects behave in certain environments, and also to predict how they (could) evolve (e.g., metonymic cause-effect relations; see Panther and Radden 1999).
It should generally be noted that the terms image schema and force dynamics are not always clearly defined or kept apart in the literature. FORCE is often listed among image schemas as a superordinate term encompassing various force-dynamic phenomena, some of which have been discussed in this section. Since the substantial body of research on image schemas in the cognitive sciences encompasses a variety of views, many scholars have addressed the difficulty involved in developing a coherent theory or typology (see, e.g., the introduction and contributions in Hampe 2005; Oakley 2007).5 It apparently remains a challenge to systematize the multiple views and evidence stemming from various research strands in order to make sound generalizations, if that is at all possible. Claiming that gesture research may further this undertaking, the present proposal aims to contribute to a truly embodied, that is, body-based, view on inherently meaningful patterns of experience by showing how gesture tends to draw on different kinds of such experiential gestalts (Gibbs 2005). The theoretical grounds of this approach will be discussed in the next section.
3 Embodied schemas as emergent experiential gestalts recreating the kinesthetic feel and structure of embodied action
As emphasized in the forgoing discussion, the present view of spontaneously produced communicative action puts a strong focus on its experiential motivation and the constitutive role played by image schemas and force gestalts. Hence it is allied with theoretical accounts of image schemas that do not just assume a bodily basis of conceptual schemas (e.g., Hampe 2005; Oakley 2007), but claims that such patterns retain their experiential bodily origins to a great and perceivable extent. Johnson (2005: 31), for instance, advocates that:
[…] the truly significant work done by image schemas is tied to the fact that they are not merely skeletons or abstractions. They are recurring patterns of organism-environment interactions that exist in the felt qualities of our experience, understanding, and thought. Image schemas are the sort of structures that demarcate the basic contours of our experience as embodied creatures.6
Besides the dynamic dimensions of image schemas, Mandler and Pagán Cánovas (2014) have stressed the important role that language development plays in relation to preverbal schemas and spatial primitives.
Gibbs, in particular, favors a strongly embodied and action-based view of image schemas, seeing them as emergent “‘experiential gestalts’ or ‘as-if body loops’ (Damasio 1994) that are actively created on-the-fly during different cognitive activity” (Gibbs 2005: 119). Building on earlier work, notably by Gibbs and Colston (1995), he has challenged the idea of image schemas as mental representations stored in long-term memory (Gibbs 2005: 114–115). Drawing on the theory of dynamic systems (e.g., Freeman 2001; Thompson and Varela 2001), he argues instead that image schemas can be seen as “in-the-moment embodied simulation” and “attractors within human self-organizing systems” (Gibbs 2005: 131). The main tenets of Gibbs’ view will now be presented in some detail, then adopted and used productively in the domain of gesture in the ensuing sections. According to Gibbs, image schemas are:
[…] simulators of real-life actions or potential actions that a person might engage in. As a simulator, image schemas provide a kinesthetic feel that is not simply the output of some abstract computational machine, but the results of full-bodied experiences that have textures and a felt-sense of three-dimensional depth. (Ibid.: 119, my emphasis)
I suggest that some of the properties of image schemas emphasized by Gibbs bring to mind some of the essential facets of the gestural phenomena explored in this paper. This pertains first and foremost to the fact that manual gestures and whole-body postures, regardless of whether they evoke literal or metaphorical meanings, have a kinesthetic feel to them, may simulate witnessed or imagined actions, articulate tactile, affective, and intersubjective dimensions of experience, and naturally evolve dynamically in 3D-space (e.g., Müller 2017b; Sheets-Johnstone 1999/2011).7 In order to pinpoint some of the assumed correspondences in structure and function between embodied experiential gestalts and coverbal gestures, let us consider some of these attributes in more depth. Gibbs (2005) further characterizes images schemas as:
“emergent fleeting entities that are part of embodied simulation used in online thought including abstract reasoning” (p. 114)
“temporary linkages between sensory experience and short-lived conceptualizations of both concrete events and abstract events” (p. 114)
“continually tied to embodied action and simulations of experience” (p. 116)
“connect(ing) the domains of embodied action with the domain of linguistic action” (p. 123)
“emergent properties of human self-organizing systems that are continually recreated and re-experienced during cognitive and perceptual activity” (p. 131)
“attractors within human self-organizing systems. Attractors, such as BALANCE, SOURCE-PATH-GOAL, RESISTENCE, VERTICALITY, etc., reflect emerging points of stability in a system as it engages in real-world interaction” (p. 131)
The point I wish to make here is that gestures, in their naturally embodied and modality-specific ways, may not only be said to instantiate image schemas and force gestalts but also to exhibit properties similar to those that Johnson and Gibbs ascribe to them. It is thus assumed that gestures enact deeply embodied patterns of experience and may thus help both speakers and listeners to construct meaning and mutual understanding grounded in dynamically evolving image-schematic structures that evoke on-the-fly sensorimotor experience (Gibbs 2006). Hence, pragmatically driven functions of image schemas also seem to be operative in the production of multimodal utterances, that is, “linguistic action” (Gibbs 2005: 123), in which gestures allow speakers to relive experiences while describing them (see also, e.g., Cienki and Müller 2008; Müller 2017a,Müller, 2017b; Müller and Ladewig 2013 on metaphor in gesture). Accordingly, gesturally evoked image schemas and force gestalts can be seen as momentary, emergent, mediating structures that give some stability, or regularity, to people’s communicative movements. As such, they may be said to underpin certain recurring hand configurations and movement patterns within a given discourse context (e.g., Mittelberg 2006, 2010) or discourse culture (see, e.g., Bressem and Müller 2014; Ladewig 2011, 2014; Müller 2017b on recurrent gestures). Partaking in such multimodal processes of meaning-making, gestures may emerge as precise articulations of the essential structures of a dynamic experience (as in Figure 2), or as a profile of an experiential gestalt, that is, a “rough approximation of the image-schematic structures” (Gibbs 2005: 125). As such, a gestural structure may be used, for example, as a source domain for a metaphoric construal (as in Figure 3). In the next section, we will consider an array of tendencies in the gestural enaction of image schemas and force gestalts in which the human communicating body takes center stage.
4 A body-centered perspective on gesturally enacted image schemas and force gestalts inspired by dynamic systems
As argued in Section 3, Gibbs’ (2005, 2006) dynamic systems view of image schemas – which highlights their function as attractors that reflect stable points in self-organizing systems engaged in real-world interaction – seems to offer a particularly apt blueprint for describing certain tendencies in emergent gestural manifestations of such strongly embodied, dynamic patterns (see also Dewell 2005). Following Gibbs’ (2005) rationale a bit further down the line, the following exposition is inspired and broadly structured in accordance with the three kinds of cycles that are purported to underlie the self-organization of organisms qua dynamic systems (Thompson and Varela 2001): (a) “organismic regulation of the entire body;” (b) “sensorimotor coupling between organism and environment;” and (c) “intersubjective interaction involving the recognition of the intentional meaning of action and linguistic communication” (Gibbs 2005: 131).
Given the complexity and variability of human semiotic behavior, the goal of the present proposal is neither to offer a neat typology of image schemas and force gestalts, nor to cover all their possible gestural manifestations. As initially stated, it is not assumed that all gestures manifest image and/or force schemas. Rather, certain tendencies in the gestural enaction of these schemas are suggested, taking into account the fact that schemas interact flexibly in dynamic and transient ways. Related questions regarding the universality or culture-specificity of image schemas and how these factors may feed into various metaphoric and metonymic processes cannot be treated in detail here. What follows is a preliminary attempt at using a dynamic systems perspective to account for how gestures may enact image schemas and force gestalts. The groupings presented below are assumed to reflect some of the essential ways in which human beings experience their own corporeal embodiment as semiotic beings (e.g., Gallagher 2005) when interacting with the environment and other people in communicative situations.
4.1 Body-inherent and body-impacting image schemas and force dynamics
The human body unites several basic image schemas and force gestalts through its very morphology and the ways in which it moves through the world. In what follows, some of the fundamental body-inherent schemas and body-impacting forces will be discussed. While they typically interact in instances of meaning-making and cannot always be easily teased apart, they will be presented in two groups in order to highlight some of their specific functions with respect to gesture. Here a connection is drawn to the first cycle purported to underlie dynamic systems: “organismic self-regulation of the entire body” (Gibbs 2005: 131; Thompson and Varela 2001).
4.1.1 The human communicating body as an image-schematic structure
CONTAINER, CONTAINMENT (CONTENT, IN-OUT)
SURFACE, SUPPORT, CONTACT, ADJACENCY
The body is a semiotically versatile PART-WHOLE structure, parts of which, such as the head, the arms, and hands, as well as the entire torso or body may be recruited as sign carriers for the purpose of gestural communication. Gesturing hands may thereby metonymically stand for the entire gesturing person or be employed to portray the behavior of objects or people, for instance, as witnessed in a movie or in a scenario in which the speaker her/himself has participated (see, e.g., Dudis 2004 on body partitioning in ASL; Calbris 1990 on body segments; Sweetser 2012 on gestural viewpoint strategies). As research on metaphor in language has shown (e.g., Gibbs 2006; Lakoff and Johnson 1980, 1999), the body can be conceived of as a CONTAINER, with an inside and an outside (IN-OUT), that holds CONTENT, for example, food, organs, and fluids. Its SURFACE characteristics may indicate certain emotional or mental states via metonymy. The head may also stand for its CONTENT; for example, pointing at the head may refer to the mind inside it (see, e.g., Calbris 2011 on body-focused gestures; Mittelberg and Waugh 2014 on body-part indices). In addition, the body may lend SUPPORT to other people and to entities (as a whole or via the hands holding someone or something). It may thus create contiguity relations, such as ADJACENCY and BALANCE, with other persons or with objects and spatial structures that may become meaningful through metonymy and pragmatic inferencing (e.g., Mittelberg 2017b).
4.1.2 Body-inherent schemas and force dynamics acting upon the human body
BALANCE, VERTICALITY, STRAIGHT
COMPULSION, COUNTERFORCE, BLOCKAGE, RELEASE
ATTRACTION, REPULSION, PUSH-PULL
Let us now consider one of the most basic embodied schemas that involve the body in its entirety. BALANCE belongs to the elementary schemas that “operate in our bodily movements, perceptual acts, and orientational awareness” (Johnson 1987: 75). It “requires an ordering of forces and weights relative to some point, axis, or plane” (ibid.: 99), as graphically represented in the equilibrium schema provided in Figure 4.8 We can sense BALANCE in our own bodily posture and inner states, or the lack thereof, as well as in those of other people and in diverse domains of experience such as artworks (e.g., paintings, buildings, and musical pieces), theories, or social relationships (e.g., Johnson 1987, 2007; Krois 2011; Lakoff 2006; Mittelberg 2002). Standing up straight is in and of itself an essential bodily experience of human existence, regulated by various body schemas, such as VERTICALITY, STRAIGHT, and BALANCE (e.g., Gibbs 2005). Interestingly, as shown by experimental work (Gibbs et al. 1994), these schemas have been found to combine in different ways, thus underpinning various related meanings, both literal and metaphorical, of the polysemous verb stand. What the different meanings all have in common is their basis in physical experience and understanding. Perhaps not surprisingly, these schemas may also structure and balance gestural enactments from within the body.
In a study of BALANCE and related schemas in paintings by Paul Klee and their multimodal descriptions (Mittelberg 2013b), the participants showed a strong tendency to ‘become’ the figures depicted in the images by enacting aspects of the figures’ spatial positioning and postures they observed in them. In all the images used for the study, human-like figures are standing and trying to stay balanced despite various forces acting on their bodies. Each painting was shown for 60 seconds on a computer screen; then the participants were asked to recall and describe the composition of each painting as well as the effect it had on them. My intent here is to point out a few central aspects regarding the body-inherent (and interacting spatial) image schemas that may be said to be at work in the image space (e.g., Johnson 1987, 2007; Krois 2011) and also in the body of the study participant describing in gesture space what was recalled.
By way of example, let us consider the Dance of a Mourning Child (1922), which was used as a stimulus. The dancing child (Figure 5 (a)) is positioned off-center (CENTER-PERIPHERY) to the left side (LEFT-RIGHT). Not holding its oversized head up straight (STRAIGHT, VERTICAL, UP-DOWN), but tilted to one side, the child seems to be giving in to GRAVITY. She is trying to COUNTERBALANCE the weight of her large head by pointing her fully extended arms in the opposite direction, and by holding on to some kind of rope with both hands. She thus manages to create a BALANCED composure, which also seems to be maintained by the flow of the dance movement hinted at via the flowing skirt. The heart-shaped mouth functions as the anchor point of both the figure of the child and the image as a whole. According to Lakoff (2006: 154), “image schemas […] give form to art” in the sense that our embodied, enacting minds impose image schemas onto visual scenes. In this way, spatial and orientational relations as well as motor control schemas are understood to link perceptual to conceptual structure, and thus link form to meaning, including rather schematic meaning. Interestingly, the study participants picked up on image-inherent schemas, resulting in bodily articulations that convey correspondingly similar schematic meanings (see Mittelberg 2013b for details).
As shown in Figure 5 (b), the participant mirrors the figure’s posture with her whole body and thus also reflects the particular distribution of weight along the vertical axis.9 She imitates the figure’s tilted head and extended arms, as well as the position of the legs and the eye gaze directed downward. After transposing the image of the figure’s mouth onto her own lips and that of the figure’s eye slits onto her own eyes, she evokes the flowing skirt around her hips and upper legs by moving her hands repeatedly back and forth. The motion-capture data plot provided in Figure 5 renders these semiotic practices visible, thus allowing us to see the set of traces evoking the flowing skirt in the lower segment of gesture space, the traces of outward-going hands enacting the extension of the arms a little higher up, and the iconic forms depicting the mouth and eye slits in the upper segment. What is particularly insightful in terms of balance is that each of these bimanually produced pairs of signs exhibits features of an internal dynamic symmetry: they are thus also balanced in their own way.
This group of embodied schemas includes additional kinds of image schemas and force dynamics that may act within or on a person’s body and that are recruited for communicative purposes. For example, the image schema STRAIGHT may also motivate formational properties of hand shapes such as finger curvature (Mittelberg and Evola 2010). Flat, completely outstretched hands can be expected to create a marked, signifying condition of gesturing hands, as opposed to the unmarked condition of relatively relaxed hands. By way of illustration, let us consider the example of a multimodal metaphoric expression given in Figure 6. The speaker is describing how her former schoolmates, even though they were only just graduating from high school, had already neatly planned their entire lives “in preset ways” (German transcript: “in geregelten Bahnen”). She underscores this verbal metaphor with a gesture (Figure 6 (a)) using both hands, almost in parallel, with open flat palms facing downward and the fingers spread, pointing straight ahead and moving slightly away from her body in the center of the gesture space, thus creating a vector path (Evola 2010). In the graph in Figure 6(b), the vertical bar indicates that the angle of the right middle finger joint was 0° just before the speaker says “in preset ways.” Based on a qualitative analysis, the instantiated image schemas are STRAIGHT (hand shape), PATH (finger orientation and hand movement), and CENTER-PERIPHERY (placement of gesture). Via a metaphorical extension, the straightness of the hands stands for the ‘right’ way in life according to the speaker’s school mates’ view (STRAIGHT IS RIGHT, Cienki 1998b). The metaphor LIFE IS A JOURNEY manifests itself in the indicated PATH and its direction.
More empirical research combining qualitative and quantitative methods is needed in this domain, especially regarding force dynamics, but it seems safe to say that speakers may readily use their heads, manual gestures, or postures to (re)enact force qualities they have experienced or imagine experiencing while talking and interacting with their interlocutors (e.g., Wehling 2017).
4.2 Speaker-environment interactions (A): manipulating objects and schematizing events
We now turn to how people’s gesturing hands may evoke basic image schemas when they are pretending to interact with the physical environment, especially with material culture. This includes describing real-world, abstract, and/or imagined actions, scenarios, or events, including movement through space that speakers themselves, others, or objects have experienced or could experience. These phenomena are broadly seen as pertaining to the second cycle of self-organizing systems, namely, “sensorimotor coupling between organism and environment” (Gibbs 2005: 131; Thompson and Varela 2001), and thus shift our focus to how speakers often pretend to be manipulating or creating objects and/or tools while talking. As stated above, individual image schemas and force gestalts may partake in more than one of the tendencies in gestural instantiations of image schemas discussed here. As we will see, examining communicative bodily action may enable us to reveal how static and dynamic properties of the image-schematic structures involved may interact.
4.2.1 Schemas underlying the handling, creation, or analysis of (virtual) entities
PART-WHOLE, ADJACENCY, CONTACT, LINK
When people talk, they often seem to be holding, moving, or otherwise manipulating (virtual) physical objects and/or tools (e.g., Müller 1998; Streeck 2009; Streeck, Goodwin and LeBaron 2011). These environment-oriented actions have also been observed when speakers are talking about abstract subject matters. Such metaphorical construals may involve not only spatial metaphor and primary metaphor but also more complex metaphoric mappings and networks (e.g., Calbris 2003, 2011; Cienki and Müller 2008; Núñez, 2008; Sweetser, 1998). Much of the inspiration for the present proposal comes from my own earlier work on image schemas, metonymy, and metaphor in gestures and multimodal explanations of grammar and linguistic theory (Mittelberg 2006). The larger proportion of the recurring hand shapes and motion patterns observed in a corpus of recorded linguistics courses turned out to be reminiscent of image-schematic patterns, ranging from PATH, OBJECT and CONTAINMENT to LINK and PART-WHOLE, to name just the most prominent ones (Mittelberg 2010; see also Section 2.1).10 It was found that all the participating linguistics professors were pretending to manipulate physical objects and structures while they were talking about linguistic categories, functions, and units of varying degrees of complexity. The handling routines they employed included simply holding generic objects with one or two hands, seemingly taking them apart, changing the order of the constituents in a linear sentence model or in a hierarchical tree structure (e.g., Mittelberg 2008). I here reiterate the original claim that most of these mental and manual activities draw on a range of basic, interacting image schemas and also some force gestalts (e.g., the forceful PUSHing gesture portraying the idea of blurring a BOUNDARY shown in Figure 3).
To further substantiate this claim, let us consider the example shown in Figure 7a. Explaining the framework of emergent grammar, the speaker points out that “a priori… you cannot define a noun from a verb.” On the mention of the word “noun,” this “palm-up open hand gesture” (PUOH, Müller 2004) constitutes a perceivable SURFACE used as a physical SUPPORT structure (Mandler 1996) on which the speaker seems to be presenting the abstract category reified as a metaphorically construed, graspable entity (EXISTENCE). In this way, a virtual object seemingly sitting on the open hand (CONTACT, ADJACENCY) may become the center of attention in the physical and pragmatic context shared by the teacher and her students (Mittelberg 2008). While the speech is nonfigurative and the idea of SUPPORT or SURFACE is neither explicitly mentioned nor implied in what the linguist is saying, the gestural portrayal can be said to manifest the image schemas SUPPORT, ADJACENCY, and OBJECT, as well as the primary metaphor IDEAS ARE OBJECTS (Grady 1997). While the SUPPORT schema may be assumed to be inherent to most PUOH gestures, which provide a flat SURFACE for presenting imagined discourse contents, exactly how this schema is applied and interacts with other schemas needs to be determined for every local use of this highly polysemous gestural form (see also Bressem and Müller 2014; Müller 2004).11
Positing a continuum between concrete and abstract domains of experience, Calbris’ (2003) thorough account of preconceptual schemas in gestures produced by French speakers reveals that schemas underlying common manual actions performed on objects, such as cutting food, may be projected onto abstract domains of reasoning and motivate gestural construals; for example, cutting movements may thus represent the intellectual process of carrying out an analysis (see also Calbris 2011).
Another basic image schema is the CONTAINER or CONTAINMENT schema (e.g. Johnson 1987), which may be easily evoked through gesture. For instance, while saying “there is… what’s called the main verb,” the speaker in Figure 7b produces a cupped palm-up open hand with his left hand, thus imitating the form of a small round container. Drawing on the affordances of a single hand, the gesture iconically and schematically instantiates the CONTAINER schema and reflects the metaphoric understanding CATEGORIES ARE CONTAINERS or CONCEPTUAL STRUCTURE IS PHYSICAL STRUCTURE (Sweetser 1998). The idea of CONTAINMENT can also be evoked in gesture by using two hands to hold or to group imaginary entities. First explorations of a multimodal corpus enriched with motion-capture data revealed some regularities in speakers’ hand shapes and movements. Regularities of this kind are here assumed to stem, at least partly, from the specific image schemas and force gestalts that operate in – and thus structurally and pragmatically organize – a given gestural performance. For instance, the use of the curvature of the hands and the distance between them to seemingly group discourse items was found to reflect basic image schemas such as OBJECT and CONTAINER; these discourse-structuring gestures (Müller 1998) often simultaneously draw on the BALANCE schema as well as spatial relations such as LEFT-RIGHT or CENTER-PERIPHERY (Mittelberg and Evola 2010).
In the following example, we can witness how the speaker uses her hands and the space around her in an intuitive yet precise fashion. While explaining that her Master’s program in Speech Therapy consisted of two major subject matters (speech and therapy), she first lays out what was covered in each domain and then goes on to point out that, all in all, the program offered a broad spectrum of topics.12 In the gestural modality, each of the two study domains is depicted in the form of a midsized container or bounded space (Figure 8, container1 and container2). The numeric motion-capture data were used to measure the respective positioning of the two gestures in space, which revealed that the second ‘container gesture’ was placed right next to the first (ADJACENCY). In this way, the two different adjacent positions correlate with two differently semanticized bounded regions in space. Furthermore, in the first two occurrences of these bimanual gestures, the distance between the two hands is almost identical (first 33 cm, then 33,5 cm). In the third, larger gesture representing the entire course of study, the distance between the two hands (67 cm) encompasses both chunks of space delineated by the two preceding gestures (PART-WHOLE). This process of spatial addition is indicated by the yellow arrows overlaying the three stills (Figure 8 (a)–(c)). The decisive moments of delineation are highlighted by the first three vertical green bars in the graph provided in Figure 9; the x-axis represents the timeline; the y-axis indicates the distance between the markers attached to the middle finger of each hand.
This example hence shows a speaker delivering an astonishingly balanced multimodal performance. It also seems to suggest that a sort of pragmatically employed common-sense geometry (Deane 2005; Mittelberg 2010) is factoring in here. These insights clearly call for further investigation of the possible regularities in hand configurations and the use of space and how these aspects correlate with certain cognitive-pragmatic functions of the multimodal utterances in question.
4.2.2 Schemas underlying descriptions of objects/persons and their movement behavior
PATH (STRAIGHT, CIRCLE, SPIRAL, HORIZONTAL)
RESISTANCE, BLOCKAGE, REMOVAL OF CONSTRAINT
Visual perception and other ways of understanding the people and the things in one’s surroundings offer additional ways of interacting with the environment, here understood to include our semiotic environment constituted by the various media we regularly use (e.g., newspapers, TV, and movies). This broad array of phenomena can obviously not be fully taken into account here. Instead, the focus is placed on the pervasive OBJECT and PATH schemas that can be said to underlie the motion events amply discussed in the gesture literature (e.g., McNeill 2000; see also Section 2.1).13 For example, when recounting action sequences encountered in animated cartoon stories, speakers of a variety of different languages have been shown to depict the PATH and/or the manner of movement of cartoon characters or objects by producing iconic, in this case, rather schematic, and typified gestural traces – for example, straight or wavy lines extending along horizontal, diagonal, or vertical axes, circles, etc. It is noteworthy that while the literal renditions of the observed behavior of movie characters or real-life persons and objects involve schematization, they do not rely on a metaphoric construal operation in the way that the portrayal of abstract ideas often does, for instance, in the PATH example in Figure 2 or the gestural portrayals of morally correct behavior described by Cienki (1998a, 1998b; see also Mittelberg in press for more examples based on motion-capture data).
At this point I would like to come back to what was said earlier (in Section 2.4) regarding the tendency of schemas to interact flexibly and dynamically. In this connection, Gärdenfors (2007: 59) argues that instantiating spatial image-schematic structures often involves action and thus “forces exerted by an agent:” movement along a PATH, for example, may involve an agent creating momentum and locomotion. This aspect is relevant for gestural action depicting motion events that involve force dynamics. Here we also need to consider viewpointed meaning construction. For example, as speakers, we either depict what we have observed from afar – from an external perspective – or we immerse ourselves in the scenario and describe it – or our own actions – from an internal perspective (e.g., Mittelberg 2017c; Parrill 2009; Sweetser 2012). In the gesture conveying aspects of the speaker’s sitcom viewing experience (Figure 2), we saw how instantiations of the SOURCE-PATH-GOAL schema dynamically interact with other schemas and spatial metaphor (see also Mittelberg 2010: 370). In gesture, the PATH schema may be instantiated in various ways and approximations. But it indeed seems that, as pointed out by Gibbs (2005: 131), the topographic structure of a PATH emerges across single instantiations as a configuration of stable properties that are inseparable from sensorimotor activity. In this regard, we also need to take into account that gestural movement through space necessarily and genuinely exhibits an inherent directionality, whose indexical dimensions sometimes determine the main function of a gesture, as in the case of deictic gestures or route descriptions (e.g., Fricke 2007). In other semiotic processes, they may serve to create an iconic figuration, such as a path, that stands for something else in its own right (e.g., Mittelberg and Waugh 2014).
The idea that “by adding force dimensions to an image schema, we obtain the basic tools for analyzing dynamic and functional properties” (Gärdenfors 2007: 73) seems particularly captivating in relation to gesture.14 When we gesture, we may generally assume the role of someone who is being acted upon by external forces or who is exerting, or pretending to exert, force on imaginary entities or persons. We can also gesture less or more forcefully; accordingly, velocity peaks can be expected to add signifying qualities to the content expressed in a given gesture. As we will see below (Figure 10), motion-capture technology allows one to measure such qualities in gestural articulations.
PATH, STRAIGHT, CYCLE, SPIRAL
Since gestures are in and of themselves sensorimotor experiences that require time and space to evolve, they seem particularly apt at allowing speakers to describe and also propel cognitive activity (e.g., Cienki and Müller 2008; McNeill 1992, McNeill 2005; Müller 2017b). Thus a subgroup is needed here to capture the contours of experiences that have the character of dynamic, time-consuming processes. These may evoke gestures that are quite different from those simulating object manipulation or delineating spatial chunks that metaphorically stand for abstract entities. PROCESS is an image schema that may take on different gestalts, depending on the image schemas and force dynamics it interacts with to construe a given experience. For example, the STRAIGHT PATH gesture in Figure 2 portrays a PROCESS construed through the aspectual framing of an activity (sitcom watching) as progressive and bounded. The blurring of a BOUNDARY portrayed by the linguist in Figure 3 involves a FORCEfully conditioned PROCESS; the gestural framing of ongoing PROCESSes also includes CYCLES and SPIRALs drawn in the air (Beecks et al. 2016; Mittelberg and Hinnell 2015; Mittelberg in press; Schüller et al. 2017).15
Image schemas have also been found to motivate other intransitive hand movements that enact the idea of PROCESS without pretending to perform transitive actions. For instance, Ladewig (2011) has found that the CYCLE schema may underlie several variants of the cyclic gesture. Depending on the exact manner of execution and the placement viv-a-vis the gesturer’s body as well as the interlocutors, rotary hand movements may express aspects of continuation, repetition, or word searching. Gestures instantiating the CYCLE and PROCESS schemas have also been shown to express the operation of REITERATION in grammar and the activity of language use (Mittelberg 2006, 2010).16 Furthermore, Streeck (2009) correlates the gestural practice of ceiving (i.e., gestural thinking) with the speaker’s recruitment of an image schema when trying to find a gestural image for an emerging idea.
4.3 Speaker-environment interactions (B): pre-structuring of gesture spaces
UP-DOWN, LEFT-RIGHT, FRONT-BACK, NEAR-FAR
Building on the phenomena of organism-environment interactions discussed in Section 4.2, we will now discuss in more detail the spatial-relation schemas (e.g., Lakoff and Johnson 1999) that may be involved in gestural performances and instill them with schematic meaning. Given gestures’ specific kinetic and visuo-spatial mediality, spatial-relations schemas and topological schemas (Talmy 2000) are of particular interest here. Such schemas have been found to underpin speakers’ systematic uses of certain segments of gesture space, for instance, to structure their discourses by seemingly ‘placing’ ideas to respective sides of their bodies, or to highlight spatial relations between ideas, locations in space, or moments in time. For example, gestural references to events in the past or future may evoke the FRONT-BACK or the LEFT-RIGHT schemas, depending on the respective, culturally dependent metaphoric construals (Calbris 2008; Cienki 2013; Núñez and Sweetser 2006). In addition, the left and right side can be associated with different values in right and left handers (Casasanto 2009); along the UP-DOWN scale, upward motion has been found to signal that a positive valence is attached to the experience being talked about (Casasanto and Dijkstra 2010). It has also been shown that gesture space is used to metaphorically represent numbers and quantity, for example, MORE IS UP, MORE IS BIGGER, and MORE IS RIGHT (Winter, Perlman and Matlock 2014). In addition, social and abstract hierarchies can translate into spatial hierarchies: a common practice in linguistics courses is to trace syntactic tree structures in the air, whereby dominating nodes are located above the elements they govern (Mittelberg 2008).
These spatial-relations schemas may thus be said to virtually prestructure gesture space, including interpersonally shared spaces, into culturally shaped semanticized regions. In this way, they may motivate the ways in which speakers direct, orient, and position their gestures in relation to their viewpointed bodies (e.g., Priesters and Mittelberg 2013; Sweetser 2012). For example, the speaker shown in Figure 6 went on to report in the same narrative that she eventually decided to leave the originally intended career PATH and to move completely away from therapy: “in the end it has turned out, of course, that I moved totally away from therapy” (German transcript: “im Endeffekt ist es natürlich so gekommen, dass ich von der Therapie total weg bin;” see Figure 10). As in the gesture expressing the idea of ‘predetermined paths’ (Figure 6), the PATH schema here is not evoked by a horizontal trace gesturally drawn in the air (as in Figure 2), but virtually projected into gesture space: it originates at the speaker’s Origo (Bühler 1982; Fricke 2007; Sweetser 2012) and runs along the sagittal axis in front of her (into the future) through the CENTER of her gesture space. Moreover, the PATH schema does not just interact with the CENTER-PERIPHERY schema: as shown in Figure 10, on the mention of “total weg” (“totally away”), she PUSHes both hands away from her body (NEAR-FAR) and sideward to the right into the periphery of her gesture space (LEFT-RIGHT).17 Exhibiting a velocity peak in relation to her average speed of gesturing, this forceful gestural action (see Figure 10 (b)) vividly expresses the speaker’s personal stance regarding her career decisions.
This last example illuminates Gärdenfors’ (2007) idea that adding force gestalts to image schemas brings out their dynamic and functional characteristics (4.2.2). This seems to pertain not only to rhetorical effects in language use (Oakley 2005; Wehling 2017) but also to interlacing pragmatic functions of gestures, as shown above. In addition, these observations bring to mind McNeill’s (1992, 2005) notion of high communicative dynamism and raise questions about the ways in which image schemas, and force gestalts in particular, factor into multimodal communicative boosts such as the one described above (Figure 10 (a)).18
4.4 Interlocutor-oriented gestures: Discourse interaction and intersubjective understanding
We now turn to the third kind of cycle that may underlie the self-organization of dynamic systems and that primarily pertain to the “intersubjective interaction involving the recognition and the intentional meaning of action and linguistic communication” (Gibbs 2005: 131; Thompson and Varela 2001). As emphasized earlier, these cycles only provide a broad orientation regarding the different functions that image and force schemas may assume in gestures. The intersubjective dimension is highly relevant to multimodal interaction and particularly when interpreting the gestures of others (e.g., see Mittelberg 2006, 2008, and 2013a for a Peircean  view on image schemas in gesture). However, how image-schematic and force-dynamic structures may be said to underpin gestural interaction and to factor into how interlocutors may sense and understand what their vis-à-vis is intending to communicate has so far received comparably little attention (see Oakley 2005 on language). Yet, Müller (2008, 2017a) provides insights into the dynamic evolution of metaphoricity, thus reflecting the discursive intentions and collaborative activity of interlocutors (Müller 2017a). Distinguishing between agonistic and antagonistic forces (Talmy 1988, 2000) at work in discourse management gestures, Wehling (2017) examines the interactive dynamics involved in negotiating power relations between participants. Guibeault (2017) discusses the dynamic fabrics of image schemas, metaphor, and viewpoint in speeches of Barack Obama and their effects on the audience.
Image schemas and force dynamics may also be understood as mediating structures underpinning processes of understanding, resonance, and empathy – not only between interlocutors, but also between an artwork and its beholder (Gallese and Lakoff 2005; Lakoff 2006; Mittelberg 2013b). According to Krois (2011: 222f), image schemas can be seen as a specific sort of logic of visual images that allows their beholders to sense emotional content and thus be ‘moved’ (see also Bredekamp 2010 and Section 4.1.2).
Generally speaking, it seems that the gestural expression and creation of social relations – including how a speaker is taking a stance towards what s/he is saying and how s/he is reacting to interlocutors – offer good entry points to understanding multimodal interpersonal dynamics more deeply. More specifically, it would be interesting to investigate how spatial-relation schemas and force gestalts (as discussed in Section 4.3) may motivate indexical gestures oriented towards the interlocutor. These phenomena include identifying image-schematic structures underpinning various functions of the group of interactive gestures proposed by Bavelas et al. (1995), as well as the use and organization of shared gesture spaces (e.g. Sweetser and Sizemore 2008).
5 Concluding remarks
A main goal of this paper was to show that gestures, due to their corporeality and pragmatic mindedness, are particularly apt at instantiating spatial and dynamic properties of conceptual structures and processes. Hence it offers further support for the theory of the embodied mind (Gibbs 2006; Johnson 1987; Lakoff and Johnson 1999) and particularly the idea of enactivism (Varela, Thompson and Rosch 1991; see also Krois 2011). With reference to crucial aspects of work on the psychological reality of image schemas (Gibbs 2005; Gibbs and Colston 1995), the gesture-based approach presented here aims to provide further evidence for the semiotic reality of such patterns of multisensory experience (e.g., Danaher 1998; Mittelberg 2008 in press) and advances the use of motion-capture technology to this end. Inspired by the dynamic systems approach to image schemas put forth by Gibbs (2005), the present perspective shifts the focus from language processing and understanding to interpreting spontaneous multimodal language use and to identifying gestural patterns therein. There are several insights we can draw from the foregoing exposition and discussion.
First, gestural action and body postures have a natural and strong propensity to enact deeply embodied facets of image-schematic reasoning by recruiting “sensorimotor processes that are critical to how we understand ourselves, other people, and the world around us” (ibid.: 117) and that are tied to real and imagined bodily actions. Based on the observations made throughout the paper, we might even say that gestures play a crucial role in how “image-schematic reasoning is always being recreated by the body as people continue to engage in sensorimotor behaviors related to BALANCE, RESISTANCE, SOURCE-PATH-GOAL, CONTAINMENT” (ibid.: 116). Through drawing on such minimal, yet meaningful, structures and relations in the give and take of multimodal interaction, gestures may invoke the “kinesthetic feel” (ibid.: 116) of schemas and other construal operations that are not necessarily identifiable in the concurrent speech chain and thus provide additional semantic and pragmatic cues. While the above discussion mostly focused on certain kinds of kinetic manifestations of image-schematic and force-dynamic structures, it is claimed that, consistent with the literature cited, they typically build the schematic essence out of which rich inferences and associations evolve. How they feed into more complex processes of situated, intersubjective meaning-making that involve, for instance, metonymy, metaphor, frames, and grammatical constructions seems to be a promising line of inquiry for future cross-linguistic gesture research (e.g., Cienki 2013; Mittelberg 2017a/b; Müller 2017a/b).
Second, the noted structural similarity between gestures, image schemas, and force gestalts is assumed to be at least partly due to their shared embodied nature and dynamic schematicity. In the data discussed in the present paper, we have seen empirical indications that gestural instantiations of embodied schemas may indeed preserve the cognitive topology of the image-schematic source domains involved in metaphorical mappings, as posited by the invariance hypothesis (Lakoff 1990). These questions also bring to mind the Jakobsonian (1960) terms variance/invariance, according to which individual gestural instantiations of image schemas and force gestalts, which are typically partial and approximate, may be regarded as flexible, context-dependent variants of the invariant topological properties of a schema. An interesting avenue for future research would be to seek to specify, both theoretically and empirically, the exact relation between a given schema and its gestural enactments.
Third, several of the schemas repeatedly dealt with – SUPPORT, CONTAINMENT, PATH, and CONTACT – belong, according to Mandler (1996: 373–378), to the preverbal, spatially structured meaning system. These early acquired schemas also feature in the different kinds of gesture-schema interactions discussed, which were inspired by the three kinds of cycles that pertain to the self-organization of dynamic systems (Gibbs 2005; Thompson and Varela 2001). For present purposes, the latter were transposed into the following tendencies: body-inherent/forces acting upon the body (self-oriented; the body as an image-schematic structure); environment-oriented (material culture including spatial structures), and interlocutor-oriented (interaction with others). From the perspective proposed here, people who are talking can be seen as embodied, self-organizing systems engaged in real-world (or imagined) interactions with their physical environment and with their interlocutors or audience. Gesturally instantiated image schemas and force gestalts allow them to pragmatically organize their multimodal expressions and to structure, at least partly, how they interact and (co-)construct emergent meaning with their conversational partners. After each communicative boost involving a gestural movement excursion, or several there of, a speaker typically gathers her/himself, for example, by resuming a rest position. This seems to be partly afforded by a reordering of the image-schematic structures and forces acting within and upon the speaker’s body, and within the interactively formed communication space shared with her/his interlocutors.
It seems worthwhile to further investigate how image schemas and force gestalts feed into the above mentioned and other subtle aspects of meaning-making, involving, for example, felt qualities of resonance in gestural articulations. In this paper, only a limited number of multimodal sequences could be discussed. In order to further substantiate the claims made here, more empirical research is clearly needed, especially with regard to how embodied patterns instantiated in gestures may motivate and structure the intersubjective mediation and negotiation of both schematic and rich, complex meanings.
The author wishes to thank the editors of this special issue on “meaning-making” as well as two anonymous reviewers for their insightful comments and suggestions. Special thanks also go to Marlon Meuters and Matthias Priesters for providing motion-capture analyses and plots as well as to Mary M. Copple for helpful feedback on the manuscript. The research reported on in this article was supported by the Excellence Initiative of the German Federal and State Governments.
Andrén, M. 2010. Children’s gestures from 18 to 30 months. Lund: Lund University.Google Scholar
Arnheim, R. 1969. Visual thinking. Berkeley: University of California Press. Google Scholar
Beecks, C., M. Hassani, J. Hinnell, D. Schüller, B. Brenger, I. Mittelberg & T. Seidl. 2016. Efficient query processing in 3D motion capture databases via lower bound approximation of the gesture matching distance. International Journal of Semantic Computing 10(1). 5–25. CrossrefGoogle Scholar
Bredekamp, H. 2010. Theorie des Bildakts. Berlin: Suhrkamp. Google Scholar
Bressem, J. 2013. A linguistic perspective on the notation of form features in gestures. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 1079–1098. Berlin & Boston: Mouton De Gruyter. Google Scholar
Bressem, J. & C. Müller. 2014. A repertoire of German recurrent gestures with pragmatic functions. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill, and S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 1575–1591. Berlin & Boston: Mouton de Gruyter. Google Scholar
Bühler, K. 1982. Sprachtheorie: Die Darstellungsfunktion der Sprache. Stuttgart & New York: Fischer. Google Scholar
Calbris, G. 1990. The semiotics of French gestures. Cambridge: Cambridge University Press. Google Scholar
Calbris, G. 2003. From cutting an object to a clear cut analysis: Gestures as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture 3(1). 19–46. CrossrefGoogle Scholar
Calbris, G. 2008. From left to right: Coverbal gestures and their symbolic use of space. In A. Cienki & C. Müller (eds.), Metaphor and gesture, 27–53. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Calbris, G. 2011. Elements of meaning in gesture. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Casasanto, D. 2009. Embodiment of abstract concepts: Good and bad in right- and left-handers. Journal of Experiential Psychology. 351–367.
Cienki, A. J. 1998a. Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J. P. Koenig (ed.), Discourse and cognition: Bridging the gap, 189–204. Stanford: CSLI. Google Scholar
Cienki, A. J. 2005. Image schemas and gesture. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 421–441. Berlin: Mouton de Gruyter. Google Scholar
Cienki, A. & C. Müller (eds.). 2008. Metaphor and gesture. Amsterdam & Philadelphia: John Benjamins.Google Scholar
Croft, W. & A. Cruse. 2004. Cognitive linguistics. Cambridge: Cambridge University Press. Google Scholar
Damasio, A. 1994. Descartes’ error: Emotion, reason, and the human brain. New York: Putnam & Sons. Google Scholar
Danaher, D. 1998. Peirce’s semiotic and conceptual metaphor theory. Semiotica 119(1/2). 171–207. Google Scholar
Dancygier, B. & E. Sweetser (eds). 2014. Figurative language. Cambridge: Cambridge University Press. Google Scholar
Deane, P. D. 2005. Multimodal spatial representations: On the semantic unity of over. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 235–282. Berlin & New York: Mouton de Gruyter. Google Scholar
Dewell, R. 2005. Dynamic patterns of containment. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 369–374. Berlin: Mouton de Gruyter. Google Scholar
Dudis, P. G. 2004. Body partitioning and real space blends. Cognitive Linguistics 15(2). 223–238. Google Scholar
Evans, V. & M. Green. 2006. Cognitive linguistics: An introduction. Edinburgh: Edinburgh University Press. Google Scholar
Evola, V. 2010. Multimodal cognitive semiotics of spiritual experiences: Beliefs and metaphors in words, gestures, and drawings. In Parrill V. Tobin & M. Turner (eds.), Form, meaning, and body, 41–60. Stanford: CSLI. Google Scholar
Fillmore, C. J. 1982. Frame semantics. In The Linguistic Society of Korea (ed.) Linguistics in the Morning Calm, 111–137. Seoul: Hanshin. Google Scholar
Forceville, C. & E. Urios-Aparisi (eds). 2009. Multimodal metaphor, Vol. 11. Berlin: Walter de Gruyter. Google Scholar
Freeman, W. 2001. How brains make up their minds. New York: Columbia University Press. Google Scholar
Fricke, E. 2007. Origo, Geste, und Raum: Lokaldeixis im Deutschen, Vol. 24. Berlin & New York: Walter de Gruyter. Google Scholar
Gallagher, S. 2005. How the body shapes the mind. Oxford: Oxford University Press. Google Scholar
Gärdenfors, P. 2007. Cognitive semantics and image schemas with embodied forces. In J. M. Krois, M. Rosengren, A. Steidele & D. Westerkamp (eds.), Embodiment in culture and cognition, 57–76. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Gibbs, R. W., Jr. 1994. The poetics of mind. Cambridge: Cambridge University Press. Google Scholar
Gibbs, R. W., Jr. 2005. The psychological status of image schemas. In B. Hampe (ed.), From perception to meaning: Image Schemas in cognitive linguistics, 113–135. Berlin & New York: Mouton de Gruyter. Google Scholar
Gibbs, R. W., Jr. 2006. Embodiment and cognitive science. Cambridge: Cambridge University Press. Google Scholar
Gibbs, R. W., Jr, D. Breitel, M. Harrington & P. Sanders. 1994. Taking a stand on the meanings of “stand:” Embodied experience as motivation for polysemy. Journal of Semantics 11. 231–251. CrossrefGoogle Scholar
Grady, J. E. 1997. Foundations of meaning: Primary metaphors and primary scenes. Berkeley: University of California dissertation. Google Scholar
Guibeault, D. 2017. How politicians express different viewpoints in gesture and speech simultaneously. Cognitive Linguistics 28(3). 417–447. Google Scholar
Hampe, B. (ed). 2005. From perception to meaning: Image schemas in cognitive linguistics. Berlin & New York: Mouton de Gruyter. Google Scholar
Hampe, B. (ed.). 2017. Metaphor: Embodied cognition and discourse. Cambridge: Cambridge University Press.Google Scholar
Jakobson, R. 1960. Linguistics and poetics. In K. Pomorska & S. Rudy (eds.), Roman Jakobson – Language in literature, 62–94. Cambridge: Harvard University Press. Google Scholar
Johnson, M. 1987. The body in the mind: The bodily basis of meaning, imagination, and reasoning. Chicago: Chicago University Press. Google Scholar
Johnson, M. 2005. The philosophical significance of image schemas. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 15–33. Berlin & New York: Mouton de Gruyter. Google Scholar
Johnson, M. 2006. Merleau-Ponty’s embodied semantics: From immanent meaning, to gesture, to language. EuAmerica 36(1). 1–27. Google Scholar
Johnson, M. 2007. The meaning of the body: Aesthetics of human understanding. Chicago: Chicago University Press. Google Scholar
Kendon, A. 2004. Gesture: Visual action as utterance. Cambridge: Cambridge University Press. Google Scholar
Kimmel, M. 2005. Culture regained: Situated and compound image schemas. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 285–311. Berlin & New York: Mouton de Gruyter. Google Scholar
Krois, J. M. 2011. Körperbilder und Bildschemata: Aufsätze zur Verkörperungstheorie ikonischer Formen. H. Bredekamp & M. Lauschke (eds.) Berlin: Akademie Verlag. Google Scholar
Krois, J. M., M. Rosengren, A. Steidele & D. Westerkamp (eds). 2007. Embodiment in cognition and culture. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Ladewig, S. H. 2011. Putting the cyclic gesture on a cognitive basis. CogniTexts. 6. Google Scholar
Ladewig, S. H. 2014. Recurrent gestures. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 1558–1574. Berlin & Boston: De Gruyter Mouton.Google Scholar
Lakoff, G. 1987. Woman, fire, and dangerous things: What categories reveal about the mind. Chicago: University of Chicago Press. Google Scholar
Lakoff, G. 2006. The neuroscience of form in art. In M. Turner (ed.), The artful mind, 155–169. Oxford & New York: Oxford University Press. Google Scholar
Lakoff, G. & M. Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press. Google Scholar
Lakoff, G. & M. Johnson. 1999. Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books. Google Scholar
Lakoff, G. & M. Turner. 1989. More than cool reason: A field guide to poetic metaphor. Chicago: Chicago University Press. Google Scholar
Langacker, R. W. 1987. Foundations of cognitive grammar, Vol. I. Stanford: Stanford University Press. Google Scholar
Mandler, J. M. 1996. Preverbal representation and language. In P. Bloom, M. A. Peterson, L. Nadel & M. F. Garrett (eds.), Language and space, 365–384. Cambridge: MIT Press. Google Scholar
Mandler, J. M. 2005. How to build a baby III: Image schemas and the transition to verbal thought. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 137–164. Berlin: Mouton de Gruyter. Google Scholar
Marghetis, T. & B. K. Bergen. 2014. Embodied meaning inside and out: The coupling of gestures and mental simulation. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 2000–2007. Berlin & Boston: De Gruyter Mouton. Google Scholar
McNeill, D. 1992. Hand and mind: What gestures reveal about thought. Chicago: Chicago University Press. Google Scholar
McNeill, D. (ed.). 2000. Language and gesture. Cambridge: Cambridge University Press. Google Scholar
McNeill, D. 2005. Gesture and thought. Chicago: Chicago University Press. Google Scholar
Merleau-Ponty, M. 1962. Phénoménologie de la perception. Paris: Gallimard. Google Scholar
Mittelberg, I. 2002. The visual memory of grammar: Iconographical and metaphorical insights. Metaphorik 2. 69–88. Google Scholar
Mittelberg, I. 2006. Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar. Ann Arbor: UMI Cornell University dissertation. Google Scholar
Mittelberg, I. 2008. Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar. In A. Cienki & C. Müller (eds.), Metaphor and gesture, 115–154. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Mittelberg, I. 2010. Geometric and image-schemantic patterns in gesture space. In V. Evans & P. A. Chilton (eds.), Language, cognition, and space: The state of the art and new directions, 351–385. London: Equinox. Google Scholar
Mittelberg, I. 2013a. The exbodied mind: Cognitive-semiotic principles as motivating forces in gesture. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 750–779. Berlin & Boston: Mouton de Gruyter. Google Scholar
Mittelberg, I. 2013b. Balancing acts: Image schemas and force dynamics as experiential essence in pictures by Paul Klee and their gestural enactments. In M. Borkent, B. Dancygier & J. Hinnell (eds.), Language and the creative mind, 325–346. Stanford: CSLI Publications. Google Scholar
Mittelberg, I. 2017a. Embodied frames and scenes: Body-based metonymy and pragmatic inferencing in gesture. Gesture 16(2). 204–245. Google Scholar
Mittelberg, I. 2017b. Experiencing and construing spatial artifacts from within: Simulated artifact immersion as a multimodal viewpoint strategy. Cognitive Linguistics 28(3). 381–415. Google Scholar
Mittelberg, I. 2017c. Multimodal existential constructions in German: Manual actions of giving as experiential substrate for grammatical and gestural patterns. Linguistics Vanguard 3. 1. doi:. CrossrefGoogle Scholar
Mittelberg, I. In press. Peirce’s Universal Categories: On their potential for gesture theory and multimodal analysis. Semiotica.Google Scholar
Mittelberg, I. & V. Evola. 2010. Image schemas in captured motion: First insights from a 3D gesture analysis. Conference poster. San Diego: CSDL. Google Scholar
Mittelberg, I, V. Evola, et al. 2014 Iconic and representational gestures. In Müller C., Cienki A., Fricke E., Ladewig S. (eds.), Language-body-communication: An international handbook on multimodality in human interaction: 1732–1746. Berlin, Boston: Mouton de Gruyter.
Mittelberg, I. & J. Hinnell. 2015. Emergent grammar in gesture: A motion-capture analysis of image-schematic aspectual contours in English speakers. Conference paper. Newcastle: ICLC. Google Scholar
Mittelberg, I. & G. Joue. 2017. Source actions ground metaphor via metonymy: Towards a frame-based account of gestural actions in multimodal discourse. In B. Hampe (ed.), Metaphor: Embodied cognition and discourse, 119–137. Cambridge: Cambridge University Press. Google Scholar
Mittelberg, I. & L. R. Waugh. 2014. Gestures and metonymy. In C. Müller, A. Cienki, E. Fricke, S. H. Ladewig, D. McNeill & S. Theßendorf (eds.), Body – language – communication: An international handbook on multimodality in human interaction, 1747–1766. Berlin & Boston: Mouton de Gruyter. Google Scholar
Müller, C. 1998. Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Spitz Verlag. Google Scholar
Müller, C. 2004. Forms and uses of the palm up open hand: A case of a gesture family? In C. Müller & R. Posner (eds.), The semantics and pragmatics of everyday gesture, 233–256. Berlin: Weidler. Google Scholar
Müller, C. 2008. What gestures reveal about the nature of metaphor. In A. Cienki & C. Müller (eds.), Metaphor and gesture, 219–245. Amsterdam: John Benjamins. Google Scholar
Müller, C. 2017a. Waking metaphors: Embodied cognition in multimodal discourse. In B. Hampe (ed.), Metaphor: Embodied cognition and discourse, 297–316. Cambridge: Cambridge University Press. Google Scholar
Müller, C. & S. H. Ladewig. 2013. Metaphors for sensorimotor experiences: Gestures as embodied and dynamic conceptualizations of balance in dance lessons. In M. Borkent, B. Dancygier & J. Hinnell (eds.), Language and the creative mind, 295–324. Stanford: CSLI Publications. Google Scholar
Núñez, R. 2008. A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor. In A. Cienki & C. Müller (eds.), Metaphor and gesture, 93–114. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Núñez, R. & E. Sweetser. 2006. With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science 30. 401–450. CrossrefPubMedGoogle Scholar
Oakley, T. 2005. Force-dynamic dimensions of rhetorical effect. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 443–473. Berlin & New York: Mouton de Gruyter. Google Scholar
Oakley, T. 2007. Image schemas. In D. Geeraerts & H. Vuyckens (eds.), The Oxford handbook of cognitive linguistics, 214–235. Oxford: Oxford University Press. Google Scholar
Panther, K. U. & G. Radden (eds). 1999. Metonymy in language and thought. Amsterdam: John Benjamins. Google Scholar
Peirce, C. S. 1960. Collected Papers of Charles Sanders Peirce (1931–1958), Vol. I.: Principles of philosophy; Vol. II.: Elements of logic. C. Harthorne & P. Weiss (eds.). Cambridge: The Belknap Press of Harvard University Press. Google Scholar
Priesters, M. & I. Mittelberg. 2013. Individual differences in speakers’ gesture spaces: Multi-angle views from a motion-capture study. Proceedings of the Tilburg Gesture Research Meeting (TiGeR), June 19–21. Google Scholar
Schüller, D., C. Beecks, M. Hassani, J. Hinnell, B. Brenger, T. Seidl & I. Mittelberg. 2017. Automated pattern analysis in gesture research: Similarity measuring in 3D motion capture models of communicative action. Digital Humanities Quarterly 11(2). Google Scholar
Sheets-Johnstone, M. 1999/2011. Primacy of movement. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Streeck, J. 2009. Gesturecraft: The manu-facture of meaning. Amsterdam: John Benjamins. Google Scholar
Streeck, J., C. Goodwin & C. LeBaron (eds). 2011. Embodied interaction: Language and the body in the material world. Cambridge: Cambridge University Press. Google Scholar
Sweetser, E. 1998. Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes. CD-ROM. Elsevier. Google Scholar
Sweetser, E. 2007. Looking at space to study mental spaces: Co-speech gestures as a crucial data source in cognitive linguistics. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson & M. Spivey (eds.), Methods in cognitive linguistics, 201–224. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Sweetser, E. 2012. Introduction: Viewpoint and perspective in language and gesture, from the ground down. In B. Dancygier & E. Sweetser (eds.), Viewpoint in language: A multimodal perspective, 1–22. Cambridge: Cambridge University Press. Google Scholar
Sweetser, E. & M. Sizemore. 2008. Personal and interpersonal gesture space: Functional contrasts in language and gesture. In A. Tyler, Y. Kim & M. Takada (eds.), Language in the context of use, 25–51. Berlin & New York: Mouton de Gruyter. Google Scholar
Talmy, L. 2000. Toward a cognitive semantics, Vols. 1 & 2. Cambridge: MIT Press. Google Scholar
Turner, M. (ed.). 2006. The artful mind: Cognitive science and the riddle of human creativity. Oxford: Oxford University Press. Google Scholar
Varela, F., E. Thompson, & E. Rosch. 1991. The embodied mind: Human cognition and experience. Cambridge: MIT Press. Google Scholar
Williams, R. F. 2008. Gesture as a conceptual mapping tool. In A. Cienki & C. Müller (eds.), Metaphor and gesture, 55–92. Amsterdam & Philadelphia: John Benjamins. Google Scholar
Winter, B., M. Perlman & T. Matlock. 2014. Using space to talk and gesture about numbers: Evidence from the TV news archive. Gesture 13. 377–408. Google Scholar
Wolf, D., L. M. Rekittke, I. Mittelberg, M. Klasen & K. Mathiak. 2017. Perceived conventionality in co-speech gestures involves the fronto-temporal language network. Frontiers of Human Neuroscience. doi:. CrossrefGoogle Scholar
Zlatev, J. 2005. What’s in a schema? Bodily mimesis and the grounding of language. In B. Hampe (ed.), From perception to meaning: Image schemas in cognitive linguistics, 313–342. Berlin & New York: Mouton de Gruyter. Google Scholar
Zlatev, J. 2014. Image schemas, mimetic schemas, and children’s gestures. Cognitive Semiotics 7(1). 3–29. Google Scholar
These data (including Figure 2) stem from a joint study on aspectual framing conducted with Jennifer Hinnell, University of Alberta (see also Beecks et al. 2016; Schüller et al. 2017 for quantitative analyses involving computational modeling).
While drawing virtual lines in the air is a common gestural practice (e.g., McNeill 1992, McNeill 2000), not all instances of producing such gestural lines involve a construal operation (see also Bressem 2013; Calbris 2008; Calbris 2011; Mittelberg 2006, 2010 on a range of schematic movement patterns).
Evans and Green (2006: 190) provide a partial list of image schemas organized according to the source of their experiential grounding, e.g., SPACE, CONTAINMENT, LOCOMOTION, BALANCE, FORCE, UNITY-MULTIPLICITY, IDENTITY, and EXISTENCE, in an attempt to unite the ways of grouping or categorizing image schemas suggested by Cienki (1998b), Gibbs and Colston (1995), Johnson (1987), Lakoff (1987), Lakoff and Turner (1989). For inventories of image schemas, see also Clausner and Croft (1999: 15) and Croft and Cruse (2004: 45). For a recent proposal as to how to define image schemas, see Mandler and Pagán Cánovas (2014).
Johnson (2007: 170) also associates images schemas with what Lakoff and Gallese called cogs (Gallese and Lakoff 2005): “Image schemas are prime examples of cogs, since they are recurring patterns of sensorimotor interactions.” Moreover, Krois (2011): 249 alerts us to not overlook the affective dimensions of image schemas.
Indeed, gesture researchers have provided ample evidence that speakers tend to simulate physical actions when gesturing (e.g., Bressem and Müller 2014; Calbris 2011; Hostetter and Alibali 2008; Marghetis and Bergen 2014; Mittelberg 2017a; 2017b; Mittelberg and Joue 2017; Müller 1998, 2004, 2017b).
Transcript of the verbal description (Figure 5): “I thought it was a girl, because, uhm she looked like she was wearing some sort of short flowing skirt uhm, and… uh her head was turned to this side… if I were mirroring what she was doing and her arms were like this. Uhm, and… uh… her mouth was almost in the shape of a heart and… uhm, I kept trying to see if her eyes were open or closed, and it looked like they were just slits.”
Other patterns derived from the gestural representations found in this corpus pertained to geometric forms such as circles, triangles, squares, etc. (Mittelberg 2010).
See also Mittelberg (2017c) on how this gesture type feeds into multimodal instantiations of existential constructions in German, thus evoking the image schema EXISTENCE.
Transcript of the verbal description in German: ”also wir greifen uns halt nicht nur die Sprache, sondern wirklich auch den therapeutischen Bereich des Sprechens, also dazu Stimmstörungen, Schluckstörungen, die ganzen neurologischen Störungen, alles, wirklich breit ah.. ehm… gefächert” (Engl. translation: “so we treat in fact not just language, but really also at the therapeutic side of speech production, so also voice disorders, swallowing disorders, all the neurological disorders, everything, really broad ah… ehm… diversified”).
See also Mittelberg and Evola (2014) for an overview on work on iconic gestures. See Mandler and Pagán Cánovas (2014) for insights into preverbal schemas and schemas in language development.
Gärdenfors (2007) distinguishes between a first-person and a third-person perspective on the experience of force dynamics, referring to the former as “power” and to the latter as “force.”
Calbris (2011) discusses semantic variation in gestural expressions of process in relation to straight-line and curved movements (without reference to image schemas).
For an overview of cyclic gestures see also Calbris (2011: 149–153).
See Bressem and Müller (2014) on away-gestures as one of the recurrent gestures in German discourse.
It would also be interesting to investigate the ways in which image schemas underpin what McNeill (2000), McNeill (2005) calls “catchments,” that is, particular gesture features that reoccur over a stretch of discourse and thus contribute to discourse cohesion.
About the article
Irene Mittelberg is Professor of Linguistics and Cognitive Semiotics at the Institute for English, American, and Romance Studies of the RWTH Aachen University, where she directs the Centre for Sign Language and Gesture (SignGes) and founded the Natural Media Lab, a gesture research lab equipped with motion-capture technology. She holds an M.A. in French linguistics and art history from Hamburg University and a Ph.D. in linguistics and cognitive studies from Cornell University. Her work combines the semiotic theories of C. S. Peirce and Roman Jakobson with contemporary embodied approaches to language, cognition, and multimodal interaction, notably to examine how image schemas, metonymy, metaphor, frames, and viewpoint motivate coverbal gestures. Recent interdisciplinary work includes pattern analysis in kinetic gesture data and the adoption of Peirce’s universal categories for neuroscientific research into gesture. She co-edited (with M. Gonzalez-Marquez, S. Coulson, and M. Spivey) Methods in Cognitive Linguistics (2007), edited a special issue on gesture for Sprache und Literatur (2010), and has (co-)published over 50 articles in journals and books.
Published Online: 2018-05-30