Gestures as image schemas and force gestalts: A dynamic systems approach augmented with motion-capture data analyses

1 Introduction

Embodied image schemas have been attributed a prominent role not only in structuring our perceptual experience and bodily being-in-the-world but also in motivating both literal and figurative expressions in language and other modalities (e.g., Gibbs 2006; Hampe 2005, 2017; Krois et al. 2007). Image schemas are generally understood as dynamic, embodied patterns of recurrent experiences such as visual perception, sensorimotor routines, and interactions with the physical and social world (e.g., Gibbs and Colston 1995; Johnson 1987, 2007; Lakoff 1987; Lakoff and Johnson 1999). According to Johnson (2007: 144–145).

Image schemas constitute a preverbal and mostly nonconscious, emergent level of meaning. […] Although they are preverbal, they play a major role in the syntax, semantics, and pragmatics of natural language. They lie at the heart of meaning, and they underlie language, abstract reasoning, and all forms of symbolic interaction.

The primary aim of this paper is to demonstrate some of the ways in which spontaneous gestures may be said to share some of the essential characteristics and semantic import of image schemas and force gestalts. Gestures may also, generally speaking, “bind body and mind” (ibid.: 145), not only in momentary instances of situated meaning-making but also across time, contexts, and individuals. More specifically, the present proposal starts from the observation that there seems to be a certain structural similarity between certain image and force schemas and certain gestural forms and movements, which partly seems to stem from their shared experiential bases, in particular, from bodily, object-oriented actions, interpersonal interaction, and movement through space (e.g., Mittelberg and Joue 2017). However, it should be pointed out right from the start that it is not maintained here that all gestures instantiate image schemas or force gestalts. Nor can one assume a one-to-one correspondence between such embodied schematic patterns and their gestural manifestations. In some cases, we might be able to see a certain phenomenological, structural similarity (e.g., Merleau-Ponty 1962; Mittelberg 2013a; 2013b). This similarity seems to reside in their common schematic nature, which may be said to retain essential structures and qualities of basic kinds of experience. When looking at “the phenomenology of the human body in action” (Gibbs 2005: 116) and notably, as it is done here, at gesture as a genuine phenomenon of human communicative behavior, a crucial factor is the degree to which we may perceive and produce gestures more consciously at certain moments of meaning-making than at others. In fact, it is precisely the less conscious aspects of gestures that make them a valuable source of insights into the less monitored functioning of cognition and language during communication (e.g., Sweetser 2007). Since gestures are essentially activations of the human body, they can be viewed as more immediate expressions than spoken or written discourse. Hence, they enable us to empirically study the origins and nature of the deeply embodied aspects of the human mind to which image schemas (and metaphor) clearly belong (e.g., Calbris 2008; Cienki 1998a, 1998b; McNeill 1992; Mittelberg 2010, 2013a; Müller 2008, 2017b; Streeck 2009; Sweetser 1998; Zlatev 2005; see Cienki 2013; Zlatev 2014 for recent overviews).

Gestures here are understood as discourse-embedded, kinetic actions that are performed with the head, hands, arms, torso, or entire body and have some communicative function(s) (e.g., Calbris 2011; Kendon 2004; McNeill 1992; Müller 1998). While gestures may consist of full-fledged, iconic imitations of human postures and actions, they are often rather schematic gestalts that metonymically evoke locally and/or prototypically salient aspects of a certain entity, scenario, action routine, or thought process (Mittelberg and Waugh 2014). Gestures typically consist of evanescent hand configurations, motion onsets, or virtual movement traces from which one may pragmatically infer, for instance, the idea of CONTAINMENT via a cupped hand, an OBJECT by seemingly holding it between two hands or pointing at it, or a PATH by tracing its progression in the air.^[1] Alternatively, a NEAR-FAR spatial relation may be established by locating a point close to one’s body and another one further away – that is, by exploiting the depth of gesture space as well as interpersonal gesture spaces (e.g., Calbris 2003, 2011; Cienki 1998b; 2005; Mittelberg 2006, 2008, 2010; Sweetser and Sizemore 2008; Williams 2008). In a similar fashion, force-dynamic gestalts (Talmy 1988) may be minimally enacted, or simulated (Gibbs 2005), via communicative postures and movements. For instance, subtle downward shifts in body posture may momentarily evoke the sensation of physical forces acting upon the speaker’s body, thus recruiting the BALANCE and GRAVITY schemas, which may metaphorically construe the emotional state of feeling down (e.g., Mittelberg 2013b). What exactly such typically polysemous articulations locally enact or portray is specified via the concurrent speech and other contextual factors (e.g., Calbris 2011; Kendon 2004; McNeill 1992, 2005; Müller 1998, 2010, 2017a).

The flexible correspondences between gestures and the image schemas of interest here seem to be anchored in, and regularly reinforced through, the mediating role that the human body plays not only in how human beings derive patterns from repeated multisensory experience, thus building up conceptual structures, but also in how they express or enact embodied semantic primitives (Lakoff 2006) and experiential gestalts (Gibbs 2005) when communicating via speech, gesture, and other modalities (Mittelberg 2008). Due to their strongly grounded nature, image and force schemas have already been shown to be enacted both on-the-fly and across similar gestural phenomena, thus engendering a certain regularity in gestural forms and functions within certain cultural communities or discourse contexts (e.g., Bressem and Müller 2014; Ladewig 2011; Mittelberg 2006, 2010; in press). It is thus assumed here that image and force schemas may build the structural essence of individual gestures as well as underpin the emergence of gestural patterns across speakers and contexts of use.

In line with Johnson (1987, 2005, 2007) and (Gibbs 2003, 2005, 2006), my intent is to present gestural evidence that image schemas and force gestalts are not “merely skeletons or abstractions” (Johnson 2005: 31) but should be seen instead as dynamic gestalts of experiential essence that not only operate in cognitive and linguistic processes but also in speakers’ pragmatically minded gestural actions. A central premise of the perspective taken here is that gestures may provide insights into the “semiotic reality” (Danaher 1998: 190; (Mittelberg 2006, 2008, 2013b) of pervasive embodied schemas. Semiotic reality here pertains to both cognitive and expressive dimensions of semiosis and is understood to encompass cognitive, linguistic, physical, and culturally shaped aspects of observable patterns of communicative action. It is further maintained that image schemas and force gestalts also partake in how gestures convey “felt qualities of our experience, meaning and thought” (Johnson 2005: 31). Producing and interpreting multimodal utterances always involves, at least to a certain degree, felt sensations of the subjective and affective qualities in one’s own articulations as well as in the communicative behavior of our conversational partners (e.g., Mittelberg 2013a; Müller 2017b).

With these tenets in mind, this paper presents a first attempt at a gesture-based, body-centered account of image schemas and force gestalts that may be observed to operate in gestural behavior. Inspired by Gibbs’ (2005) dynamic systems account of image schemas, such flexible and typically interacting tendencies in gestural image-schema enactments are seen as evolving broadly along the following lines: dynamic embodied patterns that motivate gestures a) from within the body and in response to forces acting on the body (body-inherent and acting on body); b) that simulate organism-environment interactions (environment-oriented); c) are inscribed in virtually pre-structured gesture spaces conceived of from the perspective of the speaker (pre-structuring gesture space); or d) function as organizing principles of communicative interactions and thus factor into achieving intersubjective understanding among conversational partners (interlocutor-oriented). To back up this proposal, I will draw on gestural examples taken from American-English and German multimodal discourse. Some of the recordings include motion-capture data that allow one to precisely measure the temporal and spatial dimensions of gestural articulations as well as to visualize the path, direction, and manner of movement excursions.

2 Inherently meaningful image schemas and force dynamics across modalities

Understanding communicative gestures as mediators between experiential grounding and conceptual schematicity naturally puts the focus on how multimodal communicative behavior is motivated and structured by embodied patterns stemming from habits of perception and organism-environment interactions. The present section serves to lay out some fundamental aspects of image schemas and force dynamics as well as to show how gesture, being a physical medium, may provide additional evidence for their embodied origins and functions. Since limitations of space do not allow for a comprehensive overview of the substantial, multidisciplinary body of research on image schemas (e.g., Gibbs 2006; Hampe 2005, 2017), special attention will be paid to those aspects that are particularly relevant with regard to communicative bodily actions in general and to the present proposal in particular.

The guiding idea is that such deeply embodied patterns are inherently meaningful (e.g., Gibbs 2005; Lakoff 2006) in the sense that they instill schematic meaning in certain gestural behaviors. This is assumed to function similarly to the way in which basic semantic tendencies reside in, for instance, certain grammatical structures (e.g., prepositional phrases expressing abstract spatial relations) or primary metaphors that draw on correlations of embodied experiences and subjective meaning (e.g., INTIMACY IS CLOSENESS, Grady 1997). In language, these basic patterns have been shown to factor into rich inferences and structures of higher complexity, thus underpinning more complex, semiotically and culturally embedded meaning (e.g., Dancygier and Sweetser 2014; Johnson 2007). In a similar vein, it is suggested here that when gestural behavior is motivated by embodied image schemas and/or force gestalts, its core consists of inherently meaningful structures. A central, underlying assumption is that this kind of semantic essence (or “immanent meaning” Johnson 2006) should emerge from the corresponding gestures to a certain degree, even without considering the speech content (e.g., Mittelberg in press; Wolf et al. 2017). While more empirical research is clearly needed to substantiate this claim, some indications to this effect will be presented throughout the paper.

2.1 Image schemas

According to Johnson’s (1987: xiv) original definition, image schemas are “recurring, dynamic patterns of our perceptual interactions and motor programs that give coherence and structure to our experience.” Basic embodied schemas include OBJECT, CONTAINER, PATH, BOUNDARY, LINK, CENTER-PERIPHERY, and BALANCE, as well as spatial schemas such as OVER, UNDER, and IN-OUT. Most of these have been shown to structure source domains underpinning conceptual metaphor, especially spatial and primary metaphors (e.g., Gibbs 1994, 2005; Grady 1997; Johnson 1987; Johnson 2007; Lakoff 1987; Lakoff and Johnson 1999; Langacker 1987; Mandler 1996; Mandler 2005; Sweetser 1998).

An example of a fundamental schema is the PATH schema, which consists of a beginning (SOURCE), a vector constituting a PATH, and an end point (GOAL). Figure 1 shows a static visual representation. This image-schematic structure may be instantiated across different domains of experience, for example, by traveling from one city to another, scoring a goal in a soccer match, climbing up a winding trail all the way to the top of a mountain, or gesturally drawing a diagrammatic itinerary of the same hiking experience in the air. Due to the schema’s internal structure, language users may profile one of the constitutive parts of a given event. For instance, in John left England, the beginning of the journey is focused upon; yet we may metonymically infer that the trip involved a PATH (i.e., the itinerary) and a GOAL (i.e., the destination or some other end point). In John traveled through the Chunnel to Paris, PATH and GOAL are profiled (Evans and Green 2006: 185).

Figure 1:

Static visual representation of the SOURCE (A) – PATH (vector) – GOAL (B) schema (Johnson 1987: 28).

Image-schematic patterns may also structure the understanding of domains that are more detached from physical experience, such as the passing of time, beliefs, mental states, and abstract thought, among others. The PATH schema also underpins, for instance, the metaphors LIFE IS A JOURNEY and LOVE IS A JOURNEY (Lakoff and Johnson 1980), thus projecting its internal structure, consisting of points of departure and arrival as well as a route linking them, onto the target domains LIFE or JOURNEY, respectively. Here the image schema provides the basis, or backbone, of more complex semantic structures, such as frames (Fillmore 1982); for instance, the related TRAVEL frame is mapped onto the LOVE frame and thus opens access to all the possible, implied, culturally shaped elements, roles, and processes (e.g., Dancygier and Sweetser 2014: 23–24).

The PATH schema has also been shown to lend structure to form constitution and meaning-making in visuo-spatial modalities and thus also in gesture (e.g., Forceville and Jeulik 2011; Forceville and Urios-Aparisi 2009; Hampe 2017). Consider the following gestural manifestation of the PATH schema that was spontaneously created and integrated in a multimodal discourse sequence in American English.^[2] When talking about her habit of watching a particular sitcom, the speaker in Figure 2 specifies a certain time period within a given season: “from the point of where I was till like the end of the season.” Here the PATH schema underpins the portrayal of a bounded temporal phase (BOUNDEDNESS) reaching out from one point (SOURCE) to a subsequent point in time (GOAL); in this way, the metaphor TIME IS SPACE is manifested as a HORIZONTAL fairly STRAIGHT movement extension evolving from LEFT to RIGHT (seen from the speaker’s perspective). Here, the gesturally evoked, spatial construal of the speaker’s viewing experience is quite specific as it profiles all three parts of the schema (which are also mentioned in the concurrent verbal utterance): the SOURCE (“from where I was”), the PATH through time (“till like”), and the GOAL (“the end of the season”). In doing so, she designates the point of departure with her left, almost vertically held, open hand and the end point with her open right hand, which is likewise held vertically, thus reinforcing the idea of a bounded space. Hence, the latter configuration can also be said to allude to the CONTAINER schema (see also Calbris 2008 on gesturally evoked timelines). This multimodal explanation evokes several embodied patterns in a particularly precise fashion. However, neither in gesture nor in other modalities are image schemas always so fully instantiated; rather, only certain parts and gestalt aspects may be metonymically profiled, thus alluding to the full schema or gestalt.

Figure 2:

Gesture integrating several image schemas; PATH, EXTENSION, STRAIGHT, BOUNDEDNESS, CONTAINER, HORIZONTAL, and LEFT-RIGHT (“from where I was till like the end of the season”).

In one of the first studies on image schemas in gesture, (Cienki 1998a, 1998b) offers detailed discussions of different instantiations of the schemas STRAIGHT and PATH and their metaphoric extensions in language and gesture. In conversations about cheating during exams, North-American college students were found to conceptualize morally right behavior through their gestures, for instance, as movement along a STRAIGHT line. Instantiations of the PATH schema may generally include both linear and nonlinear variants. Additional insights come from a study on dynamic gestural representations of grammar and linguistic theory in teaching contexts (Mittelberg 2006). For example, alluding to a linear model of a sentence, a linguist drew a sketchy horizontal line in the air. An alternative technique evoked the PATH schema by lining up three points in gesture space that stood for subject, verb, and object respectively and thus built a simple syntactic structure with a beginning, an extension, and an end (Mittelberg 2008). When explaining the “nonlinearity” that is typically involved in second language acquisition processes, a linguist made a gestural movement evoking a nonlinear path (Mittelberg 2010). As Williams (2008) has observed, in the process of teaching children how to tell the time, the PATH schema may also be reflected in a teacher’s circling hand movements around a clock face that functions as a material anchor (Hutchins 2005). Núñez (2008) further stresses the central function that image schemas and fictive motion play in teaching mathematical concepts.^[3] Furthermore, Cienki (2005) has demonstrated that the PATH and other basic images schemas (e.g., OBJECT, CONTAINER, CYCLE, and FORCE) may also be used as descriptors in gesture interpretation. Another crucial observation is that gestures may provide interlocutors with semantic cues that are not necessarily identifiable in the concurrent speech.

2.2 Force gestalts

Force dynamics (Talmy 1988, 2000), too, assume a constitutive role in the way that we structure and understand experiences which tend to cut across physical, cognitive, social, emotional, kinesthetic, and aesthetic domains (e.g., Gibbs and Colston 1995; Johnson 1987, 2007; Krois 2011; Turner 2006). Dynamic patterns of this sort include sensing forces within one’s own body or acting on it, for instance, when being pushed sideward by the wind while walking (DIVERSION), trying to keep an upright posture while riding a bike (BALANCE), pushing a bike uphill (GRAVITY), or not being able to PUSH a heavy door open (RESISTANCE, BLOCKAGE). Johnson (1987: 42; italics in the original) reminds us that:

[…] we easily forget that our bodies are clusters of forces and that every event of which we are part consists, minimally, of forces in interaction […]. We do notice such forces when they are extraordinarily strong, or when they are not balanced off by other forces.

While the force-dynamic system is rooted in our bodily morphology, affordances, and experience, that is, kinesthesia (i.e., a sense of movement through muscular effort), somesthesia (sensing pressure or pain), and proprioception (i.e., a sense of the position of body parts and bodily movement in space, including a sense of gravitational orientation; Sheets-Johnstone 1999/2011: 73), it may also relate to psychological and social forces through metaphorical construal and certain grammatical constructions (e.g., Dancygier and Sweetser 2014; Evans and Green 2006: 199; Gibbs 2005: 119).

While force dynamics have not yet received as much attention as image schemas in gesture research (e.g., Cienki 2005), the spatiotemporal properties of gestures seem to offer great potential for gaining empirical insights into deeply embodied dimensions of force gestalts and how they interact with image-schematic structures. Furthermore, metaphorical extensions of force gestalts may not only portray social pressure or emotional burden but also drive gestural portrayals of the speaker’s subjective stance towards what s/he is talking about, for instance, by pushing away (BLOCKAGE, RESISTENCE) an idea put forth by an interlocutor (e.g., Bressem and Müller 2014; see also Calbris 2011 on the semantics of physical refusal via the recruitment of reflexes and Wehling 2017 on discourse management gestures).

In the following example (adapted from Mittelberg 2010), a linguistics professor is explaining the theory of emergent grammar in the context of second language acquisition. She holds that the boundaries between grammar, seen as a dynamic system, and language learning become fused due to discourse-pragmatic forces: “it sees the use and grammar together… grammar coming out of the use… you learn by doing rather than you learn first the rules and then you do them… [it blurs the boundary] between… learning and doing.” As shown in Figure 3, the speaker produces a relatively forceful gestural action consisting of suddenly pushing both hands, initially held more than shoulder-width apart, towards each other. Through the comparably high level of energy and dynamicity exhibited in this gestural portrayal, the ideas expressed in speech are vividly enacted and draw on the force and image schemas PUSH, MERGE, PROCESS, and BOUNDARY.

Figure 3:

Gesture enacting the blurring (MERGE, PROCESS) of a BOUNDARY and the force gestalt PUSH.

3 Embodied schemas as emergent experiential gestalts recreating the kinesthetic feel and structure of embodied action

As emphasized in the forgoing discussion, the present view of spontaneously produced communicative action puts a strong focus on its experiential motivation and the constitutive role played by image schemas and force gestalts. Hence it is allied with theoretical accounts of image schemas that do not just assume a bodily basis of conceptual schemas (e.g., Hampe 2005; Oakley 2007), but claims that such patterns retain their experiential bodily origins to a great and perceivable extent. Johnson (2005: 31), for instance, advocates that:

[…] the truly significant work done by image schemas is tied to the fact that they are not merely skeletons or abstractions. They are recurring patterns of organism-environment interactions that exist in the felt qualities of our experience, understanding, and thought. Image schemas are the sort of structures that demarcate the basic contours of our experience as embodied creatures.^[6]

Besides the dynamic dimensions of image schemas, Mandler and Pagán Cánovas (2014) have stressed the important role that language development plays in relation to preverbal schemas and spatial primitives.

Gibbs, in particular, favors a strongly embodied and action-based view of image schemas, seeing them as emergent “‘experiential gestalts’ or ‘as-if body loops’ (Damasio 1994) that are actively created on-the-fly during different cognitive activity” (Gibbs 2005: 119). Building on earlier work, notably by Gibbs and Colston (1995), he has challenged the idea of image schemas as mental representations stored in long-term memory (Gibbs 2005: 114–115). Drawing on the theory of dynamic systems (e.g., Freeman 2001; Thompson and Varela 2001), he argues instead that image schemas can be seen as “in-the-moment embodied simulation” and “attractors within human self-organizing systems” (Gibbs 2005: 131). The main tenets of Gibbs’ view will now be presented in some detail, then adopted and used productively in the domain of gesture in the ensuing sections. According to Gibbs, image schemas are:

[…] simulators of real-life actions or potential actions that a person might engage in. As a simulator, image schemas provide a kinesthetic feel that is not simply the output of some abstract computational machine, but the results of full-bodied experiences that have textures and a felt-sense of three-dimensional depth. (Ibid.: 119, my emphasis)

I suggest that some of the properties of image schemas emphasized by Gibbs bring to mind some of the essential facets of the gestural phenomena explored in this paper. This pertains first and foremost to the fact that manual gestures and whole-body postures, regardless of whether they evoke literal or metaphorical meanings, have a kinesthetic feel to them, may simulate witnessed or imagined actions, articulate tactile, affective, and intersubjective dimensions of experience, and naturally evolve dynamically in 3D-space (e.g., Müller 2017b; Sheets-Johnstone 1999/2011).^[7] In order to pinpoint some of the assumed correspondences in structure and function between embodied experiential gestalts and coverbal gestures, let us consider some of these attributes in more depth. Gibbs (2005) further characterizes images schemas as:

1. “emergent fleeting entities that are part of embodied simulation used in online thought including abstract reasoning” (p. 114)

2. “temporary linkages between sensory experience and short-lived conceptualizations of both concrete events and abstract events” (p. 114)

3. “continually tied to embodied action and simulations of experience” (p. 116)

4. “connect(ing) the domains of embodied action with the domain of linguistic action” (p. 123)

5. “emergent properties of human self-organizing systems that are continually recreated and re-experienced during cognitive and perceptual activity” (p. 131)

6. “attractors within human self-organizing systems. Attractors, such as BALANCE, SOURCE-PATH-GOAL, RESISTENCE, VERTICALITY, etc., reflect emerging points of stability in a system as it engages in real-world interaction” (p. 131)

The point I wish to make here is that gestures, in their naturally embodied and modality-specific ways, may not only be said to instantiate image schemas and force gestalts but also to exhibit properties similar to those that Johnson and Gibbs ascribe to them. It is thus assumed that gestures enact deeply embodied patterns of experience and may thus help both speakers and listeners to construct meaning and mutual understanding grounded in dynamically evolving image-schematic structures that evoke on-the-fly sensorimotor experience (Gibbs 2006). Hence, pragmatically driven functions of image schemas also seem to be operative in the production of multimodal utterances, that is, “linguistic action” (Gibbs 2005: 123), in which gestures allow speakers to relive experiences while describing them (see also, e.g., Cienki and Müller 2008; Müller 2017a,Müller, 2017b; Müller and Ladewig 2013 on metaphor in gesture). Accordingly, gesturally evoked image schemas and force gestalts can be seen as momentary, emergent, mediating structures that give some stability, or regularity, to people’s communicative movements. As such, they may be said to underpin certain recurring hand configurations and movement patterns within a given discourse context (e.g., Mittelberg 2006, 2010) or discourse culture (see, e.g., Bressem and Müller 2014; Ladewig 2011, 2014; Müller 2017b on recurrent gestures). Partaking in such multimodal processes of meaning-making, gestures may emerge as precise articulations of the essential structures of a dynamic experience (as in Figure 2), or as a profile of an experiential gestalt, that is, a “rough approximation of the image-schematic structures” (Gibbs 2005: 125). As such, a gestural structure may be used, for example, as a source domain for a metaphoric construal (as in Figure 3). In the next section, we will consider an array of tendencies in the gestural enaction of image schemas and force gestalts in which the human communicating body takes center stage.

4 A body-centered perspective on gesturally enacted image schemas and force gestalts inspired by dynamic systems

As argued in Section 3, Gibbs’ (2005, 2006) dynamic systems view of image schemas – which highlights their function as attractors that reflect stable points in self-organizing systems engaged in real-world interaction – seems to offer a particularly apt blueprint for describing certain tendencies in emergent gestural manifestations of such strongly embodied, dynamic patterns (see also Dewell 2005). Following Gibbs’ (2005) rationale a bit further down the line, the following exposition is inspired and broadly structured in accordance with the three kinds of cycles that are purported to underlie the self-organization of organisms qua dynamic systems (Thompson and Varela 2001): (a) “organismic regulation of the entire body;” (b) “sensorimotor coupling between organism and environment;” and (c) “intersubjective interaction involving the recognition of the intentional meaning of action and linguistic communication” (Gibbs 2005: 131).

Given the complexity and variability of human semiotic behavior, the goal of the present proposal is neither to offer a neat typology of image schemas and force gestalts, nor to cover all their possible gestural manifestations. As initially stated, it is not assumed that all gestures manifest image and/or force schemas. Rather, certain tendencies in the gestural enaction of these schemas are suggested, taking into account the fact that schemas interact flexibly in dynamic and transient ways. Related questions regarding the universality or culture-specificity of image schemas and how these factors may feed into various metaphoric and metonymic processes cannot be treated in detail here. What follows is a preliminary attempt at using a dynamic systems perspective to account for how gestures may enact image schemas and force gestalts. The groupings presented below are assumed to reflect some of the essential ways in which human beings experience their own corporeal embodiment as semiotic beings (e.g., Gallagher 2005) when interacting with the environment and other people in communicative situations.

4.1 Body-inherent and body-impacting image schemas and force dynamics

The human body unites several basic image schemas and force gestalts through its very morphology and the ways in which it moves through the world. In what follows, some of the fundamental body-inherent schemas and body-impacting forces will be discussed. While they typically interact in instances of meaning-making and cannot always be easily teased apart, they will be presented in two groups in order to highlight some of their specific functions with respect to gesture. Here a connection is drawn to the first cycle purported to underlie dynamic systems: “organismic self-regulation of the entire body” (Gibbs 2005: 131; Thompson and Varela 2001).

4.1.1 The human communicating body as an image-schematic structure

OBJECT, PART-WHOLE

CONTAINER, CONTAINMENT (CONTENT, IN-OUT)

SURFACE, SUPPORT, CONTACT, ADJACENCY

The body is a semiotically versatile PART-WHOLE structure, parts of which, such as the head, the arms, and hands, as well as the entire torso or body may be recruited as sign carriers for the purpose of gestural communication. Gesturing hands may thereby metonymically stand for the entire gesturing person or be employed to portray the behavior of objects or people, for instance, as witnessed in a movie or in a scenario in which the speaker her/himself has participated (see, e.g., Dudis 2004 on body partitioning in ASL; Calbris 1990 on body segments; Sweetser 2012 on gestural viewpoint strategies). As research on metaphor in language has shown (e.g., Gibbs 2006; Lakoff and Johnson 1980, 1999), the body can be conceived of as a CONTAINER, with an inside and an outside (IN-OUT), that holds CONTENT, for example, food, organs, and fluids. Its SURFACE characteristics may indicate certain emotional or mental states via metonymy. The head may also stand for its CONTENT; for example, pointing at the head may refer to the mind inside it (see, e.g., Calbris 2011 on body-focused gestures; Mittelberg and Waugh 2014 on body-part indices). In addition, the body may lend SUPPORT to other people and to entities (as a whole or via the hands holding someone or something). It may thus create contiguity relations, such as ADJACENCY and BALANCE, with other persons or with objects and spatial structures that may become meaningful through metonymy and pragmatic inferencing (e.g., Mittelberg 2017b).

4.1.2 Body-inherent schemas and force dynamics acting upon the human body

BALANCE, VERTICALITY, STRAIGHT

GRAVITY, RESISTANCE

COMPULSION, COUNTERFORCE, BLOCKAGE, RELEASE

ATTRACTION, REPULSION, PUSH-PULL

Let us now consider one of the most basic embodied schemas that involve the body in its entirety. BALANCE belongs to the elementary schemas that “operate in our bodily movements, perceptual acts, and orientational awareness” (Johnson 1987: 75). It “requires an ordering of forces and weights relative to some point, axis, or plane” (ibid.: 99), as graphically represented in the equilibrium schema provided in Figure 4.^[8] We can sense BALANCE in our own bodily posture and inner states, or the lack thereof, as well as in those of other people and in diverse domains of experience such as artworks (e.g., paintings, buildings, and musical pieces), theories, or social relationships (e.g., Johnson 1987, 2007; Krois 2011; Lakoff 2006; Mittelberg 2002). Standing up straight is in and of itself an essential bodily experience of human existence, regulated by various body schemas, such as VERTICALITY, STRAIGHT, and BALANCE (e.g., Gibbs 2005). Interestingly, as shown by experimental work (Gibbs et al. 1994), these schemas have been found to combine in different ways, thus underpinning various related meanings, both literal and metaphorical, of the polysemous verb stand. What the different meanings all have in common is their basis in physical experience and understanding. Perhaps not surprisingly, these schemas may also structure and balance gestural enactments from within the body.

Figure 4:

EQUILIBRIUM schema (Johnson 1987: 86).

In a study of BALANCE and related schemas in paintings by Paul Klee and their multimodal descriptions (Mittelberg 2013b), the participants showed a strong tendency to ‘become’ the figures depicted in the images by enacting aspects of the figures’ spatial positioning and postures they observed in them. In all the images used for the study, human-like figures are standing and trying to stay balanced despite various forces acting on their bodies. Each painting was shown for 60 seconds on a computer screen; then the participants were asked to recall and describe the composition of each painting as well as the effect it had on them. My intent here is to point out a few central aspects regarding the body-inherent (and interacting spatial) image schemas that may be said to be at work in the image space (e.g., Johnson 1987, 2007; Krois 2011) and also in the body of the study participant describing in gesture space what was recalled.

By way of example, let us consider the Dance of a Mourning Child (1922), which was used as a stimulus. The dancing child (Figure 5 (a)) is positioned off-center (CENTER-PERIPHERY) to the left side (LEFT-RIGHT). Not holding its oversized head up straight (STRAIGHT, VERTICAL, UP-DOWN), but tilted to one side, the child seems to be giving in to GRAVITY. She is trying to COUNTERBALANCE the weight of her large head by pointing her fully extended arms in the opposite direction, and by holding on to some kind of rope with both hands. She thus manages to create a BALANCED composure, which also seems to be maintained by the flow of the dance movement hinted at via the flowing skirt. The heart-shaped mouth functions as the anchor point of both the figure of the child and the image as a whole. According to Lakoff (2006: 154), “image schemas […] give form to art” in the sense that our embodied, enacting minds impose image schemas onto visual scenes. In this way, spatial and orientational relations as well as motor control schemas are understood to link perceptual to conceptual structure, and thus link form to meaning, including rather schematic meaning. Interestingly, the study participants picked up on image-inherent schemas, resulting in bodily articulations that convey correspondingly similar schematic meanings (see Mittelberg 2013b for details).

Figure 5:

(a)–(c). P. Klee, Dance of a Mourning Child (1922); enactment of the child’s posture; motion-capture plot of isolated gestures.

As shown in Figure 5 (b), the participant mirrors the figure’s posture with her whole body and thus also reflects the particular distribution of weight along the vertical axis.^[9] She imitates the figure’s tilted head and extended arms, as well as the position of the legs and the eye gaze directed downward. After transposing the image of the figure’s mouth onto her own lips and that of the figure’s eye slits onto her own eyes, she evokes the flowing skirt around her hips and upper legs by moving her hands repeatedly back and forth. The motion-capture data plot provided in Figure 5 renders these semiotic practices visible, thus allowing us to see the set of traces evoking the flowing skirt in the lower segment of gesture space, the traces of outward-going hands enacting the extension of the arms a little higher up, and the iconic forms depicting the mouth and eye slits in the upper segment. What is particularly insightful in terms of balance is that each of these bimanually produced pairs of signs exhibits features of an internal dynamic symmetry: they are thus also balanced in their own way.

This group of embodied schemas includes additional kinds of image schemas and force dynamics that may act within or on a person’s body and that are recruited for communicative purposes. For example, the image schema STRAIGHT may also motivate formational properties of hand shapes such as finger curvature (Mittelberg and Evola 2010). Flat, completely outstretched hands can be expected to create a marked, signifying condition of gesturing hands, as opposed to the unmarked condition of relatively relaxed hands. By way of illustration, let us consider the example of a multimodal metaphoric expression given in Figure 6. The speaker is describing how her former schoolmates, even though they were only just graduating from high school, had already neatly planned their entire lives “in preset ways” (German transcript: “in geregelten Bahnen”). She underscores this verbal metaphor with a gesture (Figure 6 (a)) using both hands, almost in parallel, with open flat palms facing downward and the fingers spread, pointing straight ahead and moving slightly away from her body in the center of the gesture space, thus creating a vector path (Evola 2010). In the graph in Figure 6(b), the vertical bar indicates that the angle of the right middle finger joint was 0° just before the speaker says “in preset ways.” Based on a qualitative analysis, the instantiated image schemas are STRAIGHT (hand shape), PATH (finger orientation and hand movement), and CENTER-PERIPHERY (placement of gesture). Via a metaphorical extension, the straightness of the hands stands for the ‘right’ way in life according to the speaker’s school mates’ view (STRAIGHT IS RIGHT, Cienki 1998b). The metaphor LIFE IS A JOURNEY manifests itself in the indicated PATH and its direction.

Figure 6:

(a)–(b). The speaker expresses the metaphor of ‘preset ways’ (“geregelte Bahnen”) via hands motivated by the image schema STRAIGHT and interacting with the schemas PATH and CENTER-PERIPHERY; the bar indicates the 0° angle of the finger joint of the right middle finger.

More empirical research combining qualitative and quantitative methods is needed in this domain, especially regarding force dynamics, but it seems safe to say that speakers may readily use their heads, manual gestures, or postures to (re)enact force qualities they have experienced or imagine experiencing while talking and interacting with their interlocutors (e.g., Wehling 2017).

4.2 Speaker-environment interactions (A): manipulating objects and schematizing events

We now turn to how people’s gesturing hands may evoke basic image schemas when they are pretending to interact with the physical environment, especially with material culture. This includes describing real-world, abstract, and/or imagined actions, scenarios, or events, including movement through space that speakers themselves, others, or objects have experienced or could experience. These phenomena are broadly seen as pertaining to the second cycle of self-organizing systems, namely, “sensorimotor coupling between organism and environment” (Gibbs 2005: 131; Thompson and Varela 2001), and thus shift our focus to how speakers often pretend to be manipulating or creating objects and/or tools while talking. As stated above, individual image schemas and force gestalts may partake in more than one of the tendencies in gestural instantiations of image schemas discussed here. As we will see, examining communicative bodily action may enable us to reveal how static and dynamic properties of the image-schematic structures involved may interact.

4.2.1 Schemas underlying the handling, creation, or analysis of (virtual) entities

OBJECT, CONTAINER

SURFACE, SUPPORT

PART-WHOLE, ADJACENCY, CONTACT, LINK

PUSH-PULL

When people talk, they often seem to be holding, moving, or otherwise manipulating (virtual) physical objects and/or tools (e.g., Müller 1998; Streeck 2009; Streeck, Goodwin and LeBaron 2011). These environment-oriented actions have also been observed when speakers are talking about abstract subject matters. Such metaphorical construals may involve not only spatial metaphor and primary metaphor but also more complex metaphoric mappings and networks (e.g., Calbris 2003, 2011; Cienki and Müller 2008; Núñez, 2008; Sweetser, 1998). Much of the inspiration for the present proposal comes from my own earlier work on image schemas, metonymy, and metaphor in gestures and multimodal explanations of grammar and linguistic theory (Mittelberg 2006). The larger proportion of the recurring hand shapes and motion patterns observed in a corpus of recorded linguistics courses turned out to be reminiscent of image-schematic patterns, ranging from PATH, OBJECT and CONTAINMENT to LINK and PART-WHOLE, to name just the most prominent ones (Mittelberg 2010; see also Section 2.1).^[10] It was found that all the participating linguistics professors were pretending to manipulate physical objects and structures while they were talking about linguistic categories, functions, and units of varying degrees of complexity. The handling routines they employed included simply holding generic objects with one or two hands, seemingly taking them apart, changing the order of the constituents in a linear sentence model or in a hierarchical tree structure (e.g., Mittelberg 2008). I here reiterate the original claim that most of these mental and manual activities draw on a range of basic, interacting image schemas and also some force gestalts (e.g., the forceful PUSHing gesture portraying the idea of blurring a BOUNDARY shown in Figure 3).

To further substantiate this claim, let us consider the example shown in Figure 7a. Explaining the framework of emergent grammar, the speaker points out that “a priori… you cannot define a noun from a verb.” On the mention of the word “noun,” this “palm-up open hand gesture” (PUOH, Müller 2004) constitutes a perceivable SURFACE used as a physical SUPPORT structure (Mandler 1996) on which the speaker seems to be presenting the abstract category reified as a metaphorically construed, graspable entity (EXISTENCE). In this way, a virtual object seemingly sitting on the open hand (CONTACT, ADJACENCY) may become the center of attention in the physical and pragmatic context shared by the teacher and her students (Mittelberg 2008). While the speech is nonfigurative and the idea of SUPPORT or SURFACE is neither explicitly mentioned nor implied in what the linguist is saying, the gestural portrayal can be said to manifest the image schemas SUPPORT, ADJACENCY, and OBJECT, as well as the primary metaphor IDEAS ARE OBJECTS (Grady 1997). While the SUPPORT schema may be assumed to be inherent to most PUOH gestures, which provide a flat SURFACE for presenting imagined discourse contents, exactly how this schema is applied and interacts with other schemas needs to be determined for every local use of this highly polysemous gestural form (see also Bressem and Müller 2014; Müller 2004).^[11]

Figure 7:

a&b. Palm-up open hand instantiating the SUPPORT, SURFACE, and OBJECT schemas, co-occurring with the mention of “noun;” cupped (left) hand instantiating the CONTAINER schema, co-occurring with the mention of “main verb.”

Positing a continuum between concrete and abstract domains of experience, Calbris’ (2003) thorough account of preconceptual schemas in gestures produced by French speakers reveals that schemas underlying common manual actions performed on objects, such as cutting food, may be projected onto abstract domains of reasoning and motivate gestural construals; for example, cutting movements may thus represent the intellectual process of carrying out an analysis (see also Calbris 2011).

Another basic image schema is the CONTAINER or CONTAINMENT schema (e.g. Johnson 1987), which may be easily evoked through gesture. For instance, while saying “there is… what’s called the main verb,” the speaker in Figure 7b produces a cupped palm-up open hand with his left hand, thus imitating the form of a small round container. Drawing on the affordances of a single hand, the gesture iconically and schematically instantiates the CONTAINER schema and reflects the metaphoric understanding CATEGORIES ARE CONTAINERS or CONCEPTUAL STRUCTURE IS PHYSICAL STRUCTURE (Sweetser 1998). The idea of CONTAINMENT can also be evoked in gesture by using two hands to hold or to group imaginary entities. First explorations of a multimodal corpus enriched with motion-capture data revealed some regularities in speakers’ hand shapes and movements. Regularities of this kind are here assumed to stem, at least partly, from the specific image schemas and force gestalts that operate in – and thus structurally and pragmatically organize – a given gestural performance. For instance, the use of the curvature of the hands and the distance between them to seemingly group discourse items was found to reflect basic image schemas such as OBJECT and CONTAINER; these discourse-structuring gestures (Müller 1998) often simultaneously draw on the BALANCE schema as well as spatial relations such as LEFT-RIGHT or CENTER-PERIPHERY (Mittelberg and Evola 2010).

In the following example, we can witness how the speaker uses her hands and the space around her in an intuitive yet precise fashion. While explaining that her Master’s program in Speech Therapy consisted of two major subject matters (speech and therapy), she first lays out what was covered in each domain and then goes on to point out that, all in all, the program offered a broad spectrum of topics.^[12] In the gestural modality, each of the two study domains is depicted in the form of a midsized container or bounded space (Figure 8, container₁ and container₂). The numeric motion-capture data were used to measure the respective positioning of the two gestures in space, which revealed that the second ‘container gesture’ was placed right next to the first (ADJACENCY). In this way, the two different adjacent positions correlate with two differently semanticized bounded regions in space. Furthermore, in the first two occurrences of these bimanual gestures, the distance between the two hands is almost identical (first 33 cm, then 33,5 cm). In the third, larger gesture representing the entire course of study, the distance between the two hands (67 cm) encompasses both chunks of space delineated by the two preceding gestures (PART-WHOLE). This process of spatial addition is indicated by the yellow arrows overlaying the three stills (Figure 8 (a)–(c)). The decisive moments of delineation are highlighted by the first three vertical green bars in the graph provided in Figure 9; the x-axis represents the timeline; the y-axis indicates the distance between the markers attached to the middle finger of each hand.

Figure 8:

(a)–(c). Three gestures evoking the CONTAINER schema, illustrating the domains of the speaker’s course of study: (a) linguistics; (b) therapy; and (c) the whole spectrum.

Figure 9:

Graph showing the amplitude of the three bimanual ‘container gestures’ indicating the respective distance between the two hands (see Figure 8 (a)–(c)).

This example hence shows a speaker delivering an astonishingly balanced multimodal performance. It also seems to suggest that a sort of pragmatically employed common-sense geometry (Deane 2005; Mittelberg 2010) is factoring in here. These insights clearly call for further investigation of the possible regularities in hand configurations and the use of space and how these aspects correlate with certain cognitive-pragmatic functions of the multimodal utterances in question.

4.2.2 Schemas underlying descriptions of objects/persons and their movement behavior

OBJECT

PATH (STRAIGHT, CIRCLE, SPIRAL, HORIZONTAL)

DIVERSION, COMPULSION

RESISTANCE, BLOCKAGE, REMOVAL OF CONSTRAINT

Visual perception and other ways of understanding the people and the things in one’s surroundings offer additional ways of interacting with the environment, here understood to include our semiotic environment constituted by the various media we regularly use (e.g., newspapers, TV, and movies). This broad array of phenomena can obviously not be fully taken into account here. Instead, the focus is placed on the pervasive OBJECT and PATH schemas that can be said to underlie the motion events amply discussed in the gesture literature (e.g., McNeill 2000; see also Section 2.1).^[13] For example, when recounting action sequences encountered in animated cartoon stories, speakers of a variety of different languages have been shown to depict the PATH and/or the manner of movement of cartoon characters or objects by producing iconic, in this case, rather schematic, and typified gestural traces – for example, straight or wavy lines extending along horizontal, diagonal, or vertical axes, circles, etc. It is noteworthy that while the literal renditions of the observed behavior of movie characters or real-life persons and objects involve schematization, they do not rely on a metaphoric construal operation in the way that the portrayal of abstract ideas often does, for instance, in the PATH example in Figure 2 or the gestural portrayals of morally correct behavior described by Cienki (1998a, 1998b; see also Mittelberg in press for more examples based on motion-capture data).

At this point I would like to come back to what was said earlier (in Section 2.4) regarding the tendency of schemas to interact flexibly and dynamically. In this connection, Gärdenfors (2007: 59) argues that instantiating spatial image-schematic structures often involves action and thus “forces exerted by an agent:” movement along a PATH, for example, may involve an agent creating momentum and locomotion. This aspect is relevant for gestural action depicting motion events that involve force dynamics. Here we also need to consider viewpointed meaning construction. For example, as speakers, we either depict what we have observed from afar – from an external perspective – or we immerse ourselves in the scenario and describe it – or our own actions – from an internal perspective (e.g., Mittelberg 2017c; Parrill 2009; Sweetser 2012). In the gesture conveying aspects of the speaker’s sitcom viewing experience (Figure 2), we saw how instantiations of the SOURCE-PATH-GOAL schema dynamically interact with other schemas and spatial metaphor (see also Mittelberg 2010: 370). In gesture, the PATH schema may be instantiated in various ways and approximations. But it indeed seems that, as pointed out by Gibbs (2005: 131), the topographic structure of a PATH emerges across single instantiations as a configuration of stable properties that are inseparable from sensorimotor activity. In this regard, we also need to take into account that gestural movement through space necessarily and genuinely exhibits an inherent directionality, whose indexical dimensions sometimes determine the main function of a gesture, as in the case of deictic gestures or route descriptions (e.g., Fricke 2007). In other semiotic processes, they may serve to create an iconic figuration, such as a path, that stands for something else in its own right (e.g., Mittelberg and Waugh 2014).

The idea that “by adding force dimensions to an image schema, we obtain the basic tools for analyzing dynamic and functional properties” (Gärdenfors 2007: 73) seems particularly captivating in relation to gesture.^[14] When we gesture, we may generally assume the role of someone who is being acted upon by external forces or who is exerting, or pretending to exert, force on imaginary entities or persons. We can also gesture less or more forcefully; accordingly, velocity peaks can be expected to add signifying qualities to the content expressed in a given gesture. As we will see below (Figure 10), motion-capture technology allows one to measure such qualities in gestural articulations.

4.2.3 Processes

PROCESS

REITERATION

PATH, STRAIGHT, CYCLE, SPIRAL

FORCE

Since gestures are in and of themselves sensorimotor experiences that require time and space to evolve, they seem particularly apt at allowing speakers to describe and also propel cognitive activity (e.g., Cienki and Müller 2008; McNeill 1992, McNeill 2005; Müller 2017b). Thus a subgroup is needed here to capture the contours of experiences that have the character of dynamic, time-consuming processes. These may evoke gestures that are quite different from those simulating object manipulation or delineating spatial chunks that metaphorically stand for abstract entities. PROCESS is an image schema that may take on different gestalts, depending on the image schemas and force dynamics it interacts with to construe a given experience. For example, the STRAIGHT PATH gesture in Figure 2 portrays a PROCESS construed through the aspectual framing of an activity (sitcom watching) as progressive and bounded. The blurring of a BOUNDARY portrayed by the linguist in Figure 3 involves a FORCEfully conditioned PROCESS; the gestural framing of ongoing PROCESSes also includes CYCLES and SPIRALs drawn in the air (Beecks et al. 2016; Mittelberg and Hinnell 2015; Mittelberg in press; Schüller et al. 2017).^[15]

Image schemas have also been found to motivate other intransitive hand movements that enact the idea of PROCESS without pretending to perform transitive actions. For instance, Ladewig (2011) has found that the CYCLE schema may underlie several variants of the cyclic gesture. Depending on the exact manner of execution and the placement viv-a-vis the gesturer’s body as well as the interlocutors, rotary hand movements may express aspects of continuation, repetition, or word searching. Gestures instantiating the CYCLE and PROCESS schemas have also been shown to express the operation of REITERATION in grammar and the activity of language use (Mittelberg 2006, 2010).^[16] Furthermore, Streeck (2009) correlates the gestural practice of ceiving (i.e., gestural thinking) with the speaker’s recruitment of an image schema when trying to find a gestural image for an emerging idea.

4.3 Speaker-environment interactions (B): pre-structuring of gesture spaces

UP-DOWN, LEFT-RIGHT, FRONT-BACK, NEAR-FAR

CENTER-PERIPHERY,

VERTICAL, HORIZONTAL

Building on the phenomena of organism-environment interactions discussed in Section 4.2, we will now discuss in more detail the spatial-relation schemas (e.g., Lakoff and Johnson 1999) that may be involved in gestural performances and instill them with schematic meaning. Given gestures’ specific kinetic and visuo-spatial mediality, spatial-relations schemas and topological schemas (Talmy 2000) are of particular interest here. Such schemas have been found to underpin speakers’ systematic uses of certain segments of gesture space, for instance, to structure their discourses by seemingly ‘placing’ ideas to respective sides of their bodies, or to highlight spatial relations between ideas, locations in space, or moments in time. For example, gestural references to events in the past or future may evoke the FRONT-BACK or the LEFT-RIGHT schemas, depending on the respective, culturally dependent metaphoric construals (Calbris 2008; Cienki 2013; Núñez and Sweetser 2006). In addition, the left and right side can be associated with different values in right and left handers (Casasanto 2009); along the UP-DOWN scale, upward motion has been found to signal that a positive valence is attached to the experience being talked about (Casasanto and Dijkstra 2010). It has also been shown that gesture space is used to metaphorically represent numbers and quantity, for example, MORE IS UP, MORE IS BIGGER, and MORE IS RIGHT (Winter, Perlman and Matlock 2014). In addition, social and abstract hierarchies can translate into spatial hierarchies: a common practice in linguistics courses is to trace syntactic tree structures in the air, whereby dominating nodes are located above the elements they govern (Mittelberg 2008).

These spatial-relations schemas may thus be said to virtually prestructure gesture space, including interpersonally shared spaces, into culturally shaped semanticized regions. In this way, they may motivate the ways in which speakers direct, orient, and position their gestures in relation to their viewpointed bodies (e.g., Priesters and Mittelberg 2013; Sweetser 2012). For example, the speaker shown in Figure 6 went on to report in the same narrative that she eventually decided to leave the originally intended career PATH and to move completely away from therapy: “in the end it has turned out, of course, that I moved totally away from therapy” (German transcript: “im Endeffekt ist es natürlich so gekommen, dass ich von der Therapie total weg bin;” see Figure 10). As in the gesture expressing the idea of ‘predetermined paths’ (Figure 6), the PATH schema here is not evoked by a horizontal trace gesturally drawn in the air (as in Figure 2), but virtually projected into gesture space: it originates at the speaker’s Origo (Bühler 1982; Fricke 2007; Sweetser 2012) and runs along the sagittal axis in front of her (into the future) through the CENTER of her gesture space. Moreover, the PATH schema does not just interact with the CENTER-PERIPHERY schema: as shown in Figure 10, on the mention of “total weg” (“totally away”), she PUSHes both hands away from her body (NEAR-FAR) and sideward to the right into the periphery of her gesture space (LEFT-RIGHT).^[17] Exhibiting a velocity peak in relation to her average speed of gesturing, this forceful gestural action (see Figure 10 (b)) vividly expresses the speaker’s personal stance regarding her career decisions.

Figure 10:

(a)–(b). Gesture exhibiting high velocity plus schemas PUSH, PATH, NEAR-FAR, RIGHT-LEFT, and CENTER-PERIPHERY (“I moved totally away from therapy”); graph showing the velocity peak: the left hand pushes sideward faster than the right hand, represented by the deeper amplitude spike.

This last example illuminates Gärdenfors’ (2007) idea that adding force gestalts to image schemas brings out their dynamic and functional characteristics (4.2.2). This seems to pertain not only to rhetorical effects in language use (Oakley 2005; Wehling 2017) but also to interlacing pragmatic functions of gestures, as shown above. In addition, these observations bring to mind McNeill’s (1992, 2005) notion of high communicative dynamism and raise questions about the ways in which image schemas, and force gestalts in particular, factor into multimodal communicative boosts such as the one described above (Figure 10 (a)).^[18]

5 Concluding remarks

A main goal of this paper was to show that gestures, due to their corporeality and pragmatic mindedness, are particularly apt at instantiating spatial and dynamic properties of conceptual structures and processes. Hence it offers further support for the theory of the embodied mind (Gibbs 2006; Johnson 1987; Lakoff and Johnson 1999) and particularly the idea of enactivism (Varela, Thompson and Rosch 1991; see also Krois 2011). With reference to crucial aspects of work on the psychological reality of image schemas (Gibbs 2005; Gibbs and Colston 1995), the gesture-based approach presented here aims to provide further evidence for the semiotic reality of such patterns of multisensory experience (e.g., Danaher 1998; Mittelberg 2008 in press) and advances the use of motion-capture technology to this end. Inspired by the dynamic systems approach to image schemas put forth by Gibbs (2005), the present perspective shifts the focus from language processing and understanding to interpreting spontaneous multimodal language use and to identifying gestural patterns therein. There are several insights we can draw from the foregoing exposition and discussion.

First, gestural action and body postures have a natural and strong propensity to enact deeply embodied facets of image-schematic reasoning by recruiting “sensorimotor processes that are critical to how we understand ourselves, other people, and the world around us” (ibid.: 117) and that are tied to real and imagined bodily actions. Based on the observations made throughout the paper, we might even say that gestures play a crucial role in how “image-schematic reasoning is always being recreated by the body as people continue to engage in sensorimotor behaviors related to BALANCE, RESISTANCE, SOURCE-PATH-GOAL, CONTAINMENT” (ibid.: 116). Through drawing on such minimal, yet meaningful, structures and relations in the give and take of multimodal interaction, gestures may invoke the “kinesthetic feel” (ibid.: 116) of schemas and other construal operations that are not necessarily identifiable in the concurrent speech chain and thus provide additional semantic and pragmatic cues. While the above discussion mostly focused on certain kinds of kinetic manifestations of image-schematic and force-dynamic structures, it is claimed that, consistent with the literature cited, they typically build the schematic essence out of which rich inferences and associations evolve. How they feed into more complex processes of situated, intersubjective meaning-making that involve, for instance, metonymy, metaphor, frames, and grammatical constructions seems to be a promising line of inquiry for future cross-linguistic gesture research (e.g., Cienki 2013; Mittelberg 2017a/b; Müller 2017a/b).

Second, the noted structural similarity between gestures, image schemas, and force gestalts is assumed to be at least partly due to their shared embodied nature and dynamic schematicity. In the data discussed in the present paper, we have seen empirical indications that gestural instantiations of embodied schemas may indeed preserve the cognitive topology of the image-schematic source domains involved in metaphorical mappings, as posited by the invariance hypothesis (Lakoff 1990). These questions also bring to mind the Jakobsonian (1960) terms variance/invariance, according to which individual gestural instantiations of image schemas and force gestalts, which are typically partial and approximate, may be regarded as flexible, context-dependent variants of the invariant topological properties of a schema. An interesting avenue for future research would be to seek to specify, both theoretically and empirically, the exact relation between a given schema and its gestural enactments.

Third, several of the schemas repeatedly dealt with – SUPPORT, CONTAINMENT, PATH, and CONTACT – belong, according to Mandler (1996: 373–378), to the preverbal, spatially structured meaning system. These early acquired schemas also feature in the different kinds of gesture-schema interactions discussed, which were inspired by the three kinds of cycles that pertain to the self-organization of dynamic systems (Gibbs 2005; Thompson and Varela 2001). For present purposes, the latter were transposed into the following tendencies: body-inherent/forces acting upon the body (self-oriented; the body as an image-schematic structure); environment-oriented (material culture including spatial structures), and interlocutor-oriented (interaction with others). From the perspective proposed here, people who are talking can be seen as embodied, self-organizing systems engaged in real-world (or imagined) interactions with their physical environment and with their interlocutors or audience. Gesturally instantiated image schemas and force gestalts allow them to pragmatically organize their multimodal expressions and to structure, at least partly, how they interact and (co-)construct emergent meaning with their conversational partners. After each communicative boost involving a gestural movement excursion, or several there of, a speaker typically gathers her/himself, for example, by resuming a rest position. This seems to be partly afforded by a reordering of the image-schematic structures and forces acting within and upon the speaker’s body, and within the interactively formed communication space shared with her/his interlocutors.

It seems worthwhile to further investigate how image schemas and force gestalts feed into the above mentioned and other subtle aspects of meaning-making, involving, for example, felt qualities of resonance in gestural articulations. In this paper, only a limited number of multimodal sequences could be discussed. In order to further substantiate the claims made here, more empirical research is clearly needed, especially with regard to how embodied patterns instantiated in gestures may motivate and structure the intersubjective mediation and negotiation of both schematic and rich, complex meanings.