We routinely build abstract and complex concepts by connecting them to what our minds find more familiar: perceptual information, simple events or deeply-entrenched cultural practices. Such an everyday capacity for conceptual projection shows up in numerous aspects of meaning construction (Fauconnier 1997). The discovery of cross-cultural patterns for mappings between concepts has been a major breakthrough in cognitive science. Direct transfer from a source (mainly sensorimotor, e. g. SPACE, FORCE) to a target (typically abstract, and thus harder to structure, e. g. TIME, EMOTION) has so far been the predominant paradigm in mappings research (Gentner 1983; Lakoff & Johnson 1980).
The understanding of time in spatial terms has long been identified as an example of a metaphorically constructed concept (Guyau 1988 ). With the recent paradigm shift towards an embodied view of cognition (Wilson 2002) space-time mappings have been chosen as the classical case of projection from a concrete to an abstract domain (Boroditsky 2000; Boroditsky 2001; Casasanto & Boroditsky 2008; Lakoff 1993; Lakoff & Johnson 1980; Lakoff & Johnson 1999; Torralbo et al. 2006; Weger & Pratt 2008; and many more). In this conceptual metaphor, time travels towards an observer (“Spring will arrive in two weeks”) or vice versa (“We are approaching Sunday”); this motion can also be observed independently, from an external viewpoint (“The lecture will be followed by a reception”). Such linguistic patterns are found in many languages (Radden 2004) and there is also abundant psycholinguistic evidence for the (often automatic) activation of spatial domains when processing temporal expressions (Ulrich and Maienborn 2010).
However, in spite of its usefulness, the transfer/metaphor model has proved insufficient to account for the full complexity of space-time mappings, especially when we take into account those uses that involve ad-hoc pragmatic or aesthetic effects. For example, in our knowledge of motion, an object does not travel at two different speeds simultaneously, and its speed cannot be altered by our feelings. However, in many temporal expressions this becomes possible: “Time goes by really slowly. At the same time, it goes by really fast.” (CNN, said by a man waiting for word on a person missing in the bomb detonations in London, July, 2005) (Fauconnier and Turner 2008). These problems and many others can be solved by replacing the dominant direct transfer model with one that includes projections from different domains to a middle conceptual space, which receives information from all the inputs and “blends” the selected elements and relations projected into the hybrid, ad-hoc concept, providing a locus for emergent meanings; this is currently known as the “blending model” (Fauconnier and Turner 2002). In this model, the classic fixed unidirectional patterns can be replaced with more flexible and complex templates for the integration of concepts into new wholes (Fauconnier 2009; Pagán Cánovas 2010; Pagán Cánovas & Turner 2016; Turner 2014).
2 Timelines: Material anchors for the processing of time
One of the most widely applied blending templates to organize the domain of time is the timeline (Coulson & Pagán Cánovas 2013; Pagán Cánovas & Jensen 2013). This template merges separate sources of information into a single recipe for organizing time. The pattern integrates temporal duration, linear extent, events, and objects into a single material representation. Its usefulness is mainly based on the type of affordances that the resulting entity offers. For example, the use of a timeline allows us to “see” diachrony at a glance, aligning dispersed events on a one-dimensional space, thus facilitating the performance of temporal computations by exploiting directly the basic perceptual attributes of the object.
Timelines are examples of “material anchors”: external objects whose physical structure can be used to facilitate the computation of abstract information (Hutchins 2005). Material anchors resonate with the widespread cognitive science trend of “external” or “distributed cognition”: the idea that people use external aids for their cognitive processes, “offloading” cognition onto external objects whose structure can be manipulated and used for reducing the complexity of a cognitive task. To be more specific, these devices aid by reducing memory load (both short and long term), providing information than can be directly perceived, and typically altering the task to be performed, replacing it with easier and more efficient action sequences (Zhang and Patel 2006). This would be the case of diagrams or a great variety of artifacts and devices with cognitive functions, and, in the case of time, that of calendars, clocks, sundials, and, of course, timelines.
In Section 3, we will review some of the main characteristics of timelines and provide empirical evidence about their psychological reality, using some of the evidence in the existing literature as well as from results from our lab, stemming from psycholinguistic studies (e. g., Pagán Cánovas et al. 2015). As we shall see there, in our conceptualization of time, we can deploy timelines along the three spatial axes (sagittal, vertical, and lateral). However, oral languages cannot make explicit reference to this third axis, though its use has been attested in time conceptualization tasks in different cultures. We will then see in Section 4 how timelines can be activated in real life communication in ways that go beyond the information provided by language, namely, by the inclusion of gestural information in multimodal constructions.
3 Timelines: Spatial features
There are quite definite constraints regarding the spatial characteristics of timelines. Although pretty much any shape could in principle be valid, the fact is that graphical timelines, since their introduction as chronographic diagrams in the eighteenth century (a complete history in Rosenberg and Grafton 2010) are predominantly connected with a straight line. This line, as we mentioned before, can be also deployed along a lateral axis, even if linguistically only the sagittal and vertical axes can be explicitly mentioned. Thus, in psychological studies, this lateral configuration has been shown to underlie a wide variety of temporal representations and time-related tasks. For example, in one of the first studies on the orientation of time-space mappings (Tversky et al. 1991), American subjects associated earlier events with left locations in space, and later events with right locations, while Hebrew and Arab subjects showed the opposite pattern. Evidence for the left-to-right orientation of the mental timeline in Western societies has multiplied ever since, with recent studies providing particularly strong evidence (Santiago et al. 2007; Santiago et al. 2010; Torralbo et al. 2006; Weger & Pratt 2008; among others). While the Tversky et al. (1991) study had already correlated these effects with the writing system of participants, this congruity with reading-writing direction has been confirmed by a number of studies (see Bottini and Casasanto 2010), including cases of vertical orientation, as in the case of Mandarin (Boroditsky et al. 2011; Fuhrman et al. 2011).
Results from our lab provide new data supporting both the tight connection of the timeline to the straight shape, and its left-to-right orientation. For example, in Pagán Cánovas et al. (2015), subjects read a poem which featured a physical object, a snake, used poetically as a representation of the self in time, that is, as a material anchor for time or, more specifically, the duration of human life. The participants’ task was to make a drawing of how they imagined this snake. Our results showed that participants who drew the inferences suggested by the poem and thus used the snake as a representation of time, altered its representation in consistent ways. So, when we compared the snake that they drew with the one provided by a control group (who were just asked to draw a snake, without reading any poem), their snake-as-material-anchor-for-time was measurably less “curvy” than those of the control group. This study has thus provided some of the first direct evidence for this feature of the mental timeline. We have not found any explicit studies addressing the preferred shapes in the alignment of sequences or the representation of durations. Nevertheless, most of the stimuli depicting motion events in studies on time-space mappings are straight lines (Boroditsky 2000; Boroditsky & Ramscar 2002; Casasanto, Fotakopoulou, and Boroditsky, 2010); linear objects are also common (Casasanto & Boroditsky 2008; Bottini & Casasanto 2010).
The advantages of the straight line for representing the unidimensionality of time are so obvious that it might not seem necessary to compare it with other shapes. However, we must not forget that motion along a straight path is by no means the typical motion event in our everyday experience. The selection of the straight line for the mental timeline indicates that the spatial structure is being adjusted to produce an optimal representation. Interestingly enough, our study also revealed the strength of the left-to-right directionality within timelines. Our control group showed a consistent tendency to portray the snake with its head on the left, that is, with a right-to-left implied motion; this indeed seems to be the orientation in prototypical drawings of snakes. As a contrast, subjects who established the temporal mappings prompted by the text showed a clear tendency to revert their mental image of the snake, making it face rightwards, in consonance with the typical flow of time in their culture. Again, this inversion of directionality could be modulated by priming: making participants more aware of temporal orderings before reading the poem made them invert the directionality of the imagined snake to a higher degree.
4 Timelines in multimodal constructions
Psycholinguistic evidence seems to point in the direction of the automatic activation of timelines across most if not all our temporal conceptualization tasks. Explicit mention to such underlying timelines can indeed be found in many temporal linguistic expressions; such is the case of (most of the) expressions which include spatial terms in their description of time. Thus, sentences such as “we have a bright future ahead of us” or “the glory days of airline food are behind us” make reference to a sagittal timeline, in which the future is in front of us and the past is behind us. There are also cases in which language shows a spatialization of time, but the exact axis used is left in the background. Such is the case of expressions such as “in the near future” or “in the distant past”, which locate a temporal referent point (the future or the past in the examples supplied) in quite clear spatial terms, by using spatial adjectives such as “near” or “distant”, but do not mention explicitly the axis on which such nearness or distance is to be computed (Nuñez & Cooperrider, 2013). This brings forth a conundrum: if the lateral timeline is active in many temporal conceptualization tasks, but no mention to a lateral axis is allowed linguistically, is this lateral axis active while we are uttering temporal expressions? It seems that this is indeed the case (Casasanto & Jasmin, 2012; Walker & Cooperrider, 2016).
In this section we want to focus on a great number of constructions that seem to be especially linked to timelines; we will focus especially on what can be called “temporal demarcative expressions”. A very relevant point of interest of these constructions concerns the recurrent use of multimodal information to bring forth the use of a timeline. These are grammatical constructions (in the Goldbergian sense) that designate the extension of a complete temporal strech by indicating a beginning point and an ending point (e. g., since the inception till the completion, from beginning to end). In this section we will focus on one of these constructions, the [From X to Y] construction.
The [From X to Y] construction is a mid-level construction, with partial saturation, in which two elements are lexically filled: the spatial prepositions from and to. The other remaining elements are open variables, which can be filled with a highly diverse number of elements of varying length and inner structure. Some examples are found in (1):
a. From beginning to end
b. From start to finish
c. From the day I met you to the present moment
d. From his early works to his later writings
e. From rags to riches
f. From their humble beginnings to now
g. From shower to out the door (in five minutes!)
As can be seen in examples 1-a, both X and Y can be filled by nouns making direct reference to beginning and end points of processes (beginning, end, start, finish), noun phrases (the present moment, his later writings), adverbs (now)… As happens with many other constructional structures, it can be part of idioms in which the grammaticality of the result is “diminished”, as could be argued for 1.g (from rags to riches). This is found also in spontaneous examples, such as 1.g, in which the second section of the construction (“to out the door”) seems less natural on its own than within the whole construction.
In classic cases of Goldbergian constructions, these structures can be said to have an ‘inherent’ meaning. This is what explains the cases of ‘grammatical coercion’: this corresponds to cases in which the lexical material inserted in a given construction is thus ‘coloured’ by the overall meaning of the construction. This is what has been shown for the Caused Motion construction, which can take an intransitive verb such as sneeze and adapt it to the overall meaning of the sentence (she sneeezed the foam off the capuccino), or the Ditransitive Construction, which conveys the idea of displacement of one object to a recipient, even if the verb by itself does not convey this idea (she baked him a cake). Coercion is not limited to argument structure constructions: English mass nouns are typically used in constructions with zero article (e. g., I want water) or with the presence of determiners such as “some” or “any” (e. g., give me some sugar). Count nouns, on the other hand, cannot appear in these contexts (cf I want car or give me some car). Insertion of a count noun in one of these constructions can force a “mass” reading, coerced by the construction, e. g., you’ve got cake all over your face).
This coercion process can be found in the [From X to Y] construction as well: elements that do not have a clear temporal indication are coerced into the general meaning of the construction such that they are interpreted as beginning or end points of a temporal strecht.
A clear example of this is 1g, “from shower to out the door”. By itself, the Noun Phrase “shower” does not convey any temporal meaning; at the very least, it does not convey the sense of beginning of a temporal event. However, its insertion in the X element of the construction forces the interpretation of “shower” as the first event of a sequence. A similar process is at work with the second element, “out the door”, which is understood as the event that finishes the temporal strecht being described. Overall, the meaning of (1g) is thus clear: it is used to describe the time that elapsed between taking a shower and getting out of the house, as items within a sequence that belongs to a conventional frame of encyclopaedic knowledge (with middle elements such as getting dry, getting dressed, picking up the keys… and so forth). In fact, some of the elements we have used in this description of its meaning are not explicitly mentioned, e. g., no verb is mentioned with “out the door”, which is somehow presupposed in order to arrive at a interpretation of the construct which is coherent with its temporal demarcartive meaning.
Nevertheless, what is especially interesting about this construction is its strong connections with multimodal information. In fact, some of the concrete instantiations of this construction, especifically the construct “from beginning to end” are so strongly associated with a spatial gesture activating a timeline that, as shown in our studies, a spatial gesture can be found in 8 out of 10 cases. It should be taken into account that most of these utterances were spontaneous and thus gesturing was unconscious; this tells us about the high degree of entrenchment of the gestural information associated with the construction. Even in cases in which the discursive situation (e. g., an anchor in a TV news program, or an interviewer or interviewee in a formal setting) somehow restricted the naturalness associated with free gestures, speakers found it hard to suppress this gestural part of the construction connected to the lateral timeline and produced smaller versions of this material anchor (Figure 1).
The enormous frequency of the gesture associated with the construction makes this parameter hard to ignore. A construction is formed by the “distillation” of any recurrent information in different usage-events. Usage-based theoreticians such as Bybee have stressed that in principle, any type of information which is recurrent enough can form part of a construction. In the case of this construction, we have a great recurrence of a spatial gesture, connecting the temporal expression to the spatial timeline, which is an important part of our conceptualization of time. Shouldn’t this information be considered as part of the construction? In principle, our answer should clearly be affirmative; however, any attempt at integrating this type of information faces formidable challenges. Gesturing is a “non-verbal or non-vocal communication system,” a semiotic system different from language, even if tighthly integrated with it. A full description of the complex interactions between language and the different types of gestures lies beyond the scope of this paper. Suffice to say that types of gestures can be very different (cf. Kendon’s continuum, cf McNeill 1992), going from “emblems” such as the “OK sign” produced by forming a ring with index and thumb, to “speech-framed gestures, in which gestures complete the meaning of a sentence-as in “he went [gesture indicating downward motion]”. The different nature of each type of gesture could well need different mechanisms for spelling out the integration of this type information into grammatical constructions. In any case, the inclusion of this information pushes construction grammar beyond its more traditional linguistic realm: if we associate constructions with this type of multimodal information, they cannot be considered as mere linguistic units anymore, and must be thus construed as included in some broader “multi-modal” communicative system, which would include language, gesture and quite possibly other semiotic systems, such as bodily posture, facial expressions, gaze cues, etc., which are at the present moment largely unexplored.
In spite of the great amount of attention that space-to-time mappings have attracted, there are still many aspects of their behavior and use which have not been properly explained. We believe that research on the timeline as a template for conceptual integration can help frame the discussion in more fruitful ways. Such multimodal blends are a natural and powerful construct for explaining the complex interactions between the entrenched structures that facilitate a task as complex as temporal organization, while at the same time support the flexibility needed for the dynamically changing landscape of real-life communication.
Overall, our results are not easily accounted for by fixed, ontological space-to-time projections. A more adequate level of granularity is provided by a model including a flexible network of mappings and a blended concept with emergent structure. The final instantiation of this blending network is a dynamic process: it depends on an entrenched template, but it is also adaptable to local purposes. The combination of methods we have presented here, with measurable spatial parameters in the mental imagery generated in reading comprehension, along with the analysis of spontaneous co-speech gesture in a naturalistic setting, seems a promising way to obtain relatively clean data about the cognitive processes at work in online cross-domain mappings.
There is little doubt that spatial information underlies a great deal of our processing of temporal information. Research on the ways in which timelines are associated with specific grammatical constructions has just started, and has brought forward new challenges for construction grammar. A true multimodal construction grammar cannot be developed using the same tools and mechanisms applied so far, but will require instead a fresh approach and a careful rethinking of the whole communicative process. It is our belief than in this transition from a narrow conception of linguistic communication to a broader and more complete view of communication “in the wild,” multimodal construction grammar will play an important role.
Boroditsky, L. 2001. Does language shape thought? Mandarin and English speakers’ conceptions of time. Cognitive Psychology 43(1). 1–22. Google Scholar
Bottini, R. & D. Casasanto. 2010. Implicit spatial length modulates time estimates, but not vice versa. Spatial Cognition VII. 152–162. Google Scholar
Brewer, M. B. 2000. Research design and issues of validity. In H. T. Reis & C. T. Judd (eds.), Handbook of research methods in social and personality psychology, 3–16. Cambridge: Cambridge University Press.Google Scholar
Casasanto, D. & K. Jasmin. 2012. The hands of time: Temporal gestures in English speakers. Cognitive Linguistics 23(4). 643–674. Google Scholar
Coulson, S. & C. Pagán Cánovas. 2013. Understanding timelines. Journal of Cognitive Semiotics 5(1–2). 198–219. Google Scholar
Fauconnier, G. 1997. Mappings in Thought and Language. Cambridge: Cambridge University Press. Google Scholar
Fauconnier, G. 2009. Generalized integration networks. In V. Evans & Stéphanie Pourcel (eds.), New directions in cognitive linguistics, 147–160. Amsterdam: John Benjamins. Google Scholar
Fauconnier, G. & M. Turner. 2002. The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books. Google Scholar
Fauconnier, G. & M. Turner. 2008. Rethinking metaphor. In R. W. Gibbs (ed.), The Cambridge handbook of metaphor and thought, 57–66. Cambridge: Cambridge University Press. Google Scholar
Fuhrman, O., K. McCormick, E. Chen, H. Jiang, D. Shu, S. Mao & L. Boroditsky. 2011. How linguistic and cultural forces shape conceptions of time: English and Mandarin time in 3D. Cognitive Science 35. 1305–1328. Google Scholar
Gentner, D. 1983. Structure-mapping: A theoretical framework for analogy. Cognitive Science 7. 155–170. Google Scholar
Guyau, J. M. 1988 . The origin of the idea of time. Reprinted In J. A. Michon, V. Pouthas & J. L Jackson ((ed.)), Guyau and the idea of time. Amsterdam: North-Holland. Google Scholar
Lakoff, G. 1993. The contemporary theory of metaphor. In A. Ortony (ed.), Metaphor and thought, 202–251. Cambridge: Cambridge University Press. Google Scholar
Lakoff, G. & M. Johnson. 1980. Metaphors we live by. Chicago: University of Chicago Press. Google Scholar
Lakoff, G. & M. Johnson. 1999. Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books. Google Scholar
Lakoff, G. & M. Turner. 1989. More than cool reason: A field guide to poetic metaphor. Chicago: University Of Chicago Press.Google Scholar
McNeill David. 1992. Hand and Mind: What gestures reveal about thought. Chicago: University of Chicago Press. Google Scholar
Núñez, R. & K. Cooperrider. 2013. The tangle of space and time in human cognition. Trends in Cognitive Sciences 17(5). 220–229. Google Scholar
Núñez, R. E. & E. Sweetser. 2006. With the future behind them: Convergent evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science 30(3). 401–450.Google Scholar
Pagán Cánovas, C. & M. F. Jensen. 2013. Anchoring time-space mappings and their emotions: The timeline blend in poetic metaphors. Language and Literature 22(1). 45–59. Google Scholar
Pagán Cánovas, C. & A. Piata. Forthcoming. The way time goes by: Conceptual integration and the poetics of time. In S. Csábi (ed.), New directions in cognition and poetics. New York: Oxford University Press.Google Scholar
Pagán Cánovas, C. & M. Turner. 2016. Generic integration templates for fictive communication. In E. Pascual & S. Sandler (eds.), The conversation frame: Forms and functions of fictive interaction, 45–62. Amsterdam: John Benjamins. Google Scholar
Pagán Cánovas, C., J. Valenzuela & J. Santiago. 2015. Like the machete the snake: Integration of topic and vehicle in poetry comprehension reveals meaning construction processes. Psychology of Aesthetics, Creativity, and the Arts 9(4). 385–393. Google Scholar
Rosenberg, D. & A. Grafton. 2010. Cartographies of time: A history of the timeline. New York: Princeton Architectural Press. Google Scholar
Santiago, J., J. Lupiáñez, E. Pérez & M. J. Funes. 2007. Time (also) flies from left to right. Psychonomic Bulletin & Review 14. 512–516. Google Scholar
Santiago, J., A. Román & M. Ouellet. 2011. Flexible foundations of abstract thought: A review and a theory. Spatial Dimensions of Social Thought 41–110.Google Scholar
Santiago, J., A. Román, M. Ouellet, N. Rodríguez & P. Pérez-Azor. 2010. In hindsight, life flows from left to right. Psychological Research PRPF 74(1). 59–70. Google Scholar
Teuscher, U., M. McQuire, J. Collins & S. Coulson. 2008. Congruity effects in time and space: Behavioral and ERP measures. Cognitive Science: A Multidisciplinary Journal 32(3). 563–578.CrossrefGoogle Scholar
Torralbo, A., J. Santiago & J. Lupiáñez. 2006. Flexible conceptual projection of time onto spatial frames of reference. Cognitive Science 30(4). 745–757. Google Scholar
Turner, M. 2014. The origin of ideas: Blending, creativity, and the human spark. New York: Oxford University Press. Google Scholar
Tversky, B., S. Kugelmass & A. Winter. 1991. Cross-cultural and developmental trends in graphic productions. Cognitive Psychology 23. 515–557. Google Scholar
Valenzuela, J., C. Pagán Cánovas & I. Olza In review. Gesturing the timeline: Gestures co-occurring with temporal demarcative expressions provide evidence for anchoring and viewpoint integration.Google Scholar
Walker, E. & K. Cooperrider. 2016. The continuity of metaphor: Evidence from temporal gestures. Cognitive Science 40(2). 481–495 Google Scholar
Weger, U. W. & J. Pratt. 2008. Time flies like an arrow: Space-time compatibility effects suggest the use of a mental timeline. Psychonomic Bulletin & Review 15(2). 426–430. Google Scholar
Wilson, M. 2002. Six views of embodied cognition. Psychonomic Bulletin and Review 9. 625–636. Google Scholar
Zhang, J. & V. L. Patel. 2006. Distributed cognition, representation, and affordance. Pragmatics & Cognition 14(2). 333–341. Google Scholar