Infant-directed speech as a window into the dynamic nature of phonology

1.1 Infant-directed speech (IDS)

When adults talk to infants, they use a specific speech register known as IDS. An extensive body of research has examined in what ways IDS differs from adult-directed speech (ADS) among the world’s languages (see Soderstrom 2007 for a review). For example, IDS is characterized by overall higher pitch (Ferguson 1964; Fernald and Simon 1984), a greater pitch range (Fernald and Simon 1984), larger pitch peaks on focused words (Fernald and Mazzie 1991), slower speech rate (Bryant and Barrett 2007), and more distinct phonetic categories (Malsheen 1980; Masataka 1992; Andruski and Kuhl 1996; Kuhl et al. 1997; Burnham et al. 2002; Liu et al. 2007; Cristià 2010) when compared to ADS. In the literature, these characteristics of IDS have been argued to play important roles in communication between infants and caregivers, such as capturing the infants’ attention, communicating affect, and facilitating the infants’ language development (e.g., Fernald 1989; Kitamura et al. 2002). The prosodic and segmental characteristics of IDS can, therefore, provide us with an ideal window through which the dynamical aspects of phonology can be investigated.

At the same time, research on IDS to date has largely been based on physical measurements of the speech without reference to the phonological structure of the language. For example, ‘exaggeration of intonation’ in IDS has been measured simply by subtracting the minimum F₀ value from the maximum of each utterance. In the following sections, we will demonstrate that phonologically informed analyses of IDS can give rise to important new insights on the nature of IDS that had been overlooked, or not fully examined in previous studies. We will use two published studies of Japanese IDS as examples, one involving pitch exaggeration and the other vowel devoicing.

1.2 Data: RIKEN Japanese Mother-Infant Conversation Corpus

Both studies discussed in this paper use a corpus of Japanese IDS (RIKEN Japanese Mother-Infant Conversation Corpus, R-JMICC; Mazuka et al. 2006). The corpus consists of recordings from 22 mothers (age 25–43, average age 33.8), all native speakers from the Tokyo area, and their infants (18–24 months, average 20.4, 9 girls). Each mother was recorded in two situations: playing with her infant using a variety of toys and picture books, and speaking with the experimenter. Speech recorded during the first situation, in which the mother was alone with her infant, will be considered infant-directed, while speech recorded while speaking with the experimenter will be considered adult-directed. Approximately 40 minutes of speech in all were recorded for each mother, resulting in a total corpus length of 14 hours.

The R-JMICC contains various linguistic annotations including segmental, morphological, and intonational labels. Segmental labels, representing types and duration of vowels and consonants, were time-locked to the speech signals. Intonational labeling was based on the X-JToBI scheme (Maekawa et al. 2002), which provides, among other things, information on two levels of prosodic phrasing (Accentual Phrase, AP, and Intonational Phrase, IP), lexical pitch accents, and Boundary Pitch Movements (BPMs).

In addition, recordings of careful, read speech (RS) were collected from 20 of the original 22 mothers, who returned for additional recordings a few years later. In the RS recording, all participants read the same text, consisting of a list of 115 sentences (whose order was randomized by speaker), which were constructed so as to contain phonemes in approximately the same frequencies that they occur in typical adult-directed speech (Sagisaka et al. 1990). Speakers were alone in a sound-attenuated booth during the recording, with a phonetician monitoring the recording from a separate control room.

2.1 Analyses

To test this prediction, Igarashi et al. (2013) analyzed speech from 21 of 22 mothers in R-JMICC. One mother’s speech was not included due to the creakiness of her voice, which made it difficult to extract F₀ values. They began by analyzing the R-JMICC data in the same way as Fernald et al. (1989), measuring the F₀ maximum, mean, minimum, and range of each utterance as a whole. Results revealed that although IDS showed a significantly higher mean F₀, maximum F₀, and minimum F₀ for overall utterances, the F₀ range did not differ significantly between the two registers. These results replicate the findings in Fernald et al. (1989); that is, mothers used a higher pitched voice when talking to infants than to adults, but did not alter their overall pitch range. Moreover, all averages were comparable to those reported in Fernald et al. (1989), demonstrating that the lack of pitch range expansion in Fernald et al. (1989) was not due to some idiosyncratic characteristics of their Japanese sample. It also showed that the R-JMCC contains F₀ characteristics comparable to those of Fernald’s sample.

2.2 Discussion

Taken together, Igarashi et al.’s (2013) results demonstrate that Japanese IDS does show register-induced pitch-range expansion, counter to previous studies that reported otherwise. First, they found robust pitch-range expansion at the locations of BPMs, which occur more frequently in IDS than ADS. BPMs are tones that are associated with pragmatic interpretations (Venditti et al. 2008), and IDS utterances containing BPMs often involve mothers’ attempt to engage infants by addressing questions to them or seeking agreement by using the sentence-final particle ne (cf. Fernald and Morikawa 1993). Crucially, when BPMs occur in IDS, each of the tones is produced with an expanded pitch range. This type of pitch-range expansion is likely to be heard as ‘exaggerated’ by listeners and may account for the results of previous studies showing that the intonation of Japanese IDS is perceived to be exaggerated by Japanese adults and infants.

Second, the length-induced pitch-range expansion in the ADS BODY is not an ‘exaggeration’ of the intonation. The pitch range of an Intonation Phrase (IP) is determined primarily by the number of accents the IP contains – the longer the IP (and thus the more accented words it contains), the larger the pitch range. This is independent of register-induced pitch-range modification and occurs regardless of the difference between ADS and IDS. When a speaker produces a long utterance with a pitch range that is normal for its length, she/he is under no pressure to ‘exaggerate’ the intonation, nor is it likely to be heard as ‘exaggerated’ by a listener. In fact, when ADS and IDS utterances of equal length were compared, there was no difference in pitch ranges between the two registers.

Igarashi et al.’s findings are novel in that they successfully demonstrate that not all phonological tones are subject to the paralinguistic modification characteristic of a specialized speech register. Specifically, their analysis suggests that pitch-range expansions in IDS are not realized in the same way in every language, but are instead implemented within a language-specific system of intonation. When there is a desire or pressure to exaggerate the intonation, speakers seem to do so by expanding the pitch range at the location where flexibility in varying contours is most tolerated. In phonological terms, this is the location where pragmatically chosen tones are realized. In the case of Japanese, these are BPMs at the boundaries of prosodic phrases, while in the case of English, they are not only phrase accents and boundary tones at the phrasal boundaries, but also pitch accents at the locations of stressed syllables. It has been commonly assumed that paralinguistic pitch-range modifications (which should include intonational exaggeration in IDS) can occur globally irrespective of what tones are present in the utterance (cf. Ladd 1996: Ch. 7). The results of Igarashi et al. (2013), however, show that only certain tones can undergo paralinguistic modifications. Their study, therefore, promises to shed light on the phonetics of pitch range variation.

3.1 Devoicing in IDS

Research on IDS has shown that phonetic categories in IDS tend to be more distinct (Malsheen 1980; Masataka 1992; Andruski and Kuhl 1996; Kuhl et al. 1997; Burnham et al. 2002; Liu et al. 2007; Cristià 2010) when compared to ADS. It has been argued that the distinct phonetic categories in IDS serve to provide the infant with input that is optimized for learning certain aspects of the linguistic system (Kuhl et al. 1997; de Boer and Kuhl 2003; Kirchhoff and Schimmel 2005). These hypotheses have in common a vision of IDS as an essentially listener-oriented speech style, in which the speaker expends additional effort in order to maximize the information content in the signal, making the infant listener’s task easier. Fernald (2000), exemplifying this view, argues that IDS represents a form of hyperspeech, a reference to Lindblom’s (1990) “H and H theory,” which posits a continuum from effortful but easy to perceive “hyperspeech” on one end to less effortful but harder to perceive “hypospeech” on the other.

Infant-directed speech is only one variety of hyperspeech. Foreigner-directed speech (Scarborough et al. 2007; Uther et al. 2007), speech produced in noisy conditions (Payton et al. 1994), and read speech (Nakamura et al. 2008) have all been described as hyperarticulated speech styles, in which the information content in the signal is increased for the benefit of the listener. If the H and H theory is correct, then these various speech styles should pattern similarly – those acoustic parameters associated with clarity and intelligibility should be altered in the same direction when these varieties of hyperspeech are compared to a non-hyperspeech baseline. Martin et al. (2014) tested this hypothesis by comparing three different speech styles, all produced by the same set of speakers: spontaneous infant-directed speech, spontaneous adult-directed speech, and careful, read speech (RS).

Devoiced vowels are more difficult to perceive than their voiced counterparts (Gordon 1998), meaning that devoicing results in a loss of information regarding the vowel’s identity. Beckman and Shoji (1984) show that native Japanese listeners’ ability to identify whether a devoiced vowel is /i/ or /u/ drops substantially when the vowel is devoiced. Vowel devoicing also impedes the recognition of nonwords in running speech (Cutler et al. 2009). Devoiced vowels thus make the infant’s task of learning how to identify vowels and words that much more difficult. If IDS is a form of hyperspeech, we expect Japanese speakers to devoice less often when speaking to infants, in order to maximize the infants’ chances of correctly perceiving vowels and the words that contain them.

3.2 Analyses

The first part of Martin et al.’s (2014) analyses was based on IDS and ADS samples from the R-JMICC. A total of 7,838 vowels which occurred between two voiceless consonants were identified and coded for voicing by a trained phonetician who is a native speaker of Japanese (second author). In order to check the accuracy of these codings, 10% of the vowel tokens were selected at random and coded separately by the third and the fourth authors of the present paper (both trained phoneticians, one of whom is a native speaker of Japanese, and neither of whom was involved in the original coding of voicing). Inter-coder reliability was 93.5%. In the second part of the analyses, careful, read speech (RS) from 20 of the original 22 mothers was compared to ADS. As with the IDS and ADS samples, all vowels which occur between voiceless consonants were coded for voicing.

3.3 Discussion

Martin et al. (2014) were able to capture the dynamic nature of an articulatory process, vowel devoicing, by comparing factors that influence devoicing in infant-directed, adult-directed, and careful, read speech. They found that compared to adult-directed speech, the process was implemented less often in one class of sounds (high vowels) and more often in another (non-high vowels). They also found that in careful, read speech, the high vowel devoicing rule is implemented more consistently than in spontaneous adult-directed speech, while non-high vowels are more frequently produced with voicing.

The way other factors interacted with speech register also offers useful insights. Among many of the factors that have been previously reported to impact vowel devoicing, linguistic/phonological factors such as identity of the vowel, the consonantal context, and the syllable’s accent status, were found to impact only the devoicing of high-vowels but not non-high vowels. Factors that can be considered para-linguistic, such as speech rate and breathiness of the voice, on the other hand, were found to impact non-high vowel devoicing as well. These differences are also consistent with the two mechanisms view. The fact that the two types of devoicing (high vowels vs. non-high vowels) showed an opposite pattern with regard to the three registers, when considered alongside the way other factors interacted with speech register, seems to suggest a picture in which two distinct mechanisms underlie these processes: phonological devoicing, in which speakers intentionally produce the target vowel without voicing, and phonetic devoicing, which occurs as an unintentional consequence of gestural overlap.

Martin et al. (2014) attempted to offer an interpretation for why devoicing rates change across speech styles in the directions they do. For high vowels, the major puzzle is the fact that adult-directed speech has devoicing rates intermediate between those of infant-directed and read speech. If one takes a view that IDS and RS are both varieties of hyperspeech – speakers are trying to speak clearly to increase the intelligibility of their speech – one would expect them to diverge from ADS in the same direction. Martin et al. suggest that the answer to this question lies in the differing audiences for each speech style, and in the effects those audiences’ lexical knowledge, or lack thereof, has on the perceptibility of devoiced vowels. More specifically, the fact that infants do not yet know many words forces them to process speech bottom-up, relying primarily on the properties of individual segments – this means that they would benefit most from vowels which are pronounced as clearly as possible. Adults, on the other hand, are equipped with a vast store of top-down knowledge, meaning that their recognition is optimized when words are pronounced in their expected, canonical forms.

There is experimental evidence supporting both of these claims. As mentioned earlier, Beckman and Shoji (1984) found that when adult listeners were presented with words in isolation containing devoiced vowels, the rates at which they correctly identified the vowels (i.e., whether they were /i/ or /u/) were lower than when the vowels were voiced. Conversely, Ogasawara and Warner (2009) found that in a lexical decision task, adult listeners responded faster to the presence of a target high vowel in a devoicing environment when it was devoiced rather than voiced, presumably because their knowledge of the word led them to expect a devoiced vowel. These divergent findings are likely the result of the different tasks involved. In the Beckman and Shoji study, subjects did not need to access the lexicon, since the options they were asked to choose between in each trial were both actual words (e.g., /sikaN/ ‘military officer’ versus /syukaN/ ‘subjectivity’). To complete the task, subjects had to focus on the crucial vowel. The Ogasawara and Warner lexical decision task, however, involved searching one’s lexicon for a match to the entire stimulus (e.g., /yakisoba/). In other words, devoicing appears to impair perception of individual vowels but aid lexical access, presumably because lexical entries for words with frequently devoiced vowels are encoded with devoiced versions of those vowels.

Taken together, these experimental results suggest that for a listener with little or no lexical knowledge, such as an infant, whose primary task involves identifying individual segments, devoicing would pose an obstacle to be overcome. For an adult listener who is matching speech strings to an already developed lexicon, on the other hand, hearing words in their most typical form (i.e., with high vowels devoiced in devoicing environments) would actually help understanding.

The opposing trends in devoicing rates that we see in high vowels can thus be interpreted as speakers fine-tuning their speech to the processing needs of the intended listener. Infant-directed speech is characterized by the least amount of devoicing, accommodating infants’ need for maximally informative vowels. Adult-directed speech contains more devoicing, conforming to the adult listener’s expectations. Read speech, in which speakers are able to expend more time and energy on pronunciation, represents an even more extreme version of this adult-directed style. Speech rate, in contrast, fails to show these listener-specific effects, because slower speech is advantageous both for identifying individual vowels and for lexical lookup.

The above argument should not apply to non-high vowels, if we are correct in our speculation that devoicing in these vowels is due to articulatory factors which are beyond the control of the speaker. What, then, is the cause of the different devoicing rates across speech styles in non-high vowels? The most plausible candidate is voice quality – speech becomes breathier as we move from RS to ADS to IDS. As mentioned above, Martin et al. (2014) found a significant effect of breathiness in the non-high vowels, with greater breathiness being associated with more devoicing. But precisely because this devoicing is inadvertent, speakers are unable to reduce it even though doing so would benefit the infant listener. The increased breathiness in IDS itself serves a function – breathier speech is perceived as softer and more intimate (Laver 1980; Ishi et al. 2008), and is potentially used to communicate positive affect and maintain an emotional bond with the infant. But these advantages also exact a price, in the form of increased devoicing in the non-high vowels.

In sum, Martin et al. found that mothers alter their vowel devoicing behavior according to speech style, and furthermore do so in different ways depending on the height of the vowel. They have argued that, although infant-directed speech and careful, read speech modify devoicing in opposite directions, both can be considered forms of hyperspeech, in that they maximize intelligibility for their intended audiences. In addition, they have shown that infant-directed speech can contain properties – namely, increased devoicing in non-high vowels – which are in fact disadvantageous to infant listeners, but can be seen to be the result of other factors – in this case, increased breathiness – which themselves are beneficial to infant communication.

The picture of speech styles that emerges from this study is thus more complex than a unidimensional continuum from hypospeech at one end to hyperspeech at the other. A single acoustic parameter – the presence or absence of voicing pulses during a vowel – represents a battleground on which an array of forces compete to be expressed. Exploring the complex interplay of factors impinging on speech behavior promises to give us a richer, more nuanced picture of the role of infant-directed speech.