The interaction between predictability and pre-boundary lengthening on syllable duration in Taiwan Southern Min

1.1 Predictability and phonetic variation

Decades of research in linguistics and speech sciences have observed shortening or segmental reduction of words and syllables with higher lexical frequencies (Aylett and Turk 2004; Gahl 2008; Hashimoto 2021; Jurafsky et al. 2001; Losiewicz 1992; Pluymaekers et al. 2005b; Schweitzer and Möbius 2004; Seyfarth 2014; Tang and Bennett 2018; Tang and Shaw 2021; Van Son and Van Santen 2005; Whiteside and Varley 1998). The same observations are also found for words and syllables in contexts where they are more likely to occur (Andreeva et al. 2020; Aylett and Turk 2004; Bell et al. 2009; Bolinger 1963; Hashimoto 2021; Jurafsky et al. 2001; Lieberman 1963; Malisz et al. 2018; Pluymaekers et al. 2005a; Seyfarth 2014; Sharp 1960; Shaw and Kawahara 2019; Tang and Shaw 2021; Van Son and Pols 2003; Van Son and Van Santen 2005). A consistent finding is that higher lexical frequency and contextual likelihood correlate with shortening or segmental reduction in the acoustic signal. Diachronic change in lexical frequency and contextual likelihood has also been found to similarly correlate with diachronic variability in word duration (Sóskuthy and Hay 2017). Similar correlations are found in other aspects of phonetic realizations, where higher frequencies and contextual likelihood correlate with smaller formant transitions (Benner et al. 2007; Brandt et al. 2018, 2021; Croot and Rastle 2004; Malisz et al. 2018; Whiteside and Varley 1998), lower consonantal center of gravity (Malisz et al. 2018; Van Son and Van Santen 2005), and more segmental deletion (Bybee and Scheibman 1999; Jurafsky et al. 2001; Cohen Priva 2015; Seyfarth 2014; Whang 2019).

We can frame these findings as a “predictability effect” on phonetic realizations: Whether a unit is more frequent or more probable in a particular context affects how easy it is to predict its occurrence. From an Information-Theoretic (Pierce 1972; Shannon 1948) point of view, the correlation between higher predictability and acoustic weakening shows that a more predictable unit carries less information and thus does not require strong acoustic cues. On the other hand, if a linguistic unit is infrequent itself or occurs in a surprising context, its acoustic signal needs to be strengthened to increase its chance of being correctly recognized.

This information-theoretic account has parallels with proposals on speech production and processing such as the Hyper- & Hypoarticulation (H&H) theory (Lindblom 1990), which posits that balance between ease of articulation and effectiveness in perception together shape speech production. Another relevant account is the Smooth Signal Redundancy Hypothesis (Aylett and Turk 2004, 2006), which states that the amount of information is distributed evenly across each element of an utterance so that “linguistic redundancy” (i.e., lexical, syntactic, semantic, and pragmatic information) and “acoustic redundancy” (i.e., acoustic cues) have an inverse relationship. The smooth distribution of information is also an idea founded in Information Theory, where it is viewed as essential in ensuring robust communication in noisy environments. A schematic representation of the relationship is shown in Figure 1 (Figure 5 in Turk and Shattuck-Hufnagel 2014).

Figure 1:

A schematic demonstration of the hypothesized relationship between predictability and acoustic salience that result in a smooth distribution of likelihood of recognition across an utterance (Figure 5 in Turk and Shattuck-Hufnagel 2014).

1.2 Predictability and prosodic structure

Predictability is not the only factor that affects phonetic realizations. Especially for durational variability in acoustic cues, prosodic factors such as phrasing and prominence also play an important role. For example, it has been commonly found that lengthening occurs at the right edge of prosodic units (Fougeron and Keating 1997; Oller 1973, a.o.). The interaction between this boundary-lengthening effect and predictability has been discussed in the literature. Aylett and Turk (2004) hypothesize that prosodic prominence encodes and thus mediates the relationship between information redundancy and acoustic cues. Under this view, the relationship between predictability and phonetic cues is not direct. Instead, the placement of prominence and the organization of phrasing reflect the predictability profile of an utterance (e.g., unpredictable units are more likely to be focused or have a particular phrasing pattern). The seemingly direct correlation between phonetics and predictability follows from the phonetic correlates of focus and phrasing (see Ladd 2008 for a review of this view). Turk (2010) further extends the hypothesis to cover the role that prosodic constituency and boundary signaling play in modulating signal redundancy.

There are empirical works that support this hypothesis. Cutler and Carter (1987) found that lexical stress falls on 70% of the initial syllable of content words in English spontaneous speech. Since content words are less predictable than function words, especially in the initial syllable, this finding suggests that the metrical structure that dictates stress placement already plays a big role in accounting for the relationship between phonetic realization and predictability. This relationship between the prosodic organization and predictability also has parallels at the sentential level: Less predictable units tend to receive sentential accents, and more predictable units may be unaccented. Ladd (2008, Chapter 6) summarizes relevant findings and discusses several types of units that are likely to be unaccented or deaccented unless in narrow or contrastive focus, and these units can all be interpreted as being more predictable (having less “semantic weight” per the original discussion), such as indefinite pronouns (e.g., someone, something) and semantically empty content words (e.g., man, guy, place, as opposed to policeman, friend, barn; citing Bolinger 1972).

Aylett and Turk directly tested their hypothesis by comparing the R ² of regression models with different combinations of language redundancy (syllabic trigram probability, log word frequency, and givenness) and prosodic variables (prominence, phrasing). They found that both redundancy and prosodic structure account for syllable duration, i.e., higher levels of information redundancy lead to shorter duration, and stronger prominence and higher-level boundaries lead to longer duration. They also found that most of the contributions by language redundancy are shared by prosodic variables. In addition, prosodic variables made a larger proportion of unique contributions to predicting syllable duration. On the other hand, the unique contribution of language redundancy is highlighted only when controlling for prosodic phrasing. These findings were viewed as supporting the Smooth Signal Redundancy Hypothesis, as they show a relationship between language redundancy and durational variability. More importantly, they also show that prosodic structure predicts a large portion of durational variabilities that language redundancy could have directly explained.

More recent studies have explored the interaction between predictability and prosodic structure in affecting phonetic realizations. Malisz et al. (2018) address the interaction between surprisal and prosodic structure in accounting for phonetic variability in American English, Czech, Finnish, French, German, and Polish. After examining multiple phonetics cues (syllable duration, vowel dispersion, vocalic spectral emphasis, consonant center of gravity), they conclude that while prosodic structure accounts for phonetic variability to a certain extent, informativity also directly affects phonetic realizations. They view these findings as consistent with a weaker version of the Smooth Signal Redundancy Hypothesis. Andreeva et al. (2020) directly compare the effect of predictability on syllables at strong and weak boundaries with or without a phrasal accent. They find that predictability, estimated with surprisal from trigram language models, has a stronger effect on duration at stronger boundaries, while accent placement does not interact with surprisal.

1.3 Implications on the nature of lexical representation

Another aspect of durational variability’s relationship with predictability is whether such a relationship reflects online processing or the storage of variants in lexical representations. The online processing explanation aligns better with the Hyper- & Hypoarticulation (H&H) theory (Lindblom 1990) and the Smooth Signal Redundancy Hypothesis (Aylett and Turk 2004, 2006). The implication would be that the speaker is making constant adjustments to produce acoustic signals during speech production given the upcoming linguistic information. This consistent adjustment aims to reach a balance between the need for ease of articulation and the salience of perception (H&H) or between the amount of information redundancy and acoustic salience.

By explaining reduction and shortening based on lexical and contextual probabilities as an online process, the implication is that these variants are likely coming from canonical representations, which would be in line with studies that show the representational advantage of canonical forms over variants. The phonetic variants may only be associated with contexts that they often occur in instead of being part of the lexical representation. For example, Ernestus et al. (2002) show that less-reduced forms are more successfully recognized in isolation and a limited context (i.e., only flanked by a single vowel plus a consonant), even though in full contexts (i.e., embedded in a full sentence) the difference is drastically reduced. Similar results are reported in Kemps et al. (2004) with the phoneme-monitoring task where a listener has to identify a phoneme from reduced and non-reduced forms. In addition, Ranbom and Connine (2007) show that word forms with the canonical but less frequently produced [nt] sequence nonetheless triggered a shorter reaction time and were rated to be more well-formed than variants with the frequently-produced nasal-flap sequence.

On the other hand, there are studies showing that reduced forms of a word are stored as part of its lexical representation. Lavoie (2002) reports that homophones such as “for” and “four” have different sets of phonetic variants, suggesting that phonetic forms may be word-specific. Similar results are reported in Johnson (2007) and Gahl (2008), the latter of which shows that homophones with a higher lemma frequency have shorter phonetic realizations. If homophones have different sets of variants, it is possible that these variants are not produced via a unified online process. Instead, these variants may be stored as part of the representation.

A larger set of studies show the processing advantage of phonetic variants that are more frequent, some of which are non-canonical (Bürki et al. 2010; Connine 2004; Connine et al. 2008; Deelman and Connine 2001; Ernestus 2014; Pitt 2009; Pitt et al. 2011; Racine and Grosjean 2005; Ranbom and Connine 2007). For example, Connine (2004) shows that word-initial consonants in words having a word-medial flapping context (e.g., pretty) were identified more successfully when flap occurs than when the canonical [t] occurs. Bürki et al. (2010) show that the speakers’ report of relative frequencies of variants (schwa or non-schwa, as in [fənɛtr] vs. [fənɛtr] for the French word “fênetre”) for each word correlates with processing time in naming tasks and symbol-word association tasks, which was taken as evidence that at least in the production lexicon, both variants exist. The Ranbom and Connine (2007) study that found a processing advantage of canonical [nt] forms also found that words with a higher proportion of the nasal-flap variant show a processing advantage of this particular variant. As suggested in the literature (e.g., Ernestus 2014; Pitt 2009), this type of finding shows that if a reduced variant is a frequent and typical realized form of a word, the variant may be stored in representation to the extent that it has processing advantages.

Seyfarth (2014) discusses some possible implementations of the lexical storage of variants. The variants may still be phonologically abstract but include both reduced and unreduced forms (e.g., a /fənɛtr/ representation and a /fənɛtr/ representation). It is also possible that all episodes of production, including reduced and unreduced forms, are stored and collectively affect processing (i.e., Exemplar-based view of phonological representation, Johnson 2007; Pierrehumbert 2001). Reductions may also be stored as lexically specified changes to articulatory timing relationships (Lavoie 2002) in the framework of Articulatory Phonology (Aylett and Turk 2004; Browman and Goldstein 1990; Byrd 1996).

Another proposal that specifically aims to account for the representation of surface duration is to separate phonetic planning from phonological planning and the implementation of motor control (Turk and Shattuck-Hufnagel 2020a, b). In the proposed three-component speech production model, the specification of surface duration is handled by the phonetic planning component so that the observed surface durational variability is not simply a result of emergent timing mechanisms.

Overall, the implication concerning the relationship between phonetic variation, predictability, and lexical representation is if there are lexically-specific phonetic variants, we should be able to find word-specific reduction effects either on top of or instead of local context-specific effects, which has been shown in Mandarin (Tang and Shaw 2021). In other words, a word that is often reduced may be pronounced with a reduced form regardless of whether it occurs in a context where it often occurs since the reduced form is represented in the lexicon.

1.4 Informativity

Informativity (Cohen Priva 2008, 2012, 2015; Seyfarth 2014) is a measurement of predictability that has implications on whether phonetic variants follow word-specific or context-specific statistics. It is essentially a unit’s average contextual predictability across all the contexts where it occurs. The informativity of a unit u is given in (1), where c refers to a context, P(u|c) refers to a unit’s probability in a particular context, and P(c|u) is the weighting variable that refers to how often u occurs in that context. The probability is log-transformed with its sign flipped, as it is a measurement of surprisal.

In other words, each unit type has one informativity measurement. This is different from contextual predictability measured in n-gram probabilities or surprisal, which is variable between tokens of a word in different contexts. Previous studies on informativity have often calculated this measure based on both previous and following contexts. In terms of word informativity, “current” is an example low-informativity word mentioned in (Seyfarth 2014), since it is usually predictable (i.e., occurring with words such as “events”) and its occurrence does not add much information. In contrast, “nowadays” is an unpredictable and thus highly informative word as there are no particular contexts where it often occurs. Their contextual probabilities are plotted in Figure 2 (Figure 1, p. 142 in Seyfarth 2014).

Figure 2:

Distribution of token probabilities of current (gray) and nowadays (black) given the previous context (log-transformed with base 10). Density is higher for current on the left (higher probability) so that overall it has low informativity. The opposite is true for nowadays, with its density higher on the right, suggesting that it has high informativity, as its probability is low in the contexts that it occurs. Tokens and probabilities are taken from the Fisher scripts (Cieri et al. 2005) and COCA (Davies 2008). From Figure 1, p. 142 in Seyfarth (2014).

Informativity allows for testing the hypothesis that units with a certain level of predictability will have a particular type of phonetic variant even in a context where they are not as predictable. For example, for a high-informativity word such as “current,” the hypothesis is that it occurs in a reduced form even in contexts where it does not often occur, which would be an argument that the reduced form, being more frequent overall (as the word is generally not very informative), is stored as part of its lexical representation. These studies have found that high-informativity units exhibit reduction, and the effect is either comparable or stronger than the effect of local measurements of surprisal, at both the word level (Seyfarth 2014) and the segmental level (Cohen Priva 2008, 2012, 2015, 2017). Overall, these findings lend some support for the hypothesis that there is some consistency between a word’s general informativity and its phonetic realization that goes beyond specific local contexts.

1.5 Current study

Building on these previous works, this study aims to directly compare the effects of surprisal and informativity in different prosodic conditions. The research target is syllable duration in the spontaneous speech of Taiwan Southern Min. There are three main research questions.

First, is the effect of predictability on syllable duration consistent across different positions before a prosodic boundary? Given the known effect of pre-boundary lengthening, this question essentially asks whether the effect of predictability is neutralized by boundary effects, as predicted by the Smooth Signal Redundancy Hypothesis (SSRH) as extended by Turk (2010) to cover the relationship between boundary signaling and information redundancy.

Following similar formulations in the recent literature (Malisz et al. 2018; Andreeva et al. 2020), we take the strong form of SSRH to be predicting that once the prosodic factors are controlled for, no effects of predictability would be found. This is because the relationship between information redundancy and phonetic variability is moderated through the phonologized and conventionalized relationship between information redundancy and the prosodic structure. In the context of this study, the hypothesis posits that boundary marking already accounts for the need to strengthen acoustic signals for units toward a boundary that are often high in information content. In targeting the effect of pre-boundary lengthening as a prosodic factor, the control is enforced by examining the predictability effects at each specific prosodic position before a boundary, which is a method hinted by Turk (2010) in testing the modulating effect of boundary marking and signal redundancy. A strong form of the hypothesis is consistent with an outcome where predictability effects at a syllable position are completely erased.

A weaker version of the hypothesis, on the other hand, would be supported by a finding showing that the presence of pre-boundary lengthening in a certain prosodic position weakens predictability effects on durational variability in that very position. The third scenario is that predictability effects remain unaffected regardless of whether a particular prosodic position exhibits prosodic lengthening.

Potential results and the corresponding implications regarding this hypothesis are summarized in (2).

Second, this study examines whether the effects of informativity (if successfully replicated) and surprisal would behave differently in their interaction with prosodic phrasing. If the effect of informativity reflects the storage of variants for specific words regardless of information contexts, we would expect its effect to be less modulated by the prosodic structure as well. In other words, a more consistent effect of informativity would lend stronger support for this lexical hypothesis on phonetic variants. On the other hand, since surprisal reflects local information contexts, it is more likely to be sensitive to prosodic context as well, unless a strong view of the Smooth Signal Redundancy Hypothesis is adopted. These hypotheses are summarized in (3).

Finally, this study asks whether the effects of various predictability measurements, especially informativity, a global summary of a word’s local contextual predictability, can be replicated in Taiwan Southern Min, as research on probabilistic reduction and predictability effects has mainly been conducted in English and related Indo-European languages, with a few recent exceptions (Japanese in Hashimoto 2021; Shaw and Kawahara 2019; Mandarin in Tang and Shaw 2021, and Kaqchikel Mayan in Tang and Bennett 2018).

Taiwan Southern Min has similarities and differences with the aforementioned languages, making it an interesting target for extending the width and depth of the coverage of types of languages for research on predictability and probabilistic phonetic reduction. It is a language with lexical tones and a more syllable-timed rhythm, which marks a contrast with the pitch-accent system and mora-timed rhythm in Japanese and the lexical stress system and stress-timed rhythm in English. While Taiwan Southern Min is similar to Mandarin in these prosodic and rhythmic aspects, which extends the recent research on tone languages and predictability, it also has some differences that may contribute to differences in durational patterning. For example, it has a more complicated tonal system (seven tones, as shown in Table 1) than Mandarin. It also has a syllable structure that features plosive coda, which potentially introduces more durational variabilities as tones with plosive coda have been described as ‘short tones’.

In terms of the phonetics of prosodic boundaries, previous research has shown that Taiwanese has a consistent disyllabic domain of pre-boundary lengthening (Wang and Fon 2012, 2015). The difference between prosodic realizations at intonational phrase boundaries and intermediate phrase boundaries is mainly at the strength of lengthening at the position right before the boundary (Wang 2013). A disyllabic domain of pre-boundary lengthening makes Taiwan Southern Min similar to Taiwan Mandarin but different from Japanese and English (Fon 2002; Fon et al. 2011). Fon et al. (2011), in discussing the Mandarin results, raise a possibility that a syllable-timed rhythm and a lack of stress contrasts may attribute to Mandarin’s reliance on a larger domain to signal the presence of a boundary. By examining predictability effects on phonetic realizations in Taiwan Southern Min, this study may identify another dimension of differences between languages with different prosodic and rhythmic structures.

Tone sandhi is another element in Taiwan Southern Min’s prosodic structure that may play a role in durational variability, which does not have an apparent counterpart in Mandarin. In connected speech, all syllables except for the one near the right edge of a “tone sandhi group” undergo tonal alternation, as shown in (4). The inventory of lexical tones and tonal alternations are shown in Table 1. The tone sandhi group has been described as a level in the prosodic hierarchy of Taiwan Southern Min (Peng and Beckman 2003) and has been assumed to be heavily governed by morphosyntactic organization (Lin 1994), even though there may be more factors contributing to its variations (Pan et al. 2019; Pan and Huang 2020). It has been shown that syllables at the right edge of tone sandhi groups have a longer duration, a wider range of F0, and creakier voice quality (Kuo 2013). While Mandarin also has tone sandhi, it is limited to the alternation of a single tone (i.e., a contour tone changes it tonal value when followed by another contour tone in domain-medial positions) so that the phrasing behavior is only observable in specific contexts (see Shih (2017) for example).

In short, Taiwan Southern Min is an interesting case of study for the interaction between predictability and prosodic lengthening because it is another language that differs in prosodic and rhythmic structure from the heavily studied Indo-European languages such as English and the recently studied Japanese. Also, mirroring differences between these types of languages have been found in the phonetic realizations of duration at prosodic boundaries. Furthermore, Taiwan Southern Min has systematic differences with the prosodically and rhythmically similar Mandarin, adding depth and variability in the inquiry on predictability and phonetic realization in languages with lexical tones and a syllable-timed rhythm.

Results show a consistent effect of informativity on top of the effects of surprisal and lexical frequency. Higher informativity and surprisal are generally associated with longer syllables. As for the interaction with prosodic positions, there is a general weakening of predictability effects as a syllable is closer to the boundary, especially for the pre-boundary syllable, which has the most substantial final lengthening. However, the effect of word informativity appears to be least modulated by this effect of boundary marking. These findings suggest that word-specific duration variants may exist for Taiwan Southern Min, with its durational profile predicted from informativity. Overall, the findings are consistent with a weak version of the Smooth Signal Redundancy Hypothesis: prosodic marking affects but does not completely eradicate predictability effects. Concerning lexical representation, the findings are more in line with the hypothesis that phonetic variants are lexically stored, as informativity is shown to be a competitive predictor of syllable duration when compared with surprisal and lexical frequencies.

The rest of the paper is organized as follows. Section 2 describes the data, variables, and modeling methods. Section 3 reports the results and Section 4 discusses and implications.

2.1 Speech corpus

Durational data were extracted from a corpus of 8 hours of Taiwan Southern Min spontaneous speech (Wang and Fon 2013). Sixteen speakers each contributed around 30 min of recordings. The speakers were evenly split in gender and between two age groups. The “older” and “younger” groups are divided by whether the speakers were born before or after 1975. All speakers were from the same dialectal region (Taichung). Transcription follows the Peh-Oe-Ji Romanization system. The alignment between the transcriptions and the speech recording was done at the syllabic level. The corpus also contains annotations of discourse and prosodic boundaries. Discourse annotation was conducted independently of the sound files. The transcriptions were segmented into clausal units, and the relationship between neighboring units was labeled according to the degree of separation of their topics. Prosodic annotations identify two levels of prosodic boundaries (intonational and intermediate phrases).

For this study, additional transcriptions were added following the writing system released by Taiwan’s Ministry of Education (MOE). We also added word segmentation by using forward maximal matching based on the Taiwan Southern Min Dictionary published by the MOE. To investigate the effect of tone sandhi grouping, an additional tier of annotation was done on whether a syllable was pronounced with the base tone or the sandhi tone.

To have a more stringent control of boundary type, this study focuses on the right edges of discourse units that are also intonational phrases. Among the 9981 clausal boundaries marked in this corpus, 6567 (65.79%) correspond to an intonational phrase boundary, and 1421 (14.23%) correspond to boundaries of smaller prosodic phrases. The total proportion of discourse boundaries (80.06%) that match with prosodic breaks is similar to observations in French and Mandarin reported in Prévot et al. (2015).

Since the targets were final, penultimate, ante-penultimate, and initial/medial positions before a boundary, this study only analyzed discourse units that were longer than three syllables, which amounted to 5,699 discourse units (57.1% of all boundaries in the data set). The selected data were further filtered to exclude disfluencies and code-switching, and additional prosodic breaks within the discourse unit. The final data set going into the analyses contained 40,443 syllables (31,600 words).

The focus on syllable duration is motivated by the goal of testing how predictability effects interact with pre-boundary lengthening. Research on pre-boundary lengthening in Taiwan Southern Min and comparisons with other languages have always been made at the syllabic level instead of the word level, and typological differences have been observed in terms of the lengthening domain counted in syllable (Fon 2002; Fon et al. 2011; Wang and Fon 2012, 2015). Therefore, targeting syllable duration allows for directly comparing predictability effects in different syllable positions.

2.2 Language modeling corpus

The corpus for language modeling was a collection of Taiwan Southern Min written texts curated by Iunn (2005) originally for a study on lexical frequencies.^[1] The corpus contains texts from various genres, including literary works (fictions, proses, poems, biographies, play scripts, etc.), journalist reports, conversation transcripts, commentaries, and academic writings. Data were available in two different writing conventions: The “Hanlo” version, which contains a mixture of Chinese characters and romanizations, was used in this study. The other version was the fully romanized Peh-oe-ji system.

Additional preprocessing was performed on the “Hanlo” texts to transform the writing to be consistent with the writing convention advocated by the MOE in Taiwan. In addition, word segmentation was done following the MOE dictionary with the maximal length matching method, which is similar to the original segmentation method reported by Iunn. Given these preprocessing measures, the final word count for this corpus is 4.68M (from 5.96M syllables/characters).

Trigram language models were trained with modified Kesner-Ney smoothing (Chen and Goodman 1999) at the word level using the SRILM toolkit (Stolcke 2002; Stolcke et al. 2011). In addition to training models to obtain probabilities given the previous context, models that took sentences from the backward direction were also trained to obtain surprisal and informativity given the following context. This was motivated by Seyfarth’s (2014) similar method and the finding that informativity and surprisal given the following context account for more variances in word duration than informativity and surprisal given the previous context.

By using estimates of word-level predictability in modeling syllable duration, we are making the assumption that word-level predictability may percolate to syllable/phoneme levels and affect syllable duration, which is a divergence from other studies that directly model word duration with word-level predictability (e.g., Seyfarth 2014; Sóskuthy and Hay 2017; Tang and Shaw 2021). To entertain the alternative view that syllable/phoneme-level predictability is more relevant in modeling word duration, syllable/phoneme-level language models were also trained to obtain corresponding predictability measurements, and the results with syllable/phoneme-level predictability measurements will be presented in an additional analysis (Section 3.3.2).

It should be noted that even though trigram surprisals are also used in the literature (e.g., Malisz et al. 2018; Andreeva et al. 2020), bigram surprisals were used in this study for three reasons. First, it has been shown that using trigrams instead of bigrams only produces negligible improvement in predicting reductions in speech (Jurafsky et al. 2002). Second, using bigram surprisal allows direct comparisons between surprisal and informativity, the latter of which is based on bigrams according to the most relevant literature. Finally, by using bigram probabilities (as estimates of lexical frequency, to be discussed later) from trigram models, we obtain bigram probabilities that are adjusted given the distribution of bigrams in trigrams and avoid over-counting of bigrams that simply occur frequently in a few trigrams. As discussed in the next section, the same reason motivates our use of unigram surprisal from the language models to represent lexical frequency.

2.3 Variables in the regression analyses

Each syllable in the corpus was annotated with predictability variables (bigram informativity, bigram surprisal, unigram surprisal, and neighborhood density) and control variables on phrasing (position in a prosodic phrase, length of the prosodic phrase, sandhi boundary), word-level information (word length), and syllable-level information (baseline duration, surface tone).

2.4 Modeling procedure

We used mixed-effects models using the lmer() function in the lme4 package (Bates et al. 2015) with the fixed effects mentioned in the previous section. We also added random intercepts for Morpheme (monosyllabic), Word, and Speaker. After a full model was fit, each predictor’s contribution was tested by using the loglikelihood test to compare the full model to a reduced model with the predictor removed. If the model with an absent predictor did not significantly change the fit (alpha = 0.15 following Seyfarth 2014), the predictor was removed. Step-wise removal started from the least contributing predictor until further removal resulted in a significant difference between the full and final models.

We performed additional analyses where a model contained only one of the five predictability variables along with other control variables and random intercepts. These analyses were motivated by the discussion and recommendation of Wurm and Fisicaro (2014): in cases where collinearity is a concern, estimating the effect size of each variable in such simultaneous regression analyses is preferable to estimating effect sizes directly from multivariate regressions, regardless of whether residualized variables are used.

These procedures were performed with the entire dataset and in each of the four categories of boundary position (i.e., initial/medial, ante-penultimate, penultimate, final) to examine the effectiveness of these predictors in each position.

3.1 Overall results

All variables listed in Section 2.3 significantly contributed to the fit of the entire set of data points except for informativity given the previous word (β = −0.0046, p = 0.72) and unigram surprisal (β = 0.0133, p = 0.33). Significant fixed effects are summarized in Table 2. Summary of random effects in Table 3 shows that the random intercept of Monosyllabic morpheme accounts for more residual variances than the intercepts of Word and Speaker.

Among the predictability variables, higher informativity given the next word and surprisal given previous and next words are positively correlated with syllable duration. Informativity and surprisal are also similar in that they are better predictors of syllable duration when conditioned on the next word, as opposed to conditioned on the previous word.

Other variables in Table 2 show the expected effects. There is a positive relationship between baseline duration from a syllable’s segmental content and its actual duration. When the speech rate is faster, syllable duration is shorter. Syllable duration is longer when closer to a prosodic boundary, as shown in Figure 3. Post-hoc analysis (with the lsmeans package; Lenth 2016) of all pairs of comparison between neighboring positions show that syllables closer to the boundary are significantly longer (p < 0.0001 for all comparisons). In other words, there is a trisyllabic domain of pre-boundary lengthening. Syllables in the base tone, i.e., next to a tone sandhi group boundary, are also longer. Syllables in longer words tend to be shorter (main effect on word length). Neighborhood density shows a negative correlation with syllable duration, i.e., syllables with more phonological neighbors tend to be shorter. This is similar to the findings in Gahl et al. (2012) and Gahl and Strand (2016). Finally, the effects of surface tone show that particles and syllables with de-stressed tones are shorter.

Figure 3:

Least square means of syllable duration (z scores) in each prosodic position.

Next, we present the effect sizes of all five predictability variables in five simultaneous regression models. Each model only has one of the predictability variables and the control variables. As discussed by Wurm and Fisicaro (2014), this method allows for a more direct interpretation of the effect sizes. The results are shown in Table 4. Informativity conditioned in both direct directions has larger effect sizes than other variables, with informativity conditioned on the following word having a bigger effect. Unigram surprisal, which did not significantly improve model fit when the other five predictability measurements were present, becomes a significant effect in this setting, with an effect size that is bigger than surprisals. The discrepancy may be attributed to unigram surprisal’s high correlation with informativity and bigram surprisal, especially the former (as shown in Figure 4). As a result, informativity and surprisal explain away a large chunk of variances that unigram surprisal could have accounted for. This is a scenario that has been pointed out by Cohen Priva and Jaeger (2018) in a computational study on the relationship between frequency, informativity, and contextual predictability: They show that when informativity is not controlled for, there is an increased likelihood of observing spurious effects of frequency and predictability in the statistical analysis.

Figure 4:

Correlation between predictability variables with word-level modeling.

To summarize, informativity conditioned on the next word contributes to explaining durational variability on top of surprisal. According to multivariate mixed-effect modeling, it is a better predictor of syllable duration than lexical frequency (unigram surprisal), even though in simpler settings with only one predictability measurement, unigram surprisal has a comparable effect size.

3.2 Results in each prosodic position

This section reports the role of predictability in accounting for syllable durations in different prosodic positions. The analyses are again reported in two steps. We first present whether these measurements are significant fixed effects in the final model for each prosodic position following the step-wise elimination method. Then we report the effect sizes of the predictability variables in simultaneous regression analyses for a more straightforward interpretation.

Table 5 summarizes whether informativity and surprisal are significant predictors of syllable duration in the final model for data in each category of syllable position in an utterance. As the summary shows, there are indeed variations on whether a variable shows up as a significant predictor in each syllable position. There is no single predictor that is consistent across different positions.

Informativity conditioned on different directions exhibits a complementary relationship in this set of analyses: Informativity given the previous word is a predictor of syllable duration across ante-penultimate, penultimate, and final positions, while informativity given the next word is only a significant predictor in the initial/medial positions. This suggests that the significant effect of informativity given the following word in the main analysis may be carried by the initial and medial positions.

Surprisals conditioned in both directions are significant predictors in the initial/medial positions. In addition, surprisal given the following word is also a significant predictor in the penultimate and final positions but has a negative estimate in the final position.^[2] In other words, the positive effect of surprisal on syllable duration is mostly limited to syllables away from a discourse boundary where pre-boundary lengthening is not in effect.

Unigram surprisal, which was not a significant fixed effect in the overall analysis, is a significant predictor in penultimate and ante-penultimate positions in this analysis. Finally, neighborhood density is a consistent predictor except for the boundary position.

Next, we present the effect sizes of these predictability variables in simultaneous models in Figure 5. The major trend is that all predictors converge towards a small effect size in the final position, suggesting a nearly absolute neutralizing effect of a prosodic boundary. We also see that informativity is the most resistant to this neutralizing effect, as its effect sizes are larger than other measurements in the positive direction. These observations are confirmed by the statistical tests, summarized in Table 6.

Figure 5:

Effect sizes of predictability variables in simultaneous mixed-effects models in each prosodic position. Error bars indicate the standard error of β.

To sum up, there is evidence that predictability effects are neutralized towards a discourse and prosodic boundary, especially in the final position before the boundary, where the pre-boundary lengthening effect is the strongest. Informativity, especially when estimated given the previous word, is the most resistant to such a neutralizing effect.

3.3 Additional and exploratory analyses

3.3.1 Relationship between informativity and surprisal

This section discusses the relationship between informativity and surprisal. Since informativity is a weighted average of bigram surprisal for each word, it is important to understand how informativity and surprisal’s predictions of syllable duration diverge from each other. For this purpose, regression analyses were run by including informativity and surprisal conditioned in the same direction and the interaction between them.

Informativity and surprisal given the previous word have a nearly significant interaction in the penultimate position (β = −0.0137, SE = 0.0078, p = 0.08). The negative estimate suggests that when both informativity and surprisal were higher, the individual effect of each variable became smaller. This interaction can be observed by the less steep slope for lighter regression lines in the “penultimate” panels in Figures 6 and 7.

Figure 6:

Syllable duration (z scores) as a function of informativity given the previous word at three levels of surprisal (z-scores = 2, 0, and −2).

Figure 7:

Syllable duration (z scores) as a function of surprisal given the previous word at three levels of informativity (z-scores = 2, 0, and −2).

These figures also show that this neutralizing trend can be observed in other prosodic positions. It is also worth noting that in Figure 7, the relatively consistent pattern of lighter regression lines above darker ones, especially towards the left side of the graphs, suggests that syllables in even in low-surprisal contexts, high-informativity words were still longer than syllables in low-informativity words. The same separation is less obvious for surprisal given different levels of informativity in Figure 6.

On the other hand, informativity and surprisal conditioned on the following word show more interactions with each other in the penultimate position (β = −0.0247, SE = 0.0091, p < 0.01) and the ante-penultimate position (β = −0.023, SE = 0.0091, p < 0.01). Similar to informativity and surprisal given the previous word, the interaction was in the opposite direction of the main effects: When informativity and surprisal given the next word are both high, the lengthening effect is capped. Figures 8 and 9 visualize the interactions. In Figure 9, we again observe the trend that separation between words with different levels of surprisal was consistent across different levels of surprisal (except for the final position), which is not true for the separation between different surprisal levels across different degrees of informativity, as shown in Figure 8.

Figure 8:

Syllable duration (z scores) as a function of informativity given the next word at three levels of surprisal (z-scores = 2, 0, and −2).

Figure 9:

Syllable duration (z scores) as a function of surprisal given the next word at three levels of informavity (z-scores = 2, 0, and −2).

To sum up, informativity and surprisal mostly interact in the direction where highly informative words are not as lengthened in high-surprisal contexts and vice versa, which may be interpreted as a ceiling effect for predictability.

3.3.2 Syllable/morpheme-level modeling

We explored an alternative analysis using predictability measurements at the level of monosyllabic morphemes. Trigram language models were trained without word segmentation, i.e., monosyllabic morphemes were treated as separate units. Similar to the main analysis, language models were also trained in both directions. Bigram informativity and surprisal conditioned on previous and following contexts were obtained from the trained models.

The same set of fixed effects from the main analysis was used, except for word length, since word-level units were not taken into account in this analysis. Similarly, Word was not included as a random intercept, leaving the model with random intercepts of Speaker and Phoneme.

Fixed effects from regression analysis with step-wise elimination are shown in Table 7, with the random effects shown in Table 8. All variables except for unigram surprisal contributed significantly to model fit. This is slightly different from the main analysis reported in Section 3.1, where unigram surprisal and informativity given the previous word were not significant contributors to model fit.

Table 7:

Summary of fixed effects in the syllable/morpheme-based analysis.

	β	SE	t	p(t)	p(χ 2)
(Intercept)	−0.1179	0.0472	−2.50
Baseline duration	0.2470	0.0095	25.98		p < 0.0001
Speech rate	−0.1584	0.0041	−38.39		p < 0.0001
Neighborhood density	−0.0583	0.0111	−5.26		p < 0.0001
Informativity given previous	0.0304	0.0132	2.30		p < 0.05
Informativity given following	0.0290	0.0130	2.23		p < 0.05
Surprisal given previous	0.0875	0.0042	20.91		p < 0.0001
Surprisal given following	0.0593	0.0044	13.57		p < 0.0001
Position: ante-penultimate	0.1591	0.0100	15.90	p < 0.0001	(p < 0.0001)
Position: penultimate	0.3931	0.0105	37.54	p < 0.0001
Position: final	1.0290	0.0125	82.15	p < 0.0001
Base tone	0.2011	0.0117	17.20		p < 0.0001
Surface tone: de-stressed	−0.0848	0.0430	−1.97	p = 0.05	(p < 0.0001)
Surface tone: T2	0.0333	0.0171	1.95	p < 0.05
Surface tone: T3	−0.0808	0.0198	−4.07	p < 0.001
Surface tone: T4	−0.0617	0.0302	−2.04	p < 0.05
Surface tone: T5	0.0823	0.0293	2.81	p < 0.01
Surface tone: T7	0.0379	0.0171	2.21	p < 0.05
Surface tone: T8	−0.0657	0.0293	−2.24	p < 0.05
Surface tone: particle	−0.6445	0.0462	−13.96	p < 0.0001

Table 8:

Summary of random effects in the syllable/morpheme-based analysis.

Groups	Variance	Std. Dev.
Monosyllabic morpheme	0.06366	0.2523
Speaker	0.02958	0.1720
Residual	0.39325	0.6271

Compared with word-level informativity and surprisal, there is a smaller degree of correlation between morpheme-level informativity and surprisal. On the other hand, the correlation between unigram surprisal and informativity is still strong, and so is the correlation between informativity conditioned in two directions, as shown in Figure 10. For this reason, we again report each predictability measurement’s effect sizes in simultaneous regression models that only included one of the predictability variables along with other control variables.

Figure 10:

Correlations between morpheme-level predictability variables.

Results are shown in Table 9. Surprisal given the previous syllable/morpheme has the largest effect size, followed by informativity conditioned in both directions, which are in turn followed by surprisal given the following syllable/morpheme and unigram surprisal. These results differ from what we saw in the analysis with word-level informativity and surprisal, where unigram surprisal and informativity had larger effect sizes than surprisal. In other words, while a word’s overall frequency and weighted average of contextual predictability could predict syllable duration better than the predictability of specific word tokens, the predictability of specific syllable/morpheme tokens were better predictors of syllable duration than a syllable/morpheme’s overall frequency or average predictability.

Table 9:

Summary of fixed effects in the syllable/morpheme-based simultaneous mixed-effects models.

	β	SE	t	p(χ 2)
Informativity given previous	0.0781	0.0111	7.02	p < 0.0001
Informativity given following	0.0791	0.0109	7.26	p < 0.0001
Surprisal given previous	0.0869	0.0042	20.91	p < 0.0001
Surprisal given following	0.0579	0.0044	13.27	p < 0.0001
Unigram surprisal	0.0556	0.0097	5.74	p < 0.0001

The positioning of word boundaries, which is not a variable in this set of morpheme-based analyses, is a potential factor that explains bigram surprisal’s success in this setting. Given syllables x and y, the bigram surprisal p(x|y) is likely to be low if x and y are within the same word, and the duration of x is likely to be shortened as well. This covariation between bigram surprisal and syllable duration based on the location of word boundaries may contribute to the success of surprisal in predicting syllable duration.

To test whether this is the case, we ran another set of analyses that included two additional variables: “L/R-side #” (# = word boundary), i.e., for each syllable, whether there was a word boundary on the left or right when word segmentation was considered. We first ran two simple mixed-effect regression models that had L/R-side # as the independent variable and surprisal given the previous/next word as the dependent variable, with “monosyllabic morpheme” added as a random intercept. The results showed that surprisal was significantly lower when the bigram did not cross a word boundary (L-side # predicting surprisal given previous morpheme: β = −1.04, SE = 0.01, p < 0.0001; R-side # predicting surprisal given next morpheme: β = −1.11, SE = 0.01, p < 0.0001).

Then we ran the main regression analysis while adding these two new variables. In the main model, L-side # had a significant effect on syllable duration: when the left side of a syllable was within the same word (β = −0.1545, SE = 0.01, p < 0.0001), the syllable was shorter. The same effect, however, is not found for R-side #.

Finally, we ran simultaneous regression analyses in each prosodic position for each predictability variable. The results are shown in Figure 11. There are two main differences from word-level analysis in Figure 5. First, informativity is no longer significant in the final position, i.e., unlike word informativity, the effect of morpheme informativity was neutralized in the final position. Second, the effect of surprisal given the previous morpheme was not neutralized in the final position. Again, this is likely a result of how the presence of word boundary before a syllable strongly predicts both syllable duration and bigram surprisal, leading to a robust effect of bigram surprisal in predicting duration.

Figure 11:

Effect sizes of morpheme-level predictability variables in simultaneous mixed-effects models in each prosodic position. Error bars indicate the standard error of β.

To sum up, similar to the analysis with word-level information, higher morpheme-level informativity and surprisal were correlated with longer syllable duration. These predictability effects were also neutralized towards a discourse and prosodic boundary. In the pre-boundary position, the effect of morpheme-level informativity was not as strong as word-level informativity, while surprisal given the previous morpheme remained a significant predictor in that position.