Skip to content
Publicly Available Published by De Gruyter March 21, 2023

Measuring Communication Quality of Interest Rate Announcements

  • Jonathan Benchimol ORCID logo EMAIL logo , Itamar Caspi and Sophia Kazinnik
From the journal The Economists’ Voice

Abstract

We use text-mining techniques to measure the accessibility and quality of information within the texts of interest rate announcements published by the Bank of Israel over the past decade. We find that comprehension of interest rate announcements published by the Bank of Israel requires fewer years of education than interest rate announcements published by the Federal Reserve and the European Central Bank. In addition, we show that the sentiment within these announcements is aligned with economic fluctuations. We also find that textual uncertainty is correlated with the volatility of the domestic financial market.

JEL Classification: A12; E44; E52; E58

1 Introduction

Being able to properly account for the relationship between central bank communication and financial stability is crucial for policymakers. The Federal Reserve (Fed) aims to build resilience in the financial system and communicate its “policy strategy as clearly and transparently as possible to help align expectations and avoid market disruptions” (Powell 2018) since information conveyed through central bank communication influences volatility in the financial markets (Benchimol, Kazinnik, and Saadon 2020).

In this paper, we explore developments in the readability of announcements published by the Bank of Israel (BoI) throughout the past decades and whether the clarity of written interest rate announcements is related to financial volatility.

Previous studies that analyze central bank policy announcements involve the construction of communication quality indices against which the effect of communication on financial variables (Brand, Buncic, and Turunen 2010), financial stability (Born, Ehrmann, and Fratzscher 2011), and the future path of interest rates (Bennani et al. 2020) is measured. In this paper, we use similar indices to measure the comprehension and information quality of interest rate announcements published by the Bank of Israel (BoI). We evaluate the extent to which announcement content is (1) accessible to the public; and (2) reflects the economic uncertainty situation and events.[1] We then relate these indices with volatility changes to understand the association between text characteristics and market dynamics.

2 Institutional Background

This section describes the main changes that took place in the structure of the BoI’s interest rate announcements since understanding them is key to interpreting our quantitative results correctly.

Throughout our sample period, four governors served at the BoI: Jacob Frenkel (1993–2000), David Klein (2000–2005), Stanley Fischer (2005–2013), and Karnit Flug (2013–2018). Nadine Baudot-Trajtenberg served from November 14 to December 23. The structure of the interest rate announcements changed during this period (Figure 1) and can be divided into three main periods: (1) up to 2006, (2) 2006–2016, and (3) 2016–2019.

Figure 1: 
Number of words of the BoI’s interest rate announcements. Source: Benchimol, Kazinnik, and Saadon (2020).
Figure 1:

Number of words of the BoI’s interest rate announcements. Source: Benchimol, Kazinnik, and Saadon (2020).

In the first period, the length of the interest rate announcement ranged from 200 words to 1000 words and, starting in 2000, included tables with data in addition to the text. The structure and topics discussed were not fixed: the main topics were inflation and the need to bring and maintain it in the target area, the exchange rate and financial stability, and in particular, the consequences of fiscal policy on economic stability. A short paragraph with a central policy message usually appears at the end of the announcement. The texts during this period (2000–2006), being highly redundant and not containing enough words, cannot be used for analyzing communication quality in further sections.

In the second period, the structure of the announcement was divided into two parts: the background conditions included a reference that grew longer over the years to a wide range of economic data: the inflation environment, financial activity, money aggregates, the foreign exchange market, real activity, the global economy, and more. The number of topics discussed expanded according to economic developments. For example, monetary aggregates (M1, M2) gained importance with the global financial crisis (GFC), which led to a paragraph describing their monthly evolution. Against the background of the GFC and the later increases in housing prices, paragraphs describing the developments in the global economy and the housing market were incorporated. The number of words (see Figure 1) increased from a thousand words to approximately 2500 between 2011 and 2016. The main monetary message was reflected in the main factors for the decision that appeared at the bottom of the announcement, which also included a paragraph expressing a policy statement.

In the third period, against the background of the transition to making interest rate decisions eight times a year (instead of monthly), another change was made to the structure of the interest rate announcements. The main messages appear in summaries on the first page, and the paragraph expressing the policy statement was used to present the forward guidance policy of the monetary policy committee (MPC). The economic developments were mainly presented in the text, as an economic story, with minimal mention of numerical data: these were transferred to a separate file of data that includes a wide variety of charts about the monetary and financial environment, real activity, and the global economy. The change shortened the message significantly, and its length averaged around 1200 words.

Until 2011, the monetary policy was determined by the governor. The governors indeed decided within a forum of managers that resembled the MPC structure, but legally the decision right was reserved for the governor alone. Accordingly, the interest rate announcement reflects the governor’s decision (the members of the forum would “vote” in favor of their preferred decision; however, this vote was not binding, and, in some cases, the governor made a different decision than the majority vote). In 2010, the new Bank of Israel law was enacted, and following it, in 2011, the Monetary Committee was established for the first time, consisting of three members from the BoI (the governor, the vice-president, and another employee who is not the governor) and three public representatives. As of October 2011, the interest rate announcement reflects the MPC’s decision, which is validated by a majority of votes. The topics highlighted in the announcement, the way they were presented, including the order of appearance and the words were carefully chosen by the monetary decision-makers – the monetary forum followed by the monetary committee. Although complete sentences were often repeated from message to message, there was variation in the degree of detail devoted to the various topics, sub-topics, and wording.

3 Communication Quality

The BoI announces interest rate decisions at predetermined dates alongside explanations of the reasoning behind the decisions.[2] For our analysis, we review the content of the English-language interest rate announcements published on the BoI’s official website between 2007 and 2018.[3]

3.1 Text Accessibility

We measure the accessibility of text within the interest rate announcements using the following two indices. The first one is the type-token ratio (TTR), which captures how the vocabulary used is varied and is calculated as the number of different words (e.g. types) in the text compared to the number of times these words appear (e.g. the number of tokens per type):

(1) T T R t = 100 ( T o t a l T y p e s t T o t a l T o k e n s t )

A high TTR value indicates the use of a broad vocabulary within the text that requires more significant effort and knowledge to understand.

The second index, text complexity, estimates the number of years of U.S. grade-school-level education required to understand a given text (Kincaid et al. 1975; Flesch 1979). Since long words and sentences increase communication complexity, the index compares the number of words and sentences with the number of syllables and words,[4] as follows:

(2) C o m p l e x i t y t = 0.39 ( T o t a l W o r d s t T o t a l S e n t e n c e s t ) + 11.8 ( T o t a l S y l l a b l e s t T o t a l W o r d s t ) 15.59

The initial formula at the origins of Eq. (2), upon which Kincaid et al. (1975) based their results, was developed by Flesch (1948) in an experiment conducted on school-aged children. Using McCall-Crabbs’ standard test lessons in reading, Flesch (1948) derived from these experiments correlations between word length (wl), sentence length (sl), and the average grade of children who could answer at least 50 percent of the test questions correctly.[5] These regressions led to the proposed Reading Ease (RE) formula: RE = 206.835 − 0.846wl − 1.015sl. This formula was tested by Flesch (1948) to match various texts and RE results, showing that the correlation between RE and effective RE, measured as the ability of children to understand the suggested texts according to their grade, was significant.

When revising Flesch’s RE formula, Kincaid et al. (1975) kept the same predictors but estimated new parameters using a criterion variable that relied on both supply words and multiple-choice test performance. For his experiment, Kincaid et al. (1975) had 531 Navy enlisted personnel take reading comprehension tests called Gates-MacGinitie Reading Test (GMRT) and passages from Rate Training Manuals. Then, multiple regression analysis was conducted using these grades as the criterion variable to validate the revised grade-level version of the Flesch (1948) formula. The Army General Classification Test scores, known for each subject, with the results of multiple regressions, constitute the conversion table from RE (Flesch 1948) to grade level (Kincaid et al. 1975), represented by the coefficients of Eq. (2).

The outcomes of our analysis reveal that the vocabulary variability (Figure 2, left panel) rose sharply in 2017 after years of relative stability and, at times, declined. This increase coincided with the change of the announcement format; i.e. shorter texts were used in combination with the provision of more detailed figures about the state of the economy.

Figure 2: 
Indices of variety in vocabulary in BoI interest rate announcements. Notes: The lines and green dots represent the trend lines and index values, respectively. The TTR is calculated over each announcement. The MATTR is determined over a moving average of 100 words in each announcement.
Figure 2:

Indices of variety in vocabulary in BoI interest rate announcements. Notes: The lines and green dots represent the trend lines and index values, respectively. The TTR is calculated over each announcement. The MATTR is determined over a moving average of 100 words in each announcement.

Overall, we have 143 documents in our sample, published monthly until January 2017 and eight times a year after, and containing about 2000 words on average.

The TTR index (Figure 2, right panel) can be sensitive to text length and thus may decline with longer texts.[6] To examine the robustness of our results, we devised a moving average of TTR values in a window that included 100 words (MATTR). Using this approach, a clear increase in text complexity since 2017 can be observed (Figure 2, left panel).

The Flesch-Kincaid text complexity index (Figure 3, left panel) exhibits relative stability with slight variance over time. Overall, a U.S. undergraduate-level education (14 years) was required to understand the BoI announcements between 2007 and 2018.

Figure 3: 
Indices of text complexity in BoI interest rate announcements. Note: The lines and green dots represent trend lines and index values, respectively.
Figure 3:

Indices of text complexity in BoI interest rate announcements. Note: The lines and green dots represent trend lines and index values, respectively.

A more precise examination indicated that the average sentence length declined after 2017 (Figure 3, middle panel) from about 27 to about 22 words per sentence (although levels increased at the end of the sample), making the texts less complex. In contrast, the average number of syllables per word rose (Figure 3, right panel), increasing text complexity. In sum, the two factors offset each other such that the complexity index remained stable.

This complexity index does not consider the content of the text and any field-specific knowledge or education that may be required to understand it, regardless of education level. However, it represents an average, and aggregated effects may compensate for heterogeneities. In addition, delivering technical and professional information can also be hampered by excessively low complexity levels.

Governance changes affect interest rate communications differently over time. Under Jacob Frenkel, the length of these communications remained low (Figure 1). These communications almost doubled in length because of David Klein and the dot-com crisis. Due to the limited number of words and the high amount of redundant text, communication quality is difficult to fully assess before 2008.

With the establishment of the Monetary Policy Committee and support for the new Bank of Israel Law, the Stanley Fischer period marked the most significant changes in these communications. Over the course of his mandate, the length of texts changed from 500 to 2500 words on average (Figure 1). This increasing dynamic was also influenced by the critical role played by the GFC and financial stability in monetary policy decisions, and did not significantly change lexical diversity (Figure 2) or readability (Figure 3). However, this dynamic increased the number of syllables per word (Figure 3), indicating a slight increase in text complexity.

During the mandate of Karnit Flug, the BoI reduced the frequency of interest rate decisions from monthly to eight times a year. This change involved a drastic decrease in the interest rate communication length, switching from an average of 2500 words to an average of 1200 words per interest rate announcement (Figure 1), which also contributed to the increase in lexical diversity (Figure 2) and complexity (Figure 3). Still, overall communication quality was hardly affected as these texts were more accessible (Flesc-Kincaid declined in Figure 3) and sentence length was reduced (see the mean sentence length, Figure 3).

The changes in governors with the governance of historical (dot-com and GFC) and legal contexts (Bank of Israel Law and the establishment of the Monetary Committee) influenced the BoI’s communications.

To interpret these indices objectively, we look at the interest rate announcements made by other central banks. Figure 4 compares the complexity and linguistic variety indices of the BoI’s announcements with the indices of the communications published by the Fed and the ECB. The Fed and ECB’s announcements require an average of about 17 and 19 years of education, respectively, to be understood. The BoI announcements are relatively more comprehensible, requiring an average of about 14 years of education to be understood. Furthermore, the Fed and the ECB’s linguistic variety indices are almost three times as high as that of the BoI on average.

Figure 4: 
International comparison of text complexity indices and linguistic variety. Note: Yearly averages based on interest rate announcements. Source: Benchimol, Kazinnik, and Saadon (2020).
Figure 4:

International comparison of text complexity indices and linguistic variety. Note: Yearly averages based on interest rate announcements. Source: Benchimol, Kazinnik, and Saadon (2020).

3.2 Contextual Uncertainty

In this section, we examine the quality of the information in the interest rate announcements based on the financial context of the words they contain.[7] Instead of analyzing divergence in tone (Sadique, Veeraraghavan, and Wachtel 2013; Tillman and Walter 2019), we build a textual uncertainty index. In the first stage, using standard dictionary-based methods that are designed to analyze financial texts, we identified words in the text that conveyed uncertainty;[8] for example, “risk,” “uncertainty,” “volatility,” probability,” and “variable.”

In the second stage, we calculate an index of textual uncertainty, which increases as the rate of words classified in the category of uncertainty increases. The uncertainty index (Figure 5) shows that the announcements contained relatively high contextual uncertainty around the time of the GFC. The contextual uncertainty significantly declined after that, particularly after 2016, before rising in response to the global economic uncertainties that transpired toward the end of the sample period.[9]

Figure 5: 
Index of uncertainty in the BoI interest rate announcements and TA-35 VIX. Source: Bank of Israel and Benchimol, Kazinnik, and Saadon (2020).
Figure 5:

Index of uncertainty in the BoI interest rate announcements and TA-35 VIX. Source: Bank of Israel and Benchimol, Kazinnik, and Saadon (2020).

The literature on dictionary-based classifications typically employs a unigram approach, which examines individual words independently. Our methodology aligns with this approach, acknowledging that it may lead to inaccuracies in instances where a two-term phrase, composed of independently negative words, is incorrectly classified as negative when it conveys a positive meaning (e.g. “uncertainty declined”). Despite this potential limitation, the unigram approach is well-suited for our specific corpus of relatively short texts, the structural characteristics of these texts (redundancies), and the dictionaries utilized in previous studies. Additionally, central bank announcements tend to avoid using double-negative structures and compensations between false positives and false negatives may be at play. Also, using a bigram or phrase-level classification would significantly decrease the information present in short texts, such as interest rate announcements.

We note several instances of correlation between our uncertainty index and real-world economic events. We also examine the correlation between uncertainty indices and risk levels in domestic markets, as reflected in the TA-25 VIX index.[10] A positive Pearson correlation of 0.47 was determined across the sample, while a distance correlation of 0.54 confirmed both linear and nonlinear linkages—causality direction or the assertion that uncertainty leads to fluctuations in VIX, or vice versa, could not be inferred from the data available. The announcements may contain analyses the public is unaware of, causing market volatility. In contrast, the text may describe VIX fluctuations that would have occurred in any case, even without the announcement. Further in-depth research is required to identify causality direction.[11]

4 Conclusion

Our findings, namely that the BoI’s announcements on interest rates are easier to read than those issued by the ECB or Fed and convey meaningful signals about what is happening in the economy, are encouraging. In addition, the changes made in the interest rate announcement structure and voting procedure over the past decade did not negatively affect the readability of the text and its signal.

We interpret these results as evidence that complexity and signal do not trade-off. Making the interest rate announcement more readable or less complex does not result in a decrease in the quality of economic information.

Sharing transparent communications with the public has become an essential central bank tool that significantly contributes to financial stability. The findings of this study indicate that understanding the BoI’s interest rate announcements requires a college-level education, albeit a considerably lower level of education than that needed to understand the Fed and the ECB announcements.

Furthermore, we show that introducing a new announcement format at the beginning of 2017 has made the texts less comprehensible. Finally, by deriving a text uncertainty index from the BoI’s interest rate announcements, we find a correlation between these announcements and economic events and preliminary evidence that contextual uncertainty correlates with changes in the TA-25 VIX index.


Corresponding author: Jonathan Benchimol, Research Department, Bank of Israel, Jerusalem, Israel, E-mail:
This paper does not necessarily reflect the views of the Bank of Israel, the Federal Reserve Bank of Richmond or the Federal Reserve System. We thank participants from the Bank of Israel, Harvard University, and the University of Houston research seminars for their useful comments. The replication files of most of the results presented in this paper can be found at https://doi.org/10.24433/CO.1132001.v1.

References

Baker, S. R., N. Bloom, and S. J. Davis. 2016. “Measuring Economic Policy Uncertainty.” Quarterly Journal of Economics 131 (4): 1593–636, https://doi.org/10.1093/qje/qjw024.Search in Google Scholar

Benchimol, J., S. Kazinnik, and Y. Saadon. 2020. “Communication and Transparency through Central Bank Texts.” In Paper presented at the 132nd Annual Meeting of the American Economic Association, January 3–5, 2020, San Diego.Search in Google Scholar

Benchimol, J., S. Kazinnik, and Y. Saadon. 2022. “Text Mining Methodologies with R: An Application to Central Bank Texts.” Machine Learning with Applications 8: 100286, https://doi.org/10.1016/j.mlwa.2022.100286.Search in Google Scholar

Bennani, H., N. Fanta, P. Gertler, and R. Horvath. 2020. “Does Central Bank Communication Signal Future Monetary Policy in a (Post)-crisis Era? The Case of the ECB.” Journal of International Money and Finance 104 (C): 102167, https://doi.org/10.1016/j.jimonfin.2020.102167.Search in Google Scholar

Born, B., M. Ehrmann, and M. Fratzscher. 2011. “Communicating about Macro‐Prudential Supervision – A New Challenge for Central Banks.” International Finance 15 (2): 179–203, https://doi.org/10.1111/j.1468-2362.2012.01301.x.Search in Google Scholar

Brand, C., D. Buncic, and J. Turunen. 2010. “The Impact of ECB Monetary Policy Decisions and Communication on the Yield Curve.” Journal of the European Economic Association 8 (6): 1266–98, https://doi.org/10.1162/jeea_a_00025.Search in Google Scholar

Flesch, R. F. 1948. “A New Readability Yardstick.” Journal of Applied Psychology 32 (3): 221–33, https://doi.org/10.1037/h0057532.Search in Google Scholar

Flesch, R. F. (1979). How to Write Plain English: A Book for Lawyers and Consumers. New York: Harper & Row.Search in Google Scholar

Kincaid, J. P., R. P. J. Fishburne, R. L. Rogers, and B. S. Chissom. 1975. “Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel.” In Research Branch Report Institute for Simulation and Training, Vol. 56, 8–75.10.21236/ADA006655Search in Google Scholar

Loughran, T., and B. McDonald. 2011. “When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks.” The Journal of Finance 66 (1): 35–65, https://doi.org/10.1111/j.1540-6261.2010.01625.x.Search in Google Scholar

Powell, J. H. (2018). “Monetary Policy Influences on Global Financial Conditions and International Capital Flows.” In Speech Delivered at the 2018 Eighth High-Level Conference on the International Monetary System, May 8.Search in Google Scholar

Praet, P. (2017). “Communicating the Complexity of Unconventional Monetary Policy in EMU.” In Speech Delivered at the 2017 ECB Central Bank Communications Conference “Communications Challenges for Policy Effectiveness,” Accountability and Reputation.Search in Google Scholar

Sadique, S., F. In, M. Veeraraghavan, and P. Wachtel. 2013. “Soft Information and Economic Activity: Evidence from the Beige Book.” Journal of Macroeconomics 37 (C): 81–92, https://doi.org/10.1016/j.jmacro.2013.01.004.Search in Google Scholar

Tillmann, P., and A. Walter. 2019. “The Effect of Diverging Communication: The Case of the ECB and the Bundesbank.” Economics Letters 176 (C): 68–74, https://doi.org/10.1016/j.econlet.2018.12.035.Search in Google Scholar

Received: 2022-10-15
Accepted: 2023-01-26
Published Online: 2023-03-21

© 2023 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 5.3.2024 from https://www.degruyter.com/document/doi/10.1515/ev-2022-0023/html
Scroll to top button