We examine how new medical information on drug safety impacts preventive health care use. We exploit the release of the findings of the Women’s Health Initiative Study (WHIS) – the largest randomized controlled trial of women’s health – which demonstrated in 2002 the health risks associated with the long-term use of hormone replacement therapy (HRT). We first show that, after the release of the WHIS findings, HRT use dropped sharply among post-menopausal women. We then estimate the spillover effects of the WHIS findings on preventive care by means of a difference-in-differences methodology comparing changes in preventive care use among 60 to 69 year-old women (who have high rates of HRT use) with the change among women aged 75 and above (who have much lower rates of HRT use). Using data from the Behavioral Risk Factor Surveillance System for the period 1998–2007, we find that women aged 60–69 had statistically and economically significant declines in their annual mammography checks, checkups, cholesterol checks and blood stool tests, when compared to older women.
In the United States, consumers obtain medical information from a variety of sources, including results of new medical research, direct-to-consumer advertisements (DTCA), Food and Drug Administration (FDA) warnings and announcements and medical report cards. The production and dissemination of new medical information has substantial costs. National spending on research and development almost doubled during the past decade, from $25.5 billions in 2000 to $48.1 billions in 2012.  Annual spending on DTCA reached $4,237 millions in 2005 (Donohue, Cevasco, and Rosenthal 2007). Similarly, significant sums of public resources are devoted to the preparation and distribution of medical report cards and FDA announcements and warnings. Consequently, the question of how new medical information impacts consumers has long been of interest to both researchers and policy-makers.
While an extensive body of research shows that individuals change their consumption patterns in response to new drug-related information, the parallel question of how new medical information on drugs impacts a healthy lifestyle and preventive health behavior is largely unexplored. In this paper, we investigate how findings of new medical research and the subsequent policy warnings on the safety of preventive drugs impact preventive health care use. In particular, we examine the response to the findings of the Women’s Health Initiative Study (WHIS) which demonstrated in 2002 that long-term Hormone Replacement Therapy (HRT) increases the risk of heart attacks, stroke, blood clots and breast cancer among healthy post-menopausal women. 
We focus on the information provided by the WHIS for a number of reasons. First, there are no real substitutes for the preventive care provided by HRT. The main reason for HRT use among post-menopausal women was to prevent post-menopausal osteoporosis and heart disease, with the latter gaining more emphasis during the 1990s.  The published guidelines for the primary prevention of cardiovascular disease indicate that HRT has been the only pharmacological intervention recommended to reduce the risk of cardiovascular disease.
Second, the WHIS is an ideal source of exogenous variation in drug safety information as it is the largest randomized controlled trial of women’s health ever undertaken.  The HRT component, the most debated and closely followed part of the study, was initially scheduled to run until 2005 but was abruptly terminated on July 9, 2002 due to increased health risks among the treated. The story made major headlines across the country. Soon after, in January 2003, the FDA announced a formal name change for HRT drugs as “menopausal hormone therapy”, emphasizing its short-term indication in relieving menopausal symptoms, and mandated a boxed warning on the label of HRT drugs informing consumers about the increased risks associated with post-menopausal HRT use.
The WHIS results had a remarkable effect on the use and prescription of HRT drugs among post-menopausal women. Figure 1(a) shows the share of women aged 60–69 and 75 and older who reported having purchased at least one HRT product for the period 1998–2007.  While HRT use is significantly lower among older ages, the figure shows that HRT use declined by roughly 50% among both age groups: fraction of 60–69 (75+) year-old women using HRT declined from 31% (12%) during the WHIS period to 14% (6%) during the post-WHIS period. Similarly, Figure 1(b) shows that the share of office-visits among post-menopausal women that resulted in an HRT prescription declined by 50% from the pre-announcement period of the WHIS results (1998–2002) to the period thereafter (2003–2007).
We examine the effects of the WHIS findings on the preventive health care use of post-menopausal women by means of a difference-in-differences methodology. Given the difficulty in finding an appropriate control group to describe the counterfactual of how post-menopausal women would have behaved in the absence of the WHIS intervention, we identify the effects by comparing changes in outcomes among more intensively treated 60–69 year-old women with the change among less intensively treated 75+ year-old women.  Using data from the Behavioral Risk Factor Surveillance System (BRFSS) for the period 1998–2007, we find statistically and economically significant reductions in the preventive health care use of women aged 60–69 relative to women aged 75 and above. In particular, our results suggest that women aged 60–69 group had a decline of 4.8% in annual mammography checks, 2.14% in annual checkups, 3.54% in annual cholesterol checks and 8.37% in annual blood stool tests, when compared to older post-menopausal women. These results are robust to a host of checks.
This paper’s empirical work is most related to the theory of risk compensation. According to this theory, people adjust their behavior in response to perceived changes in risk in such a way that keeps the risk level constant. This offsetting behavior has been referred to as the “Peltzman Effect” or “lulling effect” in previous studies. Empirical evidence on the presence of offsetting behavior is mixed. For example, Kahn (1999) studies the effects of the availability of new diabetic medications on diet and does not find any evidence that access to improved medications led to worse health habits. Peltzman (2002), on the other hand, investigates the impact of the development of antibiotics on mortality and finds that the development increased the mortality risk among age groups and in regions that were most likely to benefit from the innovation. He interprets this as suggestive evidence of offsetting behavior where individuals adopt risky health behaviors in response to the availability of a curative product. These and other studies (summarized in the next section) exclusively examined the unintended effects of “positive” information on health behavior. Different from this literature, we examine the effects of “negative” information, i.e. the harmful effects of HRT. This is an important distinction as results may differ from the predictions based on studies involving positive information if there are asymmetric responses to positive and negative information. While existence of asymmetric responses has received attention in other areas (e.g. over the business cycle), this question has not been explored in this context before. Our work is also related to theories on cognitive dissonance, hyperbolic discounting, fatalism and the health belief model. We discuss these theories in Section 5.4.
While data limitations constrain our ability to pinpoint the exact mechanisms leading to the observed spillover effects, we advance the literature in several ways. As described in detail in Section 2.1, previous studies on this topic exclusively focused on the effects of marketing and availability of curative products on health behavior. In contrast, we study responses to the findings of a highly publicized randomized trial and the subsequent FDA warnings. Second, we examine how information on the safety of a preventive drug impacts health behavior, which may differ from the impact of curative products on health behavior. Third, the marketing and availability of curative products involve the provision of positive information (i.e. that there is a cure available). As noted above, our study investigates consequences of providing “negative” information. Finally, to our knowledge, our study is the first to use a quasi-experiment when investigating responses to new medical information and thus improves upon previous studies that may suffer from various endogeneity biases.
Our results have important insights for drawing a more complete picture of the value of new medical information. In particular, these findings suggest that any cost–benefit analysis on the value of medical information should take these behavioral spillovers into account.
The remainder of the paper is organized as follows. Section 2 presents a background on the related literature, HRT and WHIS. Section 3 introduces the data, while Section 4 outlines the empirical framework. The results are presented in Section 5 along with robustness checks and a discussion on potential mechanisms. Finally, Section 6 concludes.
2.1 Previous literature
Our work is related to the extensive body of research showing that consumers are generally responsive to medical information. Drug sales are found to be related to marketing information, FDA warnings and scientific evidence (Berndt et al. 1995; Azoulay 2002; Parkinson et al. 2008) and exposure to advertising is shown to increase the probability of quitting smoking (Avery et al. 2007) as well as compliance with the advertised drug therapy (Calfee, Winston, and Stempski 2002). Similarly, previous literature documents that the quality information provided by plan ratings and report cards has a significant impact on individuals’ choice of health plans and health care providers (Dranove et al. 2003; Chernew, Gowrisankaran, and Scanlon 2004; Cutler, Huckman, and Landrum 2004; Dafny and Dranove 2008).
Past research also shows that the consumption effects of medical information pertaining to specific drugs are not restricted to those drugs only, but that there are spillovers to others. Evidence suggests that DTCA on prescription drugs increases the sales of the same-brand over-the-counter medications (Ling, Berndt, and Kyle 2002) and compliance with drug therapy for all medications (Wosinska 2005), and that the FDA prescription drug withdrawals lead to reduced utilization of the non-withdrawn drugs within the same therapeutic class (Cawley and Rizzo 2008).
Despite its potentially important public policy implications, the related question of how new medical information on drugs impacts a healthy lifestyle has received very little attention. Iizuka and Jin (2005) and Iizuka and Jin (forthcoming) use instrumental variable methods and find that DTCA exposure is associated with increased doctor visits but reduced likelihood of engaging in moderate exercise.  Conducting a series of experiments, Bolton, Cohen, and Bloom (2006) and Bolton et al. (2008) find that marketing of curative drugs related to various risky health behaviors, such as smoking and having a high-fat diet, increases intentions to engage in risky behavior. Finally, as discussed in the introduction, Kahn (1999) and Peltzman (2002) examine the impact of medical breakthroughs on subsequent preventive health behavior. We add to this small but growing literature by examining the impact of new medical information on drug safety on preventive health care use.
2.2 Hormone replacement therapy
Hormone replacement therapy is a treatment that involves taking small doses of female hormones that the natural aging process takes away. Female hormones (estrogen and progesterone) are produced naturally by the ovaries. As women approach menopause, i.e. their final menstrual period, their ovaries shrink and the levels of female hormones start to fluctuate.  During the years immediately before menopause (peri-menopause) and the years that follow (post-menopause), the production of female hormones gradually slows down and eventually stops. Peri-menopausal and menopausal women can experience negative symptoms, such as hot flashes, night sweats, sleep disturbances and vaginal atrophy, as their bodies try to adapt to reduced hormone levels. Although most of these symptoms disappear during post-menopausal years, post-menopausal women face increased risk for serious health problems, such as heart disease, stroke, and bone loss that can result in osteoporosis and fractures. 
During the early twentieth century, HRT was mainly intended as a short-term therapy to relieve the negative symptoms of menopause (Watkins 2007). The idea that HRT could be used as a long-term therapy to improve the health of post-menopausal women emerged during the mid-twentieth century as scientists put forward the hypothesis linking reduced female hormones to the onset of osteoporosis, cardiovascular disease and dementia, and drew analogies between estrogen deficiency after menopause and insulin deficiency in diabetes (Watkins 2007). Watkins (2007) notes that the April 1984 osteoporosis treatment consensus group of the National Institutes of Health recommended estrogen therapy without specific limitations to its duration of use. Soon after that, the Food and Drug Administration added in April 1986 the treatment of post-menopausal osteoporosis to the indications of HRT  and recommended in a special two-day workshop that any woman over the age of 50 should be seen as a candidate for long-term HRT (Watkins 2007). The American College of Physicians agreed with these recommendations stating in their 1992 guidelines for counseling post-menopausal women that “all women, regardless of race, should consider preventive hormone therapy” (American College of Physicians 1992, p. 1038).
The available statistics from these periods reflect the increased popularity of HRT as a long-term preventive medication. The number of oral estrogen prescriptions increased from 13.6 millions in 1982 to 20.1 millions in 1987 to 36.5 millions in 1992 (Wysowski, Golden, and Burke 1995). Furthermore, the share of oral estrogens prescribed to post-menopausal women aged 60 and above increased from 19% in 1979 to 25% in 1986 to 30% in 1992 (Hemminki et al. 1988; Wysowski et al. 1995).  Premarin, the conjugated estrogen by Wyeth-Ayerst Laboratories, became the most frequently prescribed drug in 1992 and remained in the top two most prescribed drugs in the United States for every year during the 1990s (Watkins 2007).
Given the widespread HRT use and the aging of the American population, the 1990s began with an even more pronounced need for a reliable study examining the effects of long-term HRT on the health of post-menopausal women. It is then not surprising that out of this need the National Institutes of Health established in September 1990 the Office of Research on Women’s Health, which would soon found the Women’s Health Initiative Study.
2.3 Women’s Health Initiative Study
The Women’s Health Initiative Study was launched in April 1991 with the aim of evaluating the effectiveness of several strategies for preventing major diseases and promoting good health among post-menopausal women.  The study, sponsored by the National Heart, Lung, and Blood Institute, consisted of three components: a clinical trial, an observational study and a community prevention study. The clinical trial further had a Hormone Therapy component aimed to examine the preventive effects of HRT on heart disease and osteoporosis, keeping in mind the potential risks for breast cancer.
Between October 1993 and October 1998, the study recruited more than 160,000 participants, of which more than 68,000 were assigned to the clinical trials and nearly 28,000 women were assigned to the HRT trials. Although the project was initially expected to cost $625 million, the final cost was much higher. To this day, the WHIS continues to be the largest randomized controlled trial of women’s health ever undertaken.
During the recruiting years of the WHIS, HRT kept gaining its popularity as a preventive medicine. The number of HRT prescriptions kept rising throughout the 1990s, reaching 90 million in 1999 (Hersh, Stefanick, and Stafford 2004). More importantly, HRT gained acceptance for indications for which it was not FDA-approved, most notably for prevention of heart disease. In a national survey conducted by the National Heart, Lung, and Blood Institute, Rossouw (1996) found that 66% of the surveyed cardiologists, internists, family doctors and general practitioners prescribed HRT for prevention of coronary heart disease. Similarly, surveys of providers within several health maintenance organizations from this period show that over 95% of providers believed that the greatest benefit of HRT was in the prevention of cardiovascular conditions (Rolnick et al. 1999) and that physicians who were convinced of the benefits of HRT in preventing heart disease were more likely to prescribe it (Newton et al. 2001).
The HRT component of the WHIS was scheduled to run until 2005. However, the clinical trial on combined estrogen–progestin hormones was abruptly terminated on July 9, 2002, in the interest of safety, after an average follow up of 5.2 years.  The results of the trial indicated that the combined estrogen-progestin use would lead to 7 more heart attacks, 8 more strokes, 18 more blood clots and 8 more breast cancers annually per 10,000 post-menopausal women. They also indicated that the combined HRT use would result in 5 less hip fractures and 6 less colon cancers per year per 10,000 post-menopausal women (Writing Group for the Women 2002). Although the scientific results were officially published on July 17, 2002, the study was released to the public early on July 9, 2002 because of the importance of the findings.
The WHIS results made front pages in all major newspapers and were covered in broadcasts on national television.  In the wake of the WHIS news, the stock prices of the producer of HRT medications used in the trial (Wyeth-Ayerst Pharmaceuticals) fell by more than 24%.  The news was followed by statements from various policy makers and research groups. Most of these statements focused on the results pertaining to the effects of long-term HRT use on coronary heart disease. The FDA released a statement in August 2002 stating that “[t]he increased risks of breast cancer and thromboembolic disease associated with estrogen and combination estrogen/progestin had previously been known or suspected. The increased risk of cardiovascular disease, including heart attack and stroke, in healthy women, is new information” (Food and Drug Administration 2002, p. 1).  The National Heart, Lung, and Blood Institute announced September as the “menopause awareness month” and published a booklet describing the WHIS findings (National Heart, Lung and Blood Institute 2005). Similarly, the North American Menopause Society concluded in its October 2002 advisory panel that estrogen-progestin should not be used for the prevention of heart disease (Watkins 2007).
The final verdict on the preventive role of long-term HRT came in January 2003 as FDA announced a formal name change for HRT drugs as “menopausal hormone therapy” and mandated a boxed warning on the label of HRT drugs about the increased risk of heart attacks, stroke and blood clots.  Since then, FDA recommends HRT use for relief of “moderate to severe vasomotor symptoms associated with menopause, treatment of vulvar and vaginal atrophy and prevention of postmenopausal osteoporosis.” FDA has also been emphasizing that HRT products should be used at the lowest dose and for the shortest necessary duration.
In summary, the WHIS was conducted at a time when HRT was widely used and was perceived to be a miracle drug, effective as a short-term therapy in the treatment of menopausal symptoms as well as a long-term therapy for the prevention of serious diseases related to aging, most notably for the prevention of cardiovascular disease. The abrupt termination of the combined HRT trial, the highly publicized nature of the findings, the strong response of policy makers to the news, and the size of the potentially affected group makes this an ideal quasi-experiment when examining how medical information on drugs affects preventive health behavior.
To assess the effects of the WHIS findings on preventive health behavior of post-menopausal women, we use data from the Behavioral Risk Factor Surveillance System (BRFSS) for the period 1998–2007. The BRFSS is a nationally representative telephone survey designed to provide information on risky behavior and preventive health practices of the civilian, noninstitutionalized adult population. The survey is administered by individual states and the data are edited and processed by the Centers for Disease Control and Prevention (CDC).
The BRFSS survey is made up of three parts. The core component includes a set of questions that are asked by all states. However, these questions can be asked in every year (the fixed-core questions) or in alternating years (the rotating-core questions). Every year, a set of questions pertaining to the latest health issues are also added to the core component (the emerging-core questions). These questions are evaluated in the following year to determine whether they will be mainstreamed into the fixed or alternating cores in future surveys. In addition to the core questionnaire, states can also administer questionnaires on specific topics supported by the CDC (the optional modules component) or develop and add their own questions (stated added questions component).
The BRFSS is an excellent dataset for the purposes of our research because it is a very large annual survey including rich information on preventive health habits of adults as well as basic demographic data. Other individual surveys that cover similar information are either not as detailed as the BRFSS or suffer from much smaller sample sizes. For example, the Current Population Survey has detailed demographic data, as well as information on self-reported health status, but it has no information on health habits. The National Health Interview Survey includes data on a range of health and health behavior but it has much smaller sample sizes and the public-use data files do not contain state identifiers. Controlling for state and state-year effects is particularly important in this context because many states implemented welfare reforms during the 1990s and this could have potentially impacted the health behavior of individuals (Blank 2002; Bitler, Gelbach, and Hoynes 2005).
In the empirical analysis, we focus on a wide set of outcome variables concerning preventive health care use. In particular, our outcomes include indicators for having a checkup, mammography, professional breast exam, cholesterol check, blood stool or flu shot in the past 12 months.  In our regressions, we also control for a set of observable characteristics of individuals. These include dummy variables for age, race (non-Hispanic white, non-Hispanic black, non-Hispanic other, Hispanic), marital status (married, single, never married), education (less than high school, high school or GED, some college, college and more), annual household income (less than $20,000, [$20,000–$25,000), [$25,000–$35,000), [$35,000–$50,000), $50,000 and more) and current insurance status. 
Ideally we would like to include in our analysis sample post-menopausal women. Unfortunately, BFRSS does not contain information on menopausal status. For that reason, we restrict the sample to women in the post-menopausal ages instead. The medical literature indicates that 75–80% of women reach menopause by the age of 55 (Mckinlay 1996; Mckinlay, Brambilla, and Posner 2008). Since menopausal symptoms can last for a few years after its onset, we include in our sample women aged 60 and above. The size of the analysis sample changes depending on the outcome variable examined. All of our outcomes are included in the core component of the BRFSS and thus are administered by all states but none of them are part of the fixed-core questionnaire during the whole sample period.  This leaves us with a sample of 179,580–367,836 observations, depending on the outcome studied.
4 Empirical strategy
We are interested in examining the impact of drug safety information on preventive health behavior. The WHIS provides an information shock on the safety of long-term HRT use by post-menopausal women. As seen in Figure 1, before the announcement of the WHIS findings, women aged 60–69 had much higher rates of HRT use compared to women aged 75 or above.  Therefore, we expect the WHIS findings to have a larger impact on the preventive health behavior of women aged 60 to 69 relative to those aged 75 and above. Following this intuition, we set up a difference-in-differences model where we compare changes in the preventive health behavior of women aged 60–69 with changes in preventive health behavior of women aged 75 and above.  The key equation of interest can be described as:
We enrich this basic specification in different ways. We add a set of control variables describing the observable characteristics of each individual ( ). We replace the post-period dummy variable with a full set of survey month and year indicators to allow for a more flexible time series pattern when controlling for overall changes in the outcome. Finally, we include a set of state dummy variables to correct for time-invariant or slowly changing state specific factors that might impact the health or health behavior of residents.
Our key identifying assumption is that older post-menopausal women provide the counterfactual of how younger post-menopausal women would have behaved in the absence of the release of the WHIS findings. Although this is basically an untestable assumption, we can test its plausibility by checking the evolution of the outcomes in the pre-treatment period among the two groups. In particular, we restrict our sample to 1998–2002 and estimate the following equation:
Before presenting the results, two issues are worth discussing. The first issue is why we do not employ a “traditional” differences-in-difference strategy where post-menopausal women constitute the treatment group and a potentially untreated group is used to construct the counterfactual of how post-menopausal women would behave in the absence of the WHIS findings. In such a setting, there are two potential natural comparison groups. The first candidate is the group of middle-aged women who may use HRT on a short-term basis to alleviate negative menopausal symptoms. While there were no changes in the indications of short-term HRT use for the alleviation of menopausal symptoms, Daysal and Orsini (2014) show that there were substantial spillover effects to the HRT use of these women which led to economically significant reductions in their short-term employment due to the resurgence of menopausal symptoms. Given that employment is an important source of insurance coverage, it is then possible that these women changed their preventive behavior, making them unsuitable as a comparison group. The second natural comparison group is men because hormone replacement is a therapy exclusive to women. The major weakness of this comparison group is the known differences in preventive habits among men and women, especially during older ages. Furthermore, one cannot examine changes in preventive behavior pertaining to women’s health when relying on men as a comparison group.
A second point worth noting is the reason why we do not examine the impact of HRT use on preventive health behavior and instrument for HRT use with the WHIS findings. The key identification assumption required in this case is the instrument excludability: that the WHIS findings impact health behavior only through changes in HRT use. However, it is possible that the WHIS findings impacted preventive health behavior by simply reminding women of various health risks associated with post-menopause, thereby violating the exclusion restriction. With these explanations in mind, we turn to our results.
5.1 Descriptive statistics
Tables 1 and 2 provide the means and standard deviations of our outcome and control variables from BRFSS. In Table 1, we present summary statistics for the full sample and separately for more and less intensively treated groups, as well as for sub-periods determined according to the release of the WHIS findings. These figures provide some preliminary, descriptive evidence on the effects of the drug safety information provided by the WHIS findings on preventive health behavior. The table suggests that women aged 60–69 did not adopt more preventive checks during the post-period as compared to women aged 75 or above. For example, among women aged 60–69, the fraction who reported having a mammogram within the last year declined by 1.2% between the pre- and post-announcement periods of the WHIS, whereas women aged 75 and above experienced an increase of almost 4%. Similarly, annual checkups among the more intensively treated group declined by almost 4% from 84.2% in pre-period to 81% in the post-period, whereas the corresponding change for women in the comparison group was around 1%. In the next section, we provide our estimation results and check if the observed differences documented here persist after controlling for other potentially confounding factors.
|Full sample||Women: 60–69||Women: 75+|
|All Years: 1998–2007|
|Professional breast exam||0.622||0.485||0.678||0.467||0.549||0.498|
|Professional breast exam||0.629||0.483||0.681||0.466||0.556||0.497|
|Professional breast exam||0.610||0.488||0.673||0.469||0.537||0.499|
|Full sample||Women: 60–69||Women: 75+|
|Less than high school||0.185||0.388||0.153||0.360||0.224||0.417|
|College and more||0.194||0.396||0.220||0.414||0.163||0.369|
|Less than $20,000||0.258||0.437||0.215||0.411||0.311||0.463|
|$50,000 and More||0.142||0.349||0.201||0.401||0.068||0.252|
5.2 Baseline results
Table 3 presents the main results. Each row provides results on a separate outcome variable. The first column provides means of the outcome variables among women aged 60–69 during the pre-WHIS period. Columns (2)–(3) provide the effects of the drug safety information provided by the WHIS on preventive health behavior. In order to gauge the importance of time-varying characteristics, we first estimate in column (2) a simple difference-in-differences model described by eq. , and then in column (3) models from our full specification. Regressions are weighted by the final weights provided in the BRFSS. Robust standard errors clustered at the state level are shown in parenthesis below coefficients.
|Pre-period mean of the outcome among women||Simple DID||DID|
|(N = 263,543)||(0.004)||(0.004)|
|(N = 218,218)||(0.010)||(0.009)|
|(N = 197,717)||(0.009)||(0.007)|
|Professional breast exam||0.681||0.011||0.002|
|(N = 194,726)||(0.009)||(0.008)|
|(N = 179,580)||(0.006)||(0.005)|
|(N = 367,836)||(0.007)||(0.007)|
The results consistently show that women aged 60–69 reduced their preventive health care use relative to women aged 75 and above during the post-WHIS period. The estimates are statistically significant for all outcomes except annual flu shots and professional breast exams. For example, the estimate on mammography checks in column (3) suggests that women aged 60–69 were 3.3 percentage points less likely to have an annual mammogram during the post-WHIS period as compared to women aged 75+, a 4.8% reduction when compared to the mean of the outcome among women aged 60–69 during the pre-treatment period. Similarly, the results on annual checkup, cholesterol check and blood stool test indicate that individuals in the more intensively treated group were respectively 1.8, 2.7 and 2.1 percentage points less likely to have these preventive checks during the post-WHIS period as compared to the individuals in the less intensively treated group. There results correspond to 2.14%, 3.54% and 8.37% decline at the pre-period mean of the outcome among women aged 60–69.
As we described before, the key identifying assumption in difference-in-differences models is that women aged 75 and above provide the counterfactual of what would have happened to women aged 60–69 in the absence of the WHIS intervention. Although, this is basically an untestable assumption, we can shed some light on it by checking the evolution of the outcomes in the pre-treatment period among the two groups. In order to do so, we run eq.  described in Section 4. The p-values corresponding to the null hypothesis testing the significance of the interaction variables (i.e. equality of the pre-treatment trends in outcomes among women aged 60–69 and 75+) are provided in column (3) in square brackets. It is reassuring that for all of our selected outcomes, we fail to reject the equality of the pre-treatment trends in outcomes among women 60–69 and women aged 75+ with substantially high p-values.
5.3 Robustness checks
In this section, we provide additional analyses checking the robustness of the baseline estimates. Table 4 presents the results. Each column represents results using a different outcome variable. Panel A reproduces our baseline estimates from the specification including all controls for reference (compare to column (3) in Table 3). Panels B and C check the robustness of the results to functional form and model specification. In Panel B, we examine the appropriateness of using a linear model by estimating logit regressions. The average marginal effects confirm the previous baseline estimates produced by the linear specification. For the remainder of the checks, we therefore report results from linear models for the sake of brevity.
|Checkup||Cholesterol check||Mammography||Breast exam||Blood stool||Flu shot|
|C: State-specific time trend|
|D: Control: Women aged 70+|
|E: Control: Women aged 80+|
|F: Control: Men|
|G: Cluster: State-Year|
|H: Heterogeneous effects by education|
|High school and less|
|Mean of outcome||[83.7]||[74.5]||[64.3]||[63.6]||[22.1]||[50.5]|
|Some college and more|
|Mean of outcome||[84.9]||[78.6]||[74.3]||[74.2]||[28.8]||[55.6]|
|I: Placebo regressions|
|Post-period is 1999 and 2000|
|Post-period is 2000|
|Post-period is 2001|
Panel C attempts to account for a potential bias arising from changes in state-based preventive health policies. The 1990s and 2000s witnessed an increased awareness in the importance of access to preventive health services. The U.S. Department of Health and Human Services released on September 6, 1990 a report detailing national public health goals and objectives for the decade. The report (Healthy People 2000: National Health Promotion and Disease Prevention Objectives) was followed by another initiative (Healthy People 2010: Objectives for Improving Health), released on January 25, 2000, updating the goals for the year 2010. Both of these reports placed great emphasis on the role of preventive services. For example, of the 22 focus areas in the earlier report, 21 were related to health promotion, health protection and preventive services. The development and implementation of these goals represented a concerted effort at the national, state and community level. To the extent that the implementation of the programs associated with these initiatives varied across states, this could contaminate our estimates. To examine this issue, we add state-specific year effects to our baseline estimates. This should capture any effect that is common to all individuals within a unique state-year cell. The results provided in the third row are virtually the same as those in the baseline model.
The following three Panels investigate the sensitivity of our findings to the chosen comparison group.  Panel D expands the comparison group to women aged 70 and above while Panel E restricts it to women aged at least 80 years old. In both cases, we confirm our main finding: women aged 60–69 group were significantly less likely to use preventive care after the announcement of the WHIS results, as compared to the group of older women. Furthermore, the magnitudes of the estimates are consistent with our prior that these behavioral changes are driven by changes in HRT use – as the intensity of treatment in the comparison group (i.e. the share of women with HRT use) declines, the size of our estimates increases. In Panel F, we focus on individuals aged 60–69 and use men as the comparison group. We again find that women aged 60–69 were less likely to use preventive care after the WHIS relative to men of similar ages. The robustness of our results to different comparison groups lends support to the claim that our results are not driven by other policy interventions that happen around the time of the release of the WHIS findings and differentially impact the treatment and comparison groups.
In Panel G, we examine whether the level at which we cluster the standard errors changes our results qualitatively. Since the BRFSS sample individuals at the state-year level (and this may cause correlations among the error terms at this level), we re-estimate the baseline model and cluster the standard errors at the state-year level.  As the table shows, clustering at a finer level does not impact our initial conclusions.
In Panel H, we examine the heterogeneity in the response to the WHIS results by educational attainment. Economic theory suggests that education may have an important impact on preventive health care use. For example, Grossman (1972) and Schultz (1975) argue that education may improve an individual’s health production function and increase allocative efficiency through enhanced learning and ability to process information. Figure 2 shows the percentage of women in MEPS who reported using HRT separately by treatment intensity and by educational attainment.  The graph shows that HRT use is much higher among those 60 to 69 than those 75 and older, regardless of the level of education. In addition, consistent with economic theory, it shows that HRT use is higher among women with at least some college education (relative to those with at most a high school degree), regardless of treatment intensity. However, all four groups show comparable relative reductions in HRT use after the release of the WHIS results, perhaps because HRT use is mainly decided by physicians. For example, among the lower-educated women in the more (less) intensively treated group, HRT use declined from 29 (11) percentage points during the pre-period (1998–2002) to 12 (5) percentage points during the post-period (2003–207). Similarly, among the higher-educated women in the more (less) intensively treated group, HRT use declined from 35 (16) percentage points during the pre-period to 16 (7) percentage points during the post-period. Our results in Panel H of Table 4 show that, following the announcement of the WHIS findings, more intensively treated women at both levels of educational attainment had lower preventive care use when compared to women in the less intensively treated group. While the estimates among the higher educated are in general larger in absolute value, they are similar in relative terms and we cannot reject the equality of the estimates across different education groups. 
In Panel I, we conduct placebo tests in the spirit of Bertrand, Dufflo, and Mullainathan (2004). In particular, we restrict the data to pre-WHIS period and assign “fake” intervention years. We then estimate our baseline model using these data and the fake intervention indicator. Since different outcome variables are included in BRFSS in different years, the span of the pre-period (and hence the fake intervention year) changes by outcome. The results show that these fake intervention years do not have any differential impact on preventive health care use of women in the more intensively treated group relative to those in the less intensively treated group.
Finally, given the large number of outcomes we examine, we combine our various measures of preventive health care use into a single index in order to reduce the dimensionality of our analysis. This approach has the added advantage of avoiding problems of multiple testing, i.e. the fact that with many outcomes, the main independent variable will be statistically significant in at least some cases. Following Kling, Liebman, and Katz (2007), we create our index measure as the average of standardized individual measures. Using this index measure as the outcome in our baseline specification yields a highly significant coefficient of –0.175 (s.e. 0.007, N = 410,605), confirming that our baseline findings are not driven by multiple testing issues.
5.4 Potential mechanisms
Our results are consistent with four theories. According to the expected utility theory, a rational risk averse individual would be willing to sacrifice current consumption in order to have improved health in the future. Individuals may rationally deviate from the optimal choices if they display cognitive dissonance (Akerlof and Dickens 1982). Cognitive dissonance, in the most basic sense, means that individuals are uncomfortable holding contradicting beliefs. According to psychologists, most situations that lead to cognitive dissonance challenge individuals’ perception of themselves as “smart and nice” people. Whenever the individual is faced with a situation where there is a discrepancy between beliefs and behaviors, (s)he must either change the behavior or the belief in order to reduce the psychological discomfort. The abrupt termination of the WHIS and its findings on the preventive power of long-term HRT products may have created conflicting feelings among some individuals. In response to this news, they may have changed their perception regarding the trustworthiness of the medical sciences and thus may have decided to ignore all general recommendations on the use of preventive health care.
Our results are also consistent with behavioral models where time preference plays an important role. The theory of hyperbolic discounting suggests that present-biased preferences could explain the lack of motivation for healthy lifestyle habits that have immediate costs but delayed and uncertain benefits (Laibson 1997; Loewenstein, Brennan, and Volpp 2007). WHIS findings may have induced some individuals to engage in present-biased behavior by reducing their expected lifespan and thus changing their time preferences.
Fatalism is another possible explanation for our findings. Fatalism consists in the belief that no matter what a person does, his/her actions can do too little to impact future outcomes (Wu, 2005). A picturesque example is one of the athlete who “throws in the towel” when (s)he thinks that the possibility of winning a competition is low. If post-menopausal women believed that HRT drugs consisted of an important source of preventive care that enabled them to further adopt healthy life choices, it is entirely possible that the WHIS findings led some of these women to conclude that healthy lifestyles alone would not be adequate to protect them and thus “throw in the towel” and give up preventive care. Fatalistic tendencies are not at all uncommon in determining preventive health behavior. For example, Kremer (1996) shows that an increase in the probability of contracting HIV might lead individuals with a high number of sexual partners to increase sexual activity because they become fatalistic about the probability of contracting the disease.
Finally, our results are consistent with the predictions of the health belief model – one of the most accepted theories in the field of health education.  The original formulation of this theory is much like a cost–benefit calculation. An individual is predicted to adopt preventive behavior the higher the probability of getting a disease that is sufficiently serious in severity (i.e. “perceived threats”), the higher the effectiveness of the said action in reducing the threat (i.e. “perceived benefits”) and the lower the costs of taking that action (i.e. “perceived barriers”). The later formulations of the theory include two additional concepts: “cues to action” that describes situations, such as provision of educational materials and counseling, that can increase an individual’s readiness to act and “self-efficacy” that refers to one’s confidence in his/her ability to take actions to produce the required outcomes. Self-efficacy is, thus, related to our argument on fatalism. Some post-menopausal women could perceive medical products and preventive behavior as complementary goods so the loss of perceived protection by HRT could lead to a reduction in women’s confidence to prevent serious diseases associated with aging by solely relying on a healthy life style and using preventive care.
In this paper, we examine the response to the medical findings from the WHIS and the subsequent public policy actions that documented the harmful health effects of long-term HRT. We first show that following the release of the WHIS findings, HRT use and prescriptions declined substantially among women aged 60 and above. We then implement difference-in-differences models to examine the impact of these new medical information on preventive health care use. In particular, we compare the change in outcomes between the pre- and post-announcement periods of the WHIS results among 60 to 69 year-old women (who have higher rates of HRT use) with the change among women aged 75 and above (who have much lower rates of HRT use). Our results point to statistically and economically significant reductions in post-menopausal women’s likelihood of having an annual checkup, cholesterol check, mammography and blood stool test.
As the costs of producing and disseminating new medical information rise, understanding how consumers may respond to this information becomes even more important. Taken together, our results suggest that policies aimed at raising awareness on the safety of medications may have unintended spillover effects on preventive health behavior that reduce the net value of the new medical information.
Since the BRFSS does not contain information on the use of prescription medication, we employ additional data from the Medical Expenditure Panel Survey (MEPS) and the National Ambulatory Medical Care Survey (NAMCS) to document changes in HRT use. The household component of MEPS includes data from a nationally representative sample of the U.S. civilian non-institutionalized population. Respondents are interviewed about their medical expenditures and utilization of medical services over a period of 2 years through five interview rounds. Part of the household component (the prescribed medicine file) provides information on the prescription medicines purchased by the respondent in each round along with the three-digit ICD-9 codes denoting the medical conditions for which the prescription medicine was purchased. We supplement the MEPS household component data with additional information from the Physician’s Desk Reference Companion Guide (PDRCG), an annual publication on prescription drugs available on the market. The Physician’s Desk Reference Companion is regarded as one of the most respected and used handbooks by physicians to select medications for their patients (Watkins 2007). We use the “Therapeutic Indications Index,” a list of medications and the conditions for which they are indicated, to identify the HRT drugs that were available during our analysis period. Trade-name drugs that contain hormones to treat menopause and their generic substitutes are listed under the label “Menopause, Vasomotor Symptoms of.” We classify an individual as using HRT if the person reports purchasing at least one HRT product at any time during the round.
NAMCS is a national survey on the use of ambulatory medical care services in the United States. The unit of observation in the data is a physician–patient visit based on a random sample of visits to non-federally employed physicians.  It provides information on the patient’s demographic characteristics and symptoms, the physician’s diagnoses, the prescribed medications associated with the visit as well as the therapeutic class of the medications. The therapeutic class of drugs is based on the National Drug Code Directory for the period 1998–2005 and its successor the Lexicon Plus, a proprietary database of Cerner Multum, Inc. for the period 2006–2007. The public data files provide information on up to six medications for 1998–2003 and eight medications for 2004–2007. They also provide one therapeutic class for each medication for 1998–2001, three therapeutic classes for 2001–2005 and four therapeutic classes for 2006–2007. In cases when there are multiple therapeutic class codes, we use the primary code associated with the medication. We classify a visit as resulting with an HRT prescription if any of the listed medications has a (primary) National Drug Code of “1034 Estrogens/Progestins” or Multum code of “183 Estrogens” or “185 Progestins”.
We thank Philipp Wichardt, an anonymous referee, Bill Evans, Armin Falk, Ben Hansen, Tamara Konetzka, Peter Kooreman, Ellen Meara, Willard Manning, Sonia Oreffice, Mircea Trandafir, Eddy van Doorslaer and workshop participants at Erasmus University, Tilburg University, University of Chicago, 4th Annual Meeting on the Economics of Risky Behaviors and 4th Biennial Conference of the American Society of Health Economists for helpful comments and discussions. The authors bear sole responsibility for the content of this paper.
Akerlof, G. A., and W. T.Dickens. 1982. “The Economic Consequences of Cognitive Dissonance.” American Economic Review72:307–19. Search in Google Scholar
American College of Physicians. 1992. “Guidelines for Counseling Postmenopausal Women About Preventive Hormone Therapy.” Annals of Internal Medicine117:1038–41. Search in Google Scholar
Avery, R., D.Kenkel, D. R.Lillard, and A.Mathios. 2007. “Private Profits and Public Health: Does Advertising of Smoking Cessation Products Encourage Smokers to Quit?” The Journal of Political Economy115:447–81. Search in Google Scholar
Azoulay, P. 2002. “Do Pharmaceutical Sales Respond to Scientific Evidence?” Journal of Economics and Management Strategy11:551–94. Search in Google Scholar
Berndt, E. R., L.Bui, D. R.Reiley, and G. L.Urban. 1995. “Information, Marketing, and Pricing in the U.S. Antiulcer Drug Market.” The American Economic Review85:100–5. Search in Google Scholar
Bertrand, M., E.Dufflo, and S.Mullainathan. 2004. “How Much Should We Trust Difference in Differences Estimates?” Quarterly Journal of Economics119:249–75. Search in Google Scholar
Bitler, M. P., J. B.Gelbach, and H. W.Hoynes. 2005. “Welfare Reform and Health.” The Journal of Human Resources40:309–34. Search in Google Scholar
Blank, M. R. 2002. “Evaluating Welfare Reform in the United States.” Journal of Economic Literature40:1105–66. Search in Google Scholar
Bolton, L. E., I. I.Americus Reed, K. G.Volpp, and K.Armstrong. 2008. “How Does Drug and Supplement Marketing Affect a Healthy Lifestyle?” Journal of Consumer Research34:713–26. Search in Google Scholar
Bolton, L. E., J. B.Cohen, and P. N.Bloom. 2006. “Does Marketing Products as Remedies Create ‘Get Out Of Jail Free Cards’?” Journal of Consumer Research33:71–81. Search in Google Scholar
Calfee, J. E., C.Winston, and R.Stempski. 2002. “Direct-to-Consumer Advertising and the Demand for Cholesterol-Reducing Drugs.” Journal of Law and Economics45:673–90. Search in Google Scholar
Cawley, J., and J. A.Rizzo. 2008. “Spillover Effects of Prescription Drug Withdrawals.” Advances in Health Economics and Health Services Research19:119–43. Search in Google Scholar
Chernew, M., G.Gowrisankaran, and D. P.Scanlon. 2004. “Learning and the Value of Information: Evidence from Health Plan Report Cards.” Journal of Econometrics144:156–74. Search in Google Scholar
Cutler, D. M., R. S.Huckman, and M. B.Landrum. 2004. “The Role of Information in Medical Markets: An Analysis of Publicly Reported Outcomes in Cardiac Surgery.” American Economic Review94:342–6. Search in Google Scholar
Dafny, L., and D.Dranove. 2008. “Do Report Cards Tell Consumers Anything They Don’t Already Know? the Case of Medicare HMOs.” RAND Journal of Economics39:790–821. Search in Google Scholar
Daysal, N. M., and C.Orsini.2014. “The miracle drugs: Hormone replacement therapy and labor market behavior of middle aged women.” IZA Working Paper No. 6409. Search in Google Scholar
Donohue, J. M., M.Cevasco, and M. B.Rosenthal. 2007. “A Decade of Direct-to-Consumer Advertising of Prescription Drugs.” The New England Journal of Medicine357:673–81. Search in Google Scholar
Dranove, D., D.Kessler, M.McClellan, and M.Satterthwaite. 2003. “Is More Information Better? The Effects of Report Cards on Health Care Providers.” The Journal of Political Economy111:555–88. Search in Google Scholar
Food and Drug Administration. 1986. “Oral Estrogens for Postmenopausal Osteoporosis; Drug Efficacy Study Implementation; Reevaluation.” Federal Register51: 12568–70. Search in Google Scholar
Food and Drug Administration. 2002. “Women’s Health Initiative (WHI) Results Signal Need for Reassessment of Risks and Benefits of Conjugated Equine Estrogens/Medroxyprogesterone Acetate (prempro) in Postmenopausal Women.” FDA Statement, 13 August 2002. Search in Google Scholar
Food and Drug Administration. 2003. “FDA Approves Prescribing Information for Postmenopausal Hormone Therapies.” FDA Letter to Wyeth Pharmaceuticals, 6 January 2003. Search in Google Scholar
Glanz, K., B. K.Rimer, and K.Viswanath. 2008. Health Behavior and Health Education: Theory, Research and Practice. San Francisco, CA: Jossey-Bass. Search in Google Scholar
Grossman, M. 1972. “On the Concept of Health Capital and the Demand for Health.” Journal of Political Economy80:225–55. Search in Google Scholar
Guthrie, P. 2002. “Study Halts Hormone Therapy.” Pittsburgh Post-Gazette, 9 July 2002. Search in Google Scholar
Hemminki, E., D. L.Kennedy, C.Baum, and S. M.Mckinlay. 1988. “Prescribing of Noncontraceptive Estrogens and Progestins in the United States, 1974–86.” American Journal of Public Health78:1479–81. Search in Google Scholar
Hersh, A. L., M. L.Stefanick, and R. S.Stafford. 2004. “National Use of Postmenopausal Hormone Therapy.” Journal of the Medical Association291:47–53. Search in Google Scholar
Hoynes, H. W., D. L.Miller, and D.Simon. forthcoming. “Income, the Earned Income Tax Credit, and Infant Health.” American Economic Journal: Economic Policy. Search in Google Scholar
Hulley, S., D.Grady, T.Bush, C.Furberg, D.Herrington, B.Riggs, and E.Vittinghoff. 1998. “Randomized Trial of Estrogen Plus Progestin for Secondary Prevention of Coronary Heart Disease in Postmenopausal Women.” The Journal of the American Medical Association280:605–13. Search in Google Scholar
Iizuka, T., and G. Z.Jin. 2005. “The Effect of Prescription Drug Advertising on Doctor Visits.” Journal of Economics & Management Strategy14:701–27. Search in Google Scholar
Iizuka, T., and G. Z.Jin. forthcoming. “Drug Advertising and Health Habit.” Advances in Applied Microeconomics, Volume Communication Games. Search in Google Scholar
Kahn, M. E. 1999. “Diabetic Risk Taking: The Role of Information, Education and Medication.” Journal of Risk and Uncertainty18:147–64. Search in Google Scholar
Kalota, G. 2002. “Study is Halted Over Rise Seen in Cancer Risk.” NY Times, 9 July 2002. Search in Google Scholar
Kling, J., J.Liebman, and L.Katz. 2007. “Experimental Analysis of Neighborhood Effects.” Econometrica75:83–119. Search in Google Scholar
Kremer, M. 1996. “Integrating Behavioral Choice into Epidemiological Models of Aids.” Quarterly Journal of Economics61:549–73. Search in Google Scholar
Laibson, D. 1997. “Golden Eggs and Hyperbolic Discounting.” The Quarterly Journal of Economics112:443–77. Search in Google Scholar
Ling, D. C., E. R.Berndt, and M. K.Kyle. 2002. “Deregulating Direct-to-Consumer Marketing of Prescription Drugs: Effects on Prescription and Over-the-Counter Product Sales.” Journal of Law and Economics45:691–723. Search in Google Scholar
Loewenstein, G., T.Brennan, and K. G.Volpp. 2007. “Asymmetric Paternalism to Improve Health Behaviors.” The Journal of the American Medical Association298:2415–17. Search in Google Scholar
Mckinlay, S. M. 1996. “The Normal Menopause Transition: An Overview.” Maturitas23:137–45. Search in Google Scholar
Mckinlay, S. M., D. J.Brambilla, and J. G.Posner. 2008. “The Normal Menopause Transition.” Maturitas61:4–16. Search in Google Scholar
National Heart, Lung and Blood Institute. 2005. “‘Facts about Menopause Hormone Therapy.” NIH Publication No. 05–5200. Search in Google Scholar
Newton, K. M., A. Z.LaCroix, D. S. M.Buist, L. A.Anderson, and K.Delaney. 2001. “What Factors Account for Hormone Replacement Therapy Prescribing Frequency?” Maturitas39:1–10. Search in Google Scholar
Parkinson, K., J.Price, K.Simon, and S.Tennyson.2008. “The Influence of FDA Advisory Information and Black Box Warnings on Consumer Use of Prescription Antidepressants.” SSRN Working Paper. Search in Google Scholar
Peltzman, S.2002. “Offsetting Behavior and Medical Breakthroughs.” Working Paper. Search in Google Scholar
Peterson, M. 2002. “Wyeth-Stock Falls 24% after Reported.” NY Times, 10 July 2002. Search in Google Scholar
Price, J., and K.Simon. 2009. “Patient Education and the Impact of New Medical Research.” Journal of Health Economics28:1166–74. Search in Google Scholar
Rolnick, S. J., R. A.Kopher, R. B.Compo, M. E.Kelley, and T. A.DeFor. 1999. “Provider Attitudes and Self-Reported Behaviors Related to Hormone Replacement Therapy.” Menopause6:257–63. Search in Google Scholar
Rossouw, J. E. 1996. “Estrogens for Prevention of Coronary Heart Disease: Putting the Brakes on the Bandwagon.” Circulation94:2982–5. Search in Google Scholar
Rubin, R. 2002. “U.S. Halts Study on Hormone Therapy.” USA Today, 9 July 2002. Search in Google Scholar
Schultz, T. W. 1975. “The Value of the Ability to Deal with Disequilibria.” Journal of Economic Literature13:827–46. Search in Google Scholar
Soumerai, S. B., J.Avorn, S.Gortmaker, and S.Hawley. 1987. “Effect of Government and Commercial Warnings on Reducing Prescription Misuse: The Case of Propoxyphene.” The American Journal of Public Health77:1518–23. Search in Google Scholar
The North American Menopause Society. 2006. Menopause guidebook. Mayfield Heights, OH: The North American Menopause Society. Search in Google Scholar
U.S. Department of Health and Human Services. 2002. “NHLBI Stops Trial of Estrogen Plus Progestin Due to Increased Breast Cancer Risk, Lack of Overall Benefit.” NIH News Release, 9 July 2002. Search in Google Scholar
Wagner, A. K., et al. 2006. “FDA Drug Prescribing Warnings: Is the Black Box Half Empty or Half Full?” Pharmacoepidemiology and Drug Safety15:369–86. Search in Google Scholar
Watkins, E. S. 2007. The Estrogen Elixir. Baltimore, MD: The Johns Hopkins University Press. Search in Google Scholar
Weatherby, L. B., B. L.Norstrom, D.Fife, and A. M.Walker. 2002. “The Impact of Wording in ‘Dear Doctor’ Letters and in Black Box Labels.” Clinical Pharmacology and Therapeutics72:735–42. Search in Google Scholar
Wosinska, M. 2005. “Direct-to-Consumer Advertising and Drug Therapy Compliance.” Journal of Marketing Research42:323–32. Search in Google Scholar
Writing Group for the Womens Health Initiative Investigators. 2002. “Risks and Benefits of Estrogen Plus Progestin in Healthy Postmenopausal Women.” Journal of the American Medical Association288:321–33. Search in Google Scholar
Wu, S. 2005. “Fatalistic Tendencies: An Explanation of Why People Don’t Save.” Contributions to Economic Analysis and Policy4:Article 11. Search in Google Scholar
Wysowski, D. K., L.Golden, and L.Burke. 1995. “Use of Menopausal Estrogens and Medroxyprogesterone in the United States, 1982–1992.” Obstetrics and Gynecology85:6–10. Search in Google Scholar
©2015 by De Gruyter