Promotional Tone in Reviews of Menopausal Hormone Therapy After the Women's Health Initiative: An Analysis of Published Articles

Summary

Background:
Even after the Women's Health Initiative (WHI) found that the risks of menopausal hormone therapy (hormone therapy) outweighed benefit for asymptomatic women, about half of gynecologists in the United States continued to believe that hormones benefited women's health. The pharmaceutical industry has supported publication of articles in medical journals for marketing purposes. It is unknown whether author relationships with industry affect promotional tone in articles on hormone therapy. The goal of this study was to determine whether promotional tone could be identified in narrative review articles regarding menopausal hormone therapy and whether articles identified as promotional were more likely to have been authored by those with conflicts of interest with manufacturers of menopausal hormone therapy.

Methods and Findings:
We analyzed tone in opinion pieces on hormone therapy published in the four years after the estrogen-progestin arm of the WHI was stopped. First, we identified the ten authors with four or more MEDLINE-indexed reviews, editorials, comments, or letters on hormone replacement therapy or menopausal hormone therapy published between July 2002 and June 2006. Next, we conducted an additional search using the names of these authors to identify other relevant articles. Finally, after author names and affiliations were removed, 50 articles were evaluated by three readers for scientific accuracy and for tone. Scientific accuracy was assessed based on whether or not the findings of the WHI were accurately reported using two criteria: (1) Acknowledgment or lack of denial of the risk of breast cancer diagnosis associated with hormone therapy, and (2) acknowledgment that hormone therapy did not benefit cardiovascular disease endpoints. Determination of promotional tone was based on the assessment by each reader of whether the article appeared to promote hormone therapy. Analysis of inter-rater consistency found moderate agreement for scientific accuracy (κ = 0.57) and substantial agreement for promotional tone (κ = 0.65). After discussion, readers found 86% of the articles to be scientifically accurate and 64% to be promotional in tone. Themes that were common in articles considered promotional included attacks on the methodology of the WHI, arguments that clinical trial results should not guide treatment for individuals, and arguments that observational studies are as good as or better than randomized clinical trials for guiding clinical decisions. The promotional articles we identified also implied that the risks associated with hormone therapy have been exaggerated and that the benefits of hormone therapy have been or will be proven. Of the ten authors studied, eight were found to have declared payment for speaking or consulting on behalf of menopausal hormone manufacturers or for research support (seven of these eight were speakers or consultants). Thirty of 32 articles (90%) evaluated as promoting hormone therapy were authored by those with potential financial conflicts of interest, compared to 11 of 18 articles (61%) by those without such conflicts (p = 0.0025). Articles promoting the use of menopausal hormone therapy were 2.41 times (95% confidence interval 1.49–4.93) as likely to have been authored by authors with conflicts of interest as by authors without conflicts of interest. In articles from three authors with conflicts of interest some of the same text was repeated word-for-word in different articles.

Conclusion:
There may be a connection between receiving industry funding for speaking, consulting, or research and the publication of promotional opinion pieces on menopausal hormone therapy.

: Please see later in the article for the Editors' Summary

Introduction

About half of US gynecologists continue to distrust the results of the Women's Health Initiative (WHI) [1],[2],[3], which found that risks of menopausal hormone therapy outweighed benefits in asymptomatic women. Such resistance to the findings of the largest randomized, placebo-controlled trial of menopausal hormone therapy ever performed is curious.

The WHI enrolled more than 26,000 women. After more than five years of follow-up, the estrogen-progestin arm of the WHI was stopped in 2002 due to harm; the estrogen-only arm was stopped in 2004, also due to harm [4],[5]. Both therapies increased the risk of stroke, deep vein thrombosis [4],[5], dementia [6], and incontinence [7]; estrogen-progestin therapy also increased rates of breast cancer [4]. Neither therapy reduced cardiovascular risk, and neither markedly benefited health-related quality of life measures [8],[9].

Over the year after the first WHI results were announced, hormone prescriptions dropped by 80% [10]. Compared to 2002, the age-adjusted incidence of breast cancer diagnosis dropped 6.7% in 2003, a finding attributed to decreased use of hormone therapy among postmenopausal women [11],[12]. A recent report confirmed an increased breast cancer incidence and a doubling of breast cancer-associated deaths among hormone users in the WHI [13].

In the US, gynecologists were more likely than internists or family physicians to prescribe hormone therapy to asymptomatic women both before [14],[15] and after [16],[17] the WHI. Between 1999 and 2002, 90 million hormone therapy prescriptions were written annually [18]. In 2002, US gynecologists wrote 70% of estrogen or estrogen-progestin prescriptions [19]. Sixteen months after the WHI ended, a survey of 705 fellows of the American College of Obstetricians and Gynecologists found that 49.1% of gynecologists did not find the WHI convincing and that 48.1% disagreed with the decision to stop the trial [1]. Follow-up surveys found similar results [2],[3].

Publications in the medical literature by industry-paid physicians have recently received attention [20]. Several academic analyses have used internal industry documents disclosed in litigation to document the use of messaging in publications regarding rofecoxib (Vioxx) [21], gabapentin (Neurontin) [22], and sertraline (Zoloft) [23]. Thousands of internal documents are available at the Drug Industry Document Archive (http://dida.library.ucsf.edu/). Industry-funded reviews and commentaries may be designed to convey specific, but subtle, marketing messages [24]. As a possible explanation for why so many physicians continue to support the use of menopausal hormone therapy in asymptomatic women, we investigated whether promotional tone could be identified in narrative review articles regarding menopausal hormone therapy and whether articles identified as promotional were more likely to have been authored by those with conflicts of interest as determined by declared payments from hormone manufacturers.

Methods

Search Strategy

In 2006, we conducted a MEDLINE search using Ovid, limited to English language reviews or comments published from July 2002 through June 2006 (the four years after cessation of the estrogen-progestin arm of the WHI). The search terms were “estrogen replacement therapy” combined with “breast neoplasms AND menopause” or “cardiovascular disease AND menopause”; “hormone replacement therapy” combined with “breast neoplasms AND menopause” or “cardiovascular disease AND menopause”; and “menopausal hormone therapy” combined with “breast cancer.” A list of all authors was made and the number of publications per author assessed. A subset of the most prolific authors was identified. One reader (CPM) assessed article titles, eliminating articles that did not focus on benefits and risks of combined estrogen-progestin menopausal hormone therapy since the WHI. An additional MEDLINE search was conducted using the names of these authors to identify other relevant articles.

Analysis of Content

Articles were obtained, and information on identifying authors and affiliations was removed, by support staff. All articles were independently evaluated for scientific accuracy and for tone by three readers (CPM, ECB, AMB) who were then graduate students in physiology. We used graduate students in physiology as readers because these individuals have the requisite knowledge to understand the technical aspects of the literature but have no experience as health care providers or as the targets of pharmaceutical promotional efforts. Scientific accuracy was assessed based on whether or not the findings of the WHI were accurately reported using two criteria:

• Acknowledgment or lack of denial of the risk of breast cancer diagnosis associated with menopausal hormone therapy, and

• Acknowledgment that menopausal hormone therapy did not benefit cardiovascular disease endpoints.

Determination of tone was based on each reader's assessment of whether the article appeared to promote hormone therapy (such articles were deemed “promotional”) or not (articles deemed “nonpromotional” were either neutral about hormone therapy or argued against routine use). Because no validated tools exist for assessing tone, each reader made her determination according to personal criteria. Readers were asked to record features of each article supporting their assessments. Assessments were discussed only at team meetings, and readers did not otherwise communicate with each other about the articles.

The initial literature search identified 340 articles with a total of 428 authors. Ten authors had published four, five, or six articles; 47 authored two or three articles; and 371 authored one article each. We then excluded articles that focused on selective estrogen receptor modulators, androgens, specific receptors, or specific mechanisms, and other articles unlikely to discuss the risks and benefits of menopausal hormone therapy. After excluding these articles, 232 potentially relevant articles remained.

The ten authors who had published four to six articles accounted for 47 (20%) of these articles. We chose to focus on these ten authors because they contributed a fifth of the extant literature during the four-year period under study, and could be assumed to have been widely read and thus influential. An additional literature search using these author names identified 13 additional relevant articles for a total of 60 articles for blinded evaluation by the three readers. Ten articles were subsequently excluded by mutual agreement of all readers (two publications were letters, three were duplicated references, four covered topics outside the scope of the study and did not specifically address our criteria, and one was published before July 2002). The remaining 50 papers, listed in Text S1, were evaluated individually by all readers. The ten authors who published four or more articles in the four-year period under study are identified in Table 1.

After individually analyzing a batch of articles, readers met to provide their initial assessments, to discuss them, and to come to consensus, if possible, on scientific accuracy and tone. Disagreements that could not be resolved in discussion were resolved by majority vote. After all articles had been evaluated individually and together by all readers, the authors were revealed and a search was performed for potential financial conflicts of interest, defined as evidence that the authors had received payment for research, speaking, or consulting on behalf of a manufacturer of menopausal hormone therapy.

Assessment of Conflicts of Interest

Conflicts of interest were assessed by examining published declarations within MEDLINE-indexed articles, disclosures from the Council on Hormone Education (a Wyeth-funded group of consultants) [25]–[28], and a Google search of each author's name combined with the term “conflict of interest.” Conflicts of interest were assessed for the time period ending 30 June 2006. No evaluations of potential conflicts of interest disclosed after that date were attempted.

Statistical Analysis

The Fisher exact test was used to assess the distribution of articles between conflicted and nonconflicted authors. A two-sided p-value of 0.05 or less was accepted as statistically significant. Risk ratios were estimated using the SABER program from the US Centers for Disease Control and Prevention. The kappa statistic (κ) was calculated according to the method of Brennan and Prediger [29].

Results

The majority of articles (86%) were judged to be scientifically accurate according to our analysis. Thirty-two (64%) of the 50 articles were assessed as promotional.

After individual evaluation, all three readers agreed on both scientific accuracy and tone for half (50%) of the articles. Readers were in agreement regarding scientific accuracy for 34 articles (68%) and regarding promotional tone for 39 articles (78%). Analysis of inter-rater consistency found moderate agreement for scientific accuracy (κ = 0.57) and substantial agreement for promotional tone (κ = 0.65). After the consensus discussions, there was agreement regarding scientific accuracy for 49 articles (98%) and regarding promotional tone for 48 (96%) articles. Overall, readers were in 94% agreement prior to unmasking the identity of the authors.

Themes in Promotional Articles

Readers identified themes common among promotional articles. These themes included:

• The WHI was flawed.

• The WHI was a controversial trial.

• The population studied in the WHI was inappropriate or was not representative of the general population of menopausal women.

• Clinical trial results should not guide treatment for individuals.

• Observational studies are as good as or better than randomized clinical trials.

• Animal studies can guide clinical decision-making.

• The risks associated with hormone therapy have been exaggerated.

• The benefits of hormone therapy have been or will be proven; recent studies are an aberration.

For examples of statements representative of the themes identified, see Table 2. Examples of what were considered promotional and nonpromotional treatments of similar topics are given in Table 3.

Articles evaluated as promotional frequently differed from nonpromotional articles in the way that risks were presented. For example, most articles conceded that hormone therapy was associated with breast cancer, but promotional articles contained statements such as “The risk of breast cancer with hormonal therapy is put into perspective with the realization that this risk is related to hormonal dose and duration of use, and that the absolute risk remains small” (reference A14 in Text S1), or “The WHI agreed with convincing evidence in the literature that postmenopausal hormone therapy does not increase the risk of breast cancer beyond that already associated with recognized risk factors, such as a positive family history” (A10 in Text S1). A nonpromotional article stated “Although estrogen and/or progestin effectively reduce vasomotor symptoms, a recent WHI randomized trial identified an unfavorable risk/benefit balance on life-threatening diseases, including increased breast cancer, for combined estrogen plus progestin use in otherwise healthy post-menopausal women” (A40 in Text S1). See Table 3 for more examples of statements that minimize risks.

Reuse of Previously Published Text

Our analysis also found, incidentally, that three authors we identified as having financial conflicts of interest were authors on articles where sections of their previously published articles were repeated word-for-word. Neither of the authors without declared conflicts of interest were noted to have reused text in the articles analyzed.

Eighty-four percent of the text and all seven tables in one article (A9 in Text S1) were found in four other articles by Leon Speroff (A8, A10, A30, and A50 in Text S1). Seventy-three percent of the text and seven of ten tables from one of these articles (A10 in Text S1) were found in three other articles by this author (A9, A30, and A50 in Text S1). More than half (55%) of the text of one article (A8 in Text S1), published in Maturitas, was reused in the same journal a year later (A9 in Text S1). No acknowledgment of earlier publication was made.

Twenty-five percent of an article coauthored by Susan R. Davis was reused (A25 in Text S1) in another article coauthored by the same author, without acknowledgment of earlier publication (A26 in Text S1). Both articles included additional authors and we could not determine which authors were responsible for the reuse. Over two-thirds (71%) of the text of an article by Rogerio A. Lobo (A33 in Text S1), as well as its two figures and table, were reused (A14 in Text S1); in this case, adaptation, but not republication, was acknowledged (A14 in Text S1).

Conflicts of Interest

Of the ten authors in our sample, eight were found to have received payment for research, speaking, or consulting on behalf of menopause hormone manufacturers (Table 1). About half of the articles analyzed had coauthors in addition to the ten authors we assessed, but we did not examine whether there were potential financial conflicts of interest for these coauthors.

The assessments of scientific accuracy and promotional tone after the consensus discussion are summarized in Table 1 according to author's potential conflict of interest. Thirty of 32 articles (90%) evaluated as promoting hormone therapy were authored by those with conflicts of interest compared to 11 of 18 articles (61%) by those without conflicts of interest. The difference was significant (p = 0.0025). Two of nine articles (22%) by authors without known conflicts were considered promotional, and 30 of 41 articles (73%) by authors with known conflicts were considered promotional. Articles promoting the use of menopausal hormone therapy were 2.41 times (95% CI 1.49–4.93) more likely to have been written by authors with conflicts of interest than by authors without conflicts of interest.

Discussion

This study evaluated the relationship between the receipt by authors of payment from industry for speaking or consulting and authorship of articles considered to be scientifically in error or promotional in tone. We identified an association of review articles promoting the use of hormone therapy with authors with declared financial conflicts of interest. Scientific accuracy did not appear to be affected by author conflicts of interest.

The effect of industry funding on results in clinical trials [30], meta-analyses [31],[32], clinical practice guidelines [33], and pay-for-performance quality measures [34] has been well documented. Two publications have documented scientifically unsupportable statements on the risks and benefits of menopausal hormone therapy in the medical literature [19],[35]; many of these statements appeared in reviews, commentaries, editorials, and letters.

We assessed scientific accuracy based on whether or not two findings of the WHI were accurately reported: (1) There is no proven cardioprotective effect of estrogen-progestin therapy in menopausal women or in ten-year age subgroups of menopausal women, and (2) breast cancer is diagnosed more frequently in menopausal women receiving estrogen-progestin therapy [4],[36]. Promotional tone was evaluated without formal criteria, but the readers were asked to identify elements of the articles that conveyed a promotional tone. Readers evaluated articles masked as to the identity or affiliations of the authors. Prior to discussion, there was substantial agreement among the individual readers for promotional tone, and moderate agreement on scientific accuracy. After discussion, but prior to unmasking of author identities, there was consensus on both measures for all but two of the 50 papers evaluated.

We found that articles with a promotional tone were more likely to have been written by authors who had disclosed financial conflicts of interest than by authors without such disclosures. These conflicts were determined through publicly available declarations of conflicts of interest and may not be accurate or complete. It is possible, for example, that authors whom we identified as having no potential conflicts had undeclared conflicts or developed conflicts after the period we examined. One author, Nanette Wenger, for whom we identified no potential conflicts with hormone manufacturers during our study, later declared potential conflicts [37].

Our sample size of ten authors was small, although the authors assessed had written one out of five of the articles identified in our search. The prevalence of financial conflicts among authors in general is unknown, and our findings may not reflect the universe of authors. It is possible that an assessment that included less prolific authors would come to a different conclusion.

Almost all articles were evaluated as scientifically accurate regarding the effect of hormones on breast cancer diagnosis and cardiovascular risk, but readers found phrasing that minimized the risk of breast cancer or seemed to encourage reliance upon animal studies, observational studies, or expert recommendations rather than on randomized controlled trials. Our results support an in-depth interview study that found that physicians at two health plans commonly believed that WHI “was not applicable to the full range of patients seen in clinical practice” and “created uncertainty about the risks and benefits of HT” [38].

Our results suggest that authors who have received payments from industry convey more enthusiasm about the industry's products than do authors who have not declared that they received such payments. These results support the findings of a study that examined conflicts of interest in reviews (among other publications) and found that articles by authors with potential financial conflicts of interest were more likely to support the use of a specific class of drug therapy [39]. The question of whether positive feelings about hormone therapy preceded payments from industry and were perhaps a basis for selection of these physicians as speakers and consultants or whether selection as a speaker or consultant led to more positive feelings about hormone therapy is an issue that should be explored in further research.

Our findings also support an analysis by Tatsioni and colleagues of “partisan editorializing articles on HRT” in the Thomson ISI database by five editorialists who had written at least 12 commentaries in medical journals between 2002 and 2008 [40]. All five had financial relationships with hormone manufacturers; these relationships were reported in only six of the 110 articles analyzed. Although there is no overlap in the author list between the Tatsioni analysis and ours, Tatsioni and colleagues identified similar themes, noting that common arguments included “HRT is effective for menopausal and related symptoms”; “Discussion of preclinical data that showed favorable effects for HRT”; “Statements challenging/criticizing unfavorable studies” (especially against the WHI and the Million Women Study); and “Statements that HRT may decrease life-threatening and other serious outcomes.” Additionally, Tatsioni et al. note that text was sometimes repeated verbatim in several articles; examples are provided in their online supplemental materials [40].

A scientist who consults for the pharmaceutical industry has described the process by which companies formulate key marketing messages into a product narrative to affect the discourse of medicine and ultimately medical knowledge [41]. Although promotional linguistic and rhetorical strategies have been identified in television commercials for prescription drugs [42], there is a dearth of academic articles on the use of rhetoric and persuasion in medical journal articles. To our knowledge, the study by Tatsioni et al. and our study are the first to attempt to assess tone in review articles published in medical journals.

The extent of text reuse we identified was surprising. Tatsioni et al. documented different examples of text repeated verbatim in articles on menopausal hormone therapy, raising the question of how many more articles in the medical literature contain previously published passages. An editorial in The Lancet noted that text recycling in review material could be viewed as “less of a crime” than “self-plagiarism” of original research, but that the practice “constitutes intellectual laziness at best” and is unacceptable [43].

Limitations

The methodology used to evaluate promotional tone for this study has not been previously validated. Our evaluators were not physicians and it is possible that the use of physician evaluators would have yielded different results.

It is possible that the authors for whom no conflicts of interest were found actually did have conflicts of interest, either because we failed to identify a conflict or because a conflict was not disclosed. Misclassification of conflicted authors would be expected to bias the study results toward the null and is unlikely to be responsible for the difference in tone that we identified.

We cannot be certain that the ten authors we evaluated were representative of the universe of authors writing review articles on hormone therapy during the study period. We selected these authors because they were responsible for 20% of the relevant literature during the time period, but a study of authors with fewer publications during the period may have revealed different results. We also did not assess possible conflicts of interest of coauthors or the contribution these coauthors may have made to the accuracy or tone of the articles we assessed.

The assessment of multiple articles by each author may introduce an overcounting problem in the statistical analysis inasmuch as each author's perspective might be expected to stay the same. However, as can be seen in Table 1, about a quarter of articles by authors with potential financial conflicts of interest were deemed nonpromotional, and about a quarter of articles by those without potential financial conflicts of interest were deemed promotional.

Documents recently disclosed in litigation against manufacturers of hormone therapy revealed that dozens of articles ghostwritten by industry were published in the medical literature [44],[45]. The names of two of the authors whose work we assessed, Rogerio Lobo and Leon Speroff, were on the bylines of some of the reportedly ghostwritten articles [45]–[55]. We could not determine whether or not any of the articles assessed in our study were ghostwritten.

Conclusion

Our study found that narrative review articles on hormone therapy may provide accurate statements about the risks of a therapy while simultaneously providing positive impressions of that therapy for uses unsupported by evidence. There may be a connection between industry funding for research, speaking, or consulting and the publication of promotional pieces on menopausal hormone therapy. Health care providers should exercise caution if they choose to read such articles. We believe that medical journals should follow the International Committee of Medical Journal Editors Uniform Requirements for Manuscripts (http://www.icmje.org/urm_main.html), which require that all authors submit signed statements of their participation in authorship and full disclosure of any conflicts of interest. In order to prevent the bloating of journals with pages of “recycled” text, medical journals should consider using antiplagiarism software.

Supporting Information

Attachment 1