Spin Doctors: Editorial Sleight of Hand in Negative Cancer Studies

Reports of cancer studies that failed to meet a noninferiority endpoint often had a “spin” that created the appearance of a benefit that did not exist, according to a literature review.

The review identified inaccurate reporting (spin) in 39 of 52 published reports. Editorial spin appeared most often in abstracts and conclusions, and inaccurate reporting appeared throughout the manuscripts in a fourth of published articles that had spin.

A multivariable analysis showed that spin was independently related to sole support from a nonprofit organization and evaluation of novel experimental therapies, reported Koji Oba, PhD, of the University of Tokyo, and co-authors in JAMA Network Open.

“Our analysis found that the conclusions of RCTs (randomized controlled trials) showed the highest spin prevalence compared with other sections, a finding consistent with other reports and with important implications, as most physicians are interested only in the conclusions sections of publications,” the authors wrote.

“Spin-based reporting of P values resulted in distorted conclusions. Moreover, readers might interpret studies to have positive results, even with results for the primary outcome that are not significant,” they added.

The findings of the review are consistent with previous reports but are noteworthy because of the focus on clinical trials evaluating the noninferiority of one intervention versus another, noted Yichen Zhang, MSc, and Xiaodong Guan, PhD, of Peking University in China, in an accompanying editorial.

“The goal of noninferiority trials is to show that a new treatment is safer, cheaper, or easier to administer, with no worse efficacy,” they wrote. “However, for clinical trials with negative results, misrepresented findings can disintegrate oncologists’ knowledge and perceptions about drug benefits.”

“The goal of a peer-reviewed scientific journal is to publish research with impartial results, which is the cornerstone of evidence-based decision making in either clinical or political scenarios,” Zhang and Guan continued. “The study by [Oba et al.] provides unique empirical data on misleading reporting in statistically nonsignificant oncology trials. Their findings highlight the necessity of reviewing potential misrepresentations of a manuscript before publication, as an author, editor, or reviewer, and this should not be reserved for only high-impact journals.”

Editorial spin is not new to published reports about cancer clinical trials, Oba and co-authors noted in their introduction. However, investigations of spin have focused on clinical trials testing the superiority of one treatment or intervention versus another. Noninferiority trials may involve more factors that complicate interpretation, such as the statistical margin for noninferiority, the sensitivity of an assay, and the patient population chosen for analysis.

For their examination of spin in noninferiority trials, the authors conducted a systematic review of negative noninferiority RCTs published from January 2010 through December 2019. They limited their analysis to reports that focused on direct comparisons of efficacy.

Oba and colleagues defined spin as “use of specific reporting strategies, from whatever motive, to highlight the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary endpoint, or to distract the reader from statistically nonsignificant results.” They applied the definition to each section of a publication: results and conclusions in abstracts, as well as results, discussion, and conclusion sections of the main text of a report.

The authors prespecified eight spin strategies to claim benefit for an experimental treatment: Emphasizing trends in point estimates

• Despite a lack of significance for the primary endpoint
• Based on secondary endpoints
• Based on results of subgroup analyses
• Based on a secondary analysis of the primary endpoint
• Based on intragroup comparisons
• With no mention in the discussion of the safety profile when safety was the primary objective
• Based on safety alone
• “Any other situations deemed as spin by the reviewers”

The review included 52 published reports. Three-fourths (39 of 52) of the reports had evidence of spin in one or more sections. The authors identified spin in 34 abstracts and in at least one section of the main text of 38 reports.

With regard to the abstracts, 10 reports had spin language in every section, and 23 had spin only in the conclusions. Spin in the main text appeared in all sections of six reports, in the discussion section only of three reports, and in the conclusion section only of 13 reports. In abstracts and main text, spin most often appeared in the conclusions.

The most common types of spin in abstracts were discussion of secondary endpoints with no mention of a nonsignificant primary endpoint (17.3%); discussion of secondary endpoints with a mention of nonsignificant primary endpoints (19.2%); and safety-related spin (11.5%).

With regard to the main text, 11.5% had secondary-endpoint spin and 13.5% had spin in secondary analyses of the primary endpoint. With regard to the conclusions section, 17.3% had secondary-endpoint or subgroup-analysis spin without mentioning the nonsignificant primary endpoint. An additional 21.2% had the same types of spin but mentioned the nonsignificant primary results. Six reports (11.5%) had safety spin in the conclusions.

A multivariable analysis showed that spin was more often associated with lack of funding from for-profit sources (OR 5.20, 95% CI 1.21-22.29) and reports about novel treatments (OR 4.64, 95% CI 0.98-22.02).

Last Updated December 07, 2021