Aducanumab Data Finally Published, but Alzheimer’s Experts Spot Problems

The long-awaited results of the twin phase III trials for aducanumab (Aduhelm), the controversial drug approved last year for early Alzheimer’s disease, were published in the Journal of Prevention of Alzheimer’s Disease (JPAD), a journal whose editor-in-chief is aducanumab researcher and study co-author Paul Aisen, MD, of the University of Southern California in Los Angeles.

The findings from ENGAGE and EMERGE had already been presented in detail to an FDA advisory committee — which voted overwhelmingly against the data — and at meetings, but not in a peer-reviewed publication until now. Its publication in JPAD, rather than JAMA or another top-tier journal, raised questions in the Alzheimer’s research community.

“Fully 3 years after the futility stop and 9 months after the accelerated approval, it is a relief to have a published manuscript on the aducanumab phase III trials,” noted David Knopman, MD, of the Mayo Clinic in Rochester, Minnesota, who wasn’t involved with the analysis.

The primary endpoint of the trials was the change from baseline to week 78 on the Clinical Dementia Rating Sum of Boxes (CDR-SB), a scale of function and cognition with decreasing scores indicating better performance. In EMERGE, a group of participants who received high-dose aducanumab showed a decrease of -0.39 in CDR-SB scores compared with placebo (95% CI -0.69 to -0.09, P=0.012, a 22% decrease). In ENGAGE, the difference was 0.03 (95% CI -0.26 to 0.33, P=0.833, a 2% increase). The most common adverse events were amyloid-related imaging abnormalities (ARIA).

But “the report contains no information that was not otherwise available from the sponsor or the FDA proceedings,” Knopman told MedPage Today. “The venue — the Journal of Prevention of Alzheimer’s Disease — in which the report was published is a lower-tier journal that is not widely available that clearly has a laissez faire attitude towards standards for reporting clinical trials.”

“In addition, there is an appearance of a conflict of interest in that two of the authors [Aisen and Bruno Vellas, MD, of Toulouse University Hospital in France] of the report happen to be editors of this journal,” Knopman continued. (Aisen and Vellas said they did not have a role in the manuscript acceptance and review process.)

“That Biogen was forced to publish in a venue of low impact and low editorial standards is a revealing message about the unrealistically optimistic interpretation of results,” he added.

The paper clearly describes the mixed outcomes in the trials in its abstract and in the conclusion of its discussion section, Knopman noted. “The discussion contains the discredited view that inadequate exposure to high-dose aducanumab in the ENGAGE trial explained the discordant results, but overall, the problems with the results were clearly presented,” he said.

“The fundamental message is the same as was previously known: the post hoc analysis of one of the trials, EMERGE, showed a small clinical benefit for high-dose aducanumab, while the post hoc analysis of the other trial, ENGAGE, failed to show any benefit at either low- or high-dose,” Knopman continued.

“What was missing from the article was the obvious conclusion that the discrepant findings between EMERGE and ENGAGE cannot make a case for using aducanumab in our patients without a new trial,” he added.

“Glaring Error” in Statistical Reporting

To biostatistician Scott Emerson, MD, PhD, of the University of Washington in Seattle, a member of the FDA advisory committee that reviewed aducanumab data before the drug was approved, the paper has a “glaring error” in its characterization of its multiple testing strategy.

“The validity of any P-value or confidence interval depends on the sampling scheme,” Emerson told MedPage Today. “For this reason, we require prespecified endpoints and analysis plans to control for multiple comparisons. Otherwise, we are in grave danger of the ‘Texas sharpshooter fallacy'” — where someone first fires a shot at a barn then paints a target around the bullet hole, he noted.

The study protocols called for a sequential testing procedure to control the overall type I error rate in the presence of multiple comparisons for the primary and secondary endpoints, Emerson noted. The sequence was designed to start with high-dose aducanumab versus placebo on CDR-SB, then low-dose versus placebo on CDR-SB, before moving to the three secondary endpoints, he continued.

“In EMERGE, CDR-SB for high-dose versus placebo was significant, but not low-dose versus placebo. So testing stops, and you cannot claim it met the secondary endpoints while controlling type 1 error,” Emerson said.

This was a point made in the evaluation prepared by FDA statistical reviewer Tristan Massie, PhD, for the FDA advisory committee meeting, which indicated that no secondary endpoints in EMERGE would be significant following the prespecified hierarchy and multiplicity adjustment plan.

The paper, however, states that “data from EMERGE demonstrated a statistically significant change across all four primary and secondary clinical endpoints” based on the sequential testing procedure prespecified in the study protocols, Emerson observed. “They are trying to give the impression that the secondary endpoints stand up to rigorous statistical scrutiny, which they do not,” he said.

“The fact that the journal allowed the publication to differ from the prespecified statistical analysis plan implicates the journal review process,” Emerson pointed out.

“Biogen is reported to have withdrawn its prior submission of a publication to JAMA, and I wonder whether a failure to adhere to the statistical analysis plan is a reason: the peer review and editorial process for the top-line medical journals pay close attention to the prespecified protocols and statistical analysis plans,” he added.

“Personally, the misrepresentation in the publication starts bordering on scientific misconduct, unless the sponsor can justify the statement,” he added. “I do not have access to all the communications between the sponsor and the FDA regarding the statistical analysis plan, but presumably the FDA statistical review team did, and that review noted that the secondary endpoints were not statistically significant.”

MedPage Today attempted to reach Biogen’s Samantha Budd Haeberlein, PhD, the paper’s lead author, to review the prespecified statistical analysis plan. Budd Haeberlein did not respond; a Biogen media spokesperson answered instead and said the testing of the secondary endpoints in the high-dose group was not contingent on the result of the primary endpoint in the low-dose group, per the prespecified study protocol.

“The clinical principle underpinning the testing strategy was that high-dose (10 mg/kg) was the target dose,” the spokesperson wrote. “Therefore, failure of low-dose on any endpoint should not preclude testing of the high-dose.”

As to why the study was published in JPAD instead of a well-known journal, the Biogen spokesperson wrote, “It was always our intention for our phase III data to be peer-reviewed, and we have worked to that end. The peer review process can take time. We are not providing further details of our publication process.”