Another Casualty of COVID-19: Evidence-Based Medicine

MIAMI BEACH — COVID-19 has not only exacted a heavy human and economic toll, but the pandemic effectively “killed evidence-based medicine in this country,” a New York oncologist argued in a presentation here.

During the first 18 months of the pandemic, more than 200,000 COVID-related articles were published in various types of journals, said Patrick Borgen, MD, of Maimonides Medical Center in New York City. Submissions to publishing giant Elsevier increased by 58% from February to May 2020 as compared with the same period in 2019.

The “appalling” loss of life and morbidity during the early months of the pandemic, driven by a lack of effective therapies, led to “an explosion of people trying to help, through hundreds of thousands of publications.”

“I like to argue that we were building the plane while we were flying it during COVID, that we were grasping at therapeutic straws,” Borgen said during a departure from scientific presentations at the Miami Breast Cancer Conference. “This is known as apophenia: the tendency to identify meaningful patterns where none truly exist. It’s a part of human nature, and I believe it’s exactly what happened.”

“I don’t think that the explosion of publications was nefarious. I don’t believe that it was self-serving. I believe that it was apophenia. We were desperately looking for help,” he stated.

The publishing tidal wave continued into 2021, reaching almost 250,000 publications by the end of the first quarter, including 235,000 primary studies. Hydroxychloroquine accounted for 250 studies, most involving fewer than 100 patients. During that time the peer review process “was put on steroids,” said Borgen. An analysis of 11 highly cited journals showed that the median time for peer review decreased from 90 days in 2019 to 19 days for COVID-related articles in 2020.

COVID-related literature flourished on preprint websites, which accept unpublished articles that have not been peer reviewed. More than 30,000 articles about the pandemic have been published as preprints, said Borgen. MedRxiv, launched in June 2019, had 11,000 submissions during the first year of its existence, fueled by the pandemic. The site published 90% of submissions, but only 10% subsequently appeared in peer-reviewed publications. Moreover, the website removed only 18 articles (0.002%) after posting.

“If you’re a young writer looking for an emerging story, you go to preprints,” said Borgen. “Many of [the preprints] were picked up by the media and disseminated as evidence-based medicine.”

As an example, he cited an early study of convalescent plasma, showing that patients with hematologic malignancies who received convalescent plasma had a 13% improvement in survival. One widely read news article said the study showed that patients who received convalescent plasma were 86% less likely to die of COVID-19.

On March 16, 2020, MedRxiv posted the manuscript for a 36-patient study of hydroxychloroquine’s ability to clear the virus from nasal swabs. After 6 days, two of 16 patients in the control group had clean swabs as compared with 12 of 20 who received hydroxychloroquine. The results were reported as “highly positive” for hydroxychloroquine.

Also on March 16, the article was submitted to the International Journal of Antimicrobial Agents (whose editor was a co-author of the article), approved for publication March 17, published on March 20, and tweeted by former President Trump on March 21. The FDA approved an Emergency Use Authorization (EUA) on March 28.

“This is the fastest EUA approval in the history of the United States,” said Borgen. “Nothing else has ever come close. Based on a study of 30 patients with nasal swab PCR clearance.”

“If you look at all of the trials on hydroxychloroquine, they are profoundly biased across eight domains of bias … and the one that led to the EUA approval was clearly the most biased of all the studies.”

As a final example of COVID’s adverse impact on scientific research and reporting, Borgen cited a literature review and meta-analysis published by three economists, who evaluated the effectiveness of lockdowns on COVID-19 mortality. The authors whittled an initial list of 18,590 publications down to 34 published studies.

The authors concluded that lockdowns have had little or no public health benefits, imposed enormous economic and social costs, were ill-founded, and “should be rejected as a pandemic policy instrument.”

Borrowing from a critique that described the study as “not a met-analysis but a simple narrative review,” Borgen said, “Whether or not you believe [the authors’ conclusions], this study didn’t show it. They didn’t address the biology of the virus. They didn’t address modes of transmission. They didn’t address the time delay between exposure and development of symptoms. They simply showed it from the economic literature.”

“It excluded all of the science and yet it was picked up by 1,300 news media outlets and disseminated as good science,” he noted.

In closing, Borgen said a fundamental mistrust of medical science has arisen from a polarized society. In general, medical school has “very poor training in evidence-based medicine.”

“We’ve got to have our trainees, our fellows, residents, and medical studies better at critically appraising literature,” he said. “That’s really the fix; better education and emphasizing how important it is to be your judge. In breast cancer, am I going to take this study at face value, am I going to simply believe what the media tells me about it?”