Opinion | Challenges in Developing Long COVID Criteria

In a recent JAMA article, researchers provided a new approach for identifying individuals with post-acute sequelae of SARS-CoV-2 (PASC), commonly referred to as long COVID. Because this article — from Tanayott Thaweethai, PhD, and colleagues — was part of the large, federally funded U.S. RECOVER study of long COVID, it will be influential among scientists around the world who are struggling with developing a case definition for long COVID. In this opinion piece, we suggest issues that might limit the interpretation of some of the study’s findings — in particular the identification of people who have long COVID.

In their introduction, Thaweethai and coauthors indicate that “Most existing PASC studies have focused on individual symptom frequency…” This is true, and it is a limitation in past studies, as either the “occurrence” of symptoms or just the “frequency” of symptoms are inadequate to help investigators understand the true burden of a post-viral symptom on a patient’s life. Studies need to incorporate severity ratings into their scales. For example, some symptoms occur frequently, but their severity is so low that they may not burden the patient.

My concern is that the study from Thaweethai and colleagues still failed to broadly and sufficiently incorporate the “severity” measure, and used imprecise definitions and phrasing in some instances that may have skewed patient responses.

The Measure of Severity

Thaweethai and colleagues inform the reader in the Methods section of their paper that “Symptom frequency was defined as the proportion reporting a symptom and exceeding corresponding moderate to severe symptom severity threshold.”

The authors identified 12 symptoms differentiating those who were infected versus not infected. A symptom score was provided for each symptom, and anyone who scored 12 or greater was considered to be PASC positive. Within Table 2, which lists the 12 key symptoms used to construct the PASC positive score, only three had severity ratings. In other words, the majority of the 12 symptoms in Table 2 that were used to identify individuals as PASC positive were based on occurrence or frequency rather than severity. Severity ratings may be critical to disease identification, and have been used previously to differentiate PASC from other post-viral illnesses (for example, Oliveira et al. found that unrefreshing sleep and flu-like symptoms were the best symptoms differentiating PASC from myalgic encephalomyelitis/chronic fatigue syndrome [ME/CFS]).

Among the 12 symptoms, smell/taste was the best discriminator and was associated with 8 points. So, if a respondent had this symptom, it counted for 8 of the needed 12 points to meet the criteria for long COVID. However, “Loss of or change in smell or taste” involved occurrence data rather than either the frequency or severity of this symptom. Their study found that 13% of those infected by SARS-CoV-2 had this symptom, but among those who were ultimately classified as PASC positive, 41% had this symptom. We examined the same item, “loss of or change in smell and/or taste,” in another large dataset of infected participants using both frequency and severity to measure this symptom rather than just assessing occurrence. Using the established ME/CFS criteria of the symptom frequency of at least half the time and severity level of at least moderate, only 18% met the criteria for this symptom. Having clear frequency and severity criteria led to a decrease in this critical symptom from 41% in the RECOVER study to 18% in a study with better measures.

Symptom Definitions and Question Phrasing

Another key symptom did not make it onto the list of 12 most important criteria differentiating those with and without PASC in the RECOVER study. In the article by Thaweethai and colleagues, “sleep disturbance” occurred in 31% of infected individuals, but when a severity measure was used, only 12% met this criteria. Their criteria for “sleep disturbance” was an eight-question survey asked of participants who reported currently having the following symptom: “Stopping breathing during sleep or sleep problems (such as snoring, trouble falling asleep, nighttime awakenings, or trouble staying awake during the day) 3 or more times a week.” Among those classified as PASC positive, 32% had the symptom of sleep disturbance. However, in the other dataset, if the more precise term “unrefreshing sleep” was used, then 78% of participants met the criteria of at least moderate severity and frequency of at least half the time. Two other studies using this phrasing of “unrefreshing sleep” yielded similar results. The term “unrefreshing sleep” has been used for decades in ME/CFS research, and had it been used along with frequency and severity data, unrefreshing sleep might have been selected by RECOVER as one of the key symptoms differentiating those with long COVID from those without. Our understanding of critical long COVID symptoms is impacted by whether imprecise occurrence measures are employed or more specific frequency and severity measures and thresholds are used, as well as the way symptoms are phrased.

It’s worth examining in more detail how questions are asked. For years, studies in the ME/CFS field only assessed the occurrence of symptoms, but in the 1990s it became apparent that the somatic symptoms in the ME/CFS criteria were common in the general population, so occurrence measures were determined to be too imprecise. The field then moved to assess the frequency of symptoms in the ME/CFS case definitions. However, researchers encountered another conceptual problem as patients with major depressive disorder had a similarly high frequency of fatigue as patients with ME/CFS. In other words, one of the most prevalent mental health disorders could not be differentiated from ME/CFS using measures of frequency alone. However, when measures of severity were introduced, those with ME/CFS had significantly higher levels of fatigue severity than patients with major depressive disorder. By assessing frequency and severity, it was possible to make this important diagnostic differentiation between ME/CFS and psychiatric conditions.

Another limitation in the RECOVER study was the authors’ PASC definition, which required a score of 12 or greater to be classified as PASC positive among the 12 symptoms. A person with severe symptoms of dizziness, palpitations, and gastrointestinal symptoms could be considered significantly burdened and impacted, yet would only register a score of 4 rather than 12, and thus not meet the criteria of PASC positive. In Figure 2c, 93 of the 7,733 individuals (1.2%) with symptom scores less than 12 indicated their general physical health as fair or poor on the PROMIS Global 10 criteria. One severe symptom can disable a person, and it is important that diagnostic systems allow such individuals to be included within a PASC classification system. Such a suggestion that severely impacted patients do not meet PASC criteria can have significant consequences for patients and their care.

In conclusion, measures of post-viral illnesses that assess both frequency and severity have methodologic advantages. When used with psychometrically sound questionnaires, they increase investigators’ ability to interpret the data. It’s important we get this right for long COVID.

Leonard Jason, PhD, is a professor of psychology and director of the Center for Community Research at DePaul University in Chicago. He is also chairperson of the Diagnostics Testing and Test Algorithms subcommittee of the NIH RECOVER Commonalities With Other Post Viral Syndromes Task Force, and serves as ME/CFS expert for ILLInet RECOVER. Nicole Hansel is research assistant at the Center for Community Research, and is currently working on a study involving a prospective study of ME/CFS in college students.