Table 4

Additional limitations affecting the COVID-19 body of evidence and suggestions to mitigate their deleterious consequences

LimitationDefinition and exampleRecommendations to address limitation
ConfoundingDefinition: Extraneous factors influence the effect of interest and can lead to misrepresentation of a causal relationship.35 92
  • DAGs can be used to assess confounders in exposure–outcome relationships.36 93 94

  • Comprehensive reporting of patient characteristics.7

Example: Age, diabetes mellitus and obesity are likely confounders for mortality risk in patients with diabetes mellitus and hypertension.7 These variables have not always been reported in the primary literature, limiting the rigour of sensitivity analyses in MAs.
Collider bias
Definition: Two variables, exposure and outcome, influence a third variable, the collider, resulting in a false association between exposure and outcome.95
  • Weighted regression analysis to account for over-representation or under-representation of certain individuals.37

  • DAGs can be used to assess confounders in exposure–outcome relationships.36 93 94

Example: Testing for COVID-19, as currently conducted, may over-represent patients who are symptomatic, hospitalised and have better access to healthcare. Artificial associations may be established when using non-generalisable sample populations.34 95
Publishing demands raise concern regarding scientific qualityDefinition: The peer-review system is designed to mitigate the possibility that critically flawed articles reach circulation.
  • No compromise should be made to the rigour of the peer-review process.4

Example: From 1 January to 1 June 2020, JAMA has received more than 11 000 articles, compared with 4000 in the same time frame the previous year.4 Lancet Global Health has seen a 185% increase in submissions for June 2020 compared with June 2019.5
Preprint servers
Definition: Repositories for studies that have not yet undergone the formal peer-review process.
  • AI-powered literature reviews summarise key findings in articles to aid in identifying high-quality studies.1 11

  • Databases, such as the Novel Coronavirus Research Compendium, feature high-quality articles each with their own appraisal.11

Examples: Since 14 July 2020, 25 publications have been retracted, 10 of which have been preprint.96
  • AI, artificial intelligence; DAGs, directed acyclic graphs; MAs, meta-analyses.