Text

Background: It is essential to evaluate the impact of resource-intensive clinical trials on clinical practice to monitor the use of research funding. One approach is to analyze whether publications reporting clinical trial results are cited in systematic reviews or guidelines, as these inform clinical practice. Cochrane Reviews are regarded as the gold standard of evidence synthesis and may inform guideline development.

Aim/scientific question: To assess the extent to which results publications from a sample of clinical trials associated with German university medical centres (UMCs) were cited in Cochrane Reviews and how they contributed to the body of evidence and the results of the Cochrane Review.

Material & Methods: We analyzed an existing sample of journal publications linked to registered clinical trials associated with German UMCs (completed 2009–2017). Using forward citation tracking in OpenAlex and PubMed, we identified citing Cochrane Reviews and categorized as included in the body of evidence excluded or other. Primary outcome was the proportion of publications included in the body of evidence of at least one Cochrane Review. Details of the citation context were qualitatively explored in a subsample of 10 publications.

Results: Among 1921 publications, 386 (20.1%) were cited in at least one Cochrane Review and 222 (11.6%) were included in the body of evidence. Included studies were more often randomized (98.6% vs. 67.0%), blinded (57.2% vs. 39.3%), and multinational (24.3% vs. 15.3%) compared to studies that were not cited or not included. Qualitative analysis showed substantial heterogeneity in how included trials contributed to the results of a Cochrane Reviews, ranging from no contribution to major influence across multiple meta-analyses.

Discussion: Only a limited proportion of clinical trials in our sample were incorporated into Cochrane Reviews, indicating restricted translation into clinical practice via this pathway. Included trials were mostly randomised and blinded. Our analyses indicate that citation-based analyses might overestimate the actual impact of clinical trials. More nuanced approaches (e.g. on citation context) are needed to assess the impact of clinical trials.

Take Home Message: To improve translation into clinical practice, trials should be designed in ways that enable inclusion and meaningful contribution (i.e., blinded and randomized). Assessing the impact of clinical research to inform clinical practice requires in-depth analysis of the context of the contribution to the quantitative evidence synthesis.

Competing interests: None