Text

Background: Nonrandomized studies on treatment effects have a serious risk of bias when they do not control for all relevant confounders. Guidance recommends identifying potential confounders independently, systematically, and comprehensively, based on literature and expert opinion.

Aim/scientific question: To systematically identify and assess known potential confounders for the design of a target trial emulation on multiple sclerosis and to investigate its feasibility.

Material & Methods: We used the comparison of two drugs for relapsing-remitting multiple sclerosis (RRMS). We conducted a systematic literature analysis (date of last search November 2, 2023) of observational studies and randomized trials, qualitative interviews, and a quantitative survey with clinicians. All identified potential confounders were categorized and classified by clinical importance. We estimated the resource use per study step and per identified potential confounder.

Results: We assessed 53 observational studies and 27 randomized trials, surveyed 8 clinicians, and conducted 8 interviews with clinical experts. We identified a total of 160 individual potential confounders. While most of these were identified in the literature (n=125, 78%), interviews revealed 35 (22%) more. Of the 17 confounders rated by clinicians as very important for treatment decisions, 9 were identified by interviews (53%). All potential confounders related to patient perspectives were identified in interviews only. Across all data sources, the 5 most frequently considered factors were age (n=71), gender/sex (n=62), disability status (n=62), disease duration (n=53), and relapses (n=44); these were all considered in 16 (30%) of observational studies. Sixty-two person-days were deemed necessary to identify the 160 potential confounders.

Discussion: There was a substantial discrepancy between factors considered when making treatment decisions and the confounders considered in studies. Even with extensive efforts, it is likely that observational studies assessing treatment effects currently do not meet their key assumption of no unmeasured confounding.

Take Home Message: What researchers consider in studies to control for biases and what are potential sources of confounding bias based on clinical considerations and patient perspectives can differ substantially. Even with substantial efforts and a systematic approach, a target trial emulation approach will currently have challenges to provide credible real-world evidence due to serious concerns of confounding bias.

Competing interests: Julian Hirt, Tim Woelfle, Perrine Janiaud, and Lars G. Hemkens declare no competing interests. The authors institution (RC2NB) is supported by Foundation Clinical Neuroimmunology and Neuroscience Basel, unrelated to this work. RC2NB had a contract with Roche for a steering committee participation of LGH, unrelated to this work.