¹Institute for Biostatistics, Hannover Medical School, Hannover, Germany
²European Medicines Agency (EMA), Amsterdam, Netherlands

Text

Introduction: The precise pre-specification of primary analysis methods is essential for controlling the type-I-error (T1E) rate in confirmatory clinical trials at the intended level (typically 5%). Mixed Models for Repeated Measures (MMRM) are frequently applied for analyzing longitudinal data in such trials. However, MMRMs are complex, requiring specific decisions not only on fixed and random effects but also on numerous technical aspects, such as covariance structures, estimation methods or computational algorithms. Prior empirical research shows that these details of MMRMs are poorly specified in study protocols, creating multiple, distinct MMRM analysis approaches that may all be justifiable under the ambiguous specification. In consequence, numerous plausible MMRM pathways introduce a hidden multiplicity problem by inflating the overall probability of falsely rejecting a true null hypothesis (T1E inflation). However, the actual magnitude of the inflation remains unclear, necessitating quantification. Our investigation sought to determine the extent of T1E inflation arising from incomplete or imprecise MMRM specifications.

Methods: A simulation study was performed, mimicking a randomized, double-blind, parallel-group, phase III clinical trial. Data were generated under the null hypothesis of no treatment effect. Multiple clinical scenarios were simulated, varying the total sample size (N=50, 100, 500) and allocation ratio (1:1 or 2:1). For each simulated dataset (10,000 replications per scenario), analyses were performed using defined “clusters” of MMRMs. Each cluster represented a set of specific MMRM analyses compatible with a given level of specification ambiguity. These ranged from a cluster, where none of the technical parameters were specified (allowing choice from 560 MMRM variants), to clusters where only a single model parameter was left unspecified. T1E rate for each cluster was calculated as the proportion of simulations where at least one MMRM variant within the cluster yielded a statistically significant (p<0.05) treatment effect.

Results: Simulations demonstrated that ambiguous MMRM specifications can lead to substantial T1E inflation. When model parameters were left completely unspecified, the observed T1E rate reached a maximum of 7.6% (95% CI: 7.1%–8.1%) in the challenging scenario of a small (N=50) and unbalanced (2:1 allocation ratio) trial, indicating a significant increase over the nominal 5% level. Significant inflation was also observed when only single components like the estimation method (max T1E: 6.3%), covariance structure (max T1E: 5.6%), or hypothesis testing/DDF method (max T1E: 5.9%) were left unspecified. Leaving only the computation method unspecified did not result in significant inflation. The magnitude of inflation generally increased with the number of unspecified items and was more pronounced in trials with smaller sample sizes and unbalanced designs. Even in large (N=500), balanced trials, ambiguity led to slight T1E inflation, although often not statistically significant.

Conclusion: Ambiguous or incomplete pre-specification of MMRMs poses a considerable threat to maintaining the intended T1E control in confirmatory clinical trials. Substantial T1E inflation can compromise the validity of confirmatory results. Even ambiguity in single technical parameters, often considered nuisances, can be problematic, particularly in smaller trials. Complete and unambiguous pre-specification of all MMRM components in study protocols are crucial to safeguard integrity and regulatory acceptability of pivotal clinical trials.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.

Literatur

[1] Häckl S, Koch A, Lasch F. Type-I-error rate inflation in mixed models for repeated measures caused by ambiguous or incomplete model specifications. Pharm Stat. 2023;22(6):1046–61.
[2] Häckl S, Koch A, Lasch F. Empirical evaluation of the implementation of the EMA guideline on missing data in confirmatory clinical trials: Specification of mixed models for longitudinal data in study protocols. Pharm Stat. 2019;18(6):636–44.
[3] International Conference on Harmonisation (ICH). Statistical Principles for Clinical Trials (ICH E9). 1998.
[4] European Medicines Agency (EMA). Points to consider on multiplicity issues in clinical trials discussion in the efficacy working party. 2002.
[5] European Medicines Agency (EMA). Guideline on Missing Data in Confirmatory Clinical Trials. 2011. p.1–12.