Real-world benchmarking of statistical software for feature selection in longitudinal biomedical data

25gmds070 10.3205/25gmds070 urn:nbn:de:0183-25gmds0703 Meeting Abstract Real-world benchmarking of statistical software for feature selection in longitudinal biomedical data Gieswinkel Gieswinkel Alexander A

Preventive Cardiology and Preventive Medicine, Department of Cardiology, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany Institute of Mathematics, Johannes Gutenberg University Mainz, Mainz, Germany German Center for Cardiovascular Research (DZHK), partner site Rhine Main, Mainz, Germany

author Buch Buch Gregor G

author ten Cate ten Cate Vincent V

Preventive Cardiology and Preventive Medicine, Department of Cardiology, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany German Center for Cardiovascular Research (DZHK), partner site Rhine Main, Mainz, Germany Clinical Epidemiology and Systems Medicine, Center for Thrombosis and Hemostasis, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany

author Gül Gül Gökhan G

Preventive Cardiology and Preventive Medicine, Department of Cardiology, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany Clinical Epidemiology and Systems Medicine, Center for Thrombosis and Hemostasis, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany

author Hartung Hartung Lisa L

Institute of Mathematics, Johannes Gutenberg University Mainz, Mainz, Germany

author Wild Wild Philipp P

author German Medical Science GMS Publishing House

Düsseldorf

610 longitudinal data simulation study systematic review statistical software feature selection 20251103 engl This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). M0631 070 Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie 70. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS) V: Machine learning and AI applications 2 Jena 20250907 20250911 Abstr. 246 TextIntroduction: Recent advances in biochemical technology enable increasing availability of high-dimensional omics data for multiple time points in prospective cohort studies. Supervised feature selection is often required in these settings to overcome dimensionality problems and to achieve biomedical interpretability. To date, an overview of existing methods for this framework is lacking. This highlights the need for a systematic review and evaluation of this area.Methods: A systematic search of statistical software was conducted to identify appropriate methods. The Comprehensive R Archive Network (CRAN) was examined via the R package packagefinder using a search query containing relevant keywords . Eligible software was identified by manually screening the package descriptions, and through computational testing with a fixed application example. The inter- and intra-class correlation structure of longitudinal proteomic data was analysed to generate synthetic data for an ADEMP-designed simulation study to support the implication of the findings to real-world cohort data , . Feature selection performance of identified methods was evaluated in real-world scenarios, considering varying sample sizes, total number of predictors and true positives, time points and signal-to-noise ratios. Only frequentist implementations with given default settings were included in the Monte Carlo simulations for a fair comparison. The estimated true positive rate (eTPR) and estimated false discovery rate (eFDR) were chosen as targeted performance measures.Results: Of 21,528 accessible packages on CRAN (status: June 2024), 324 packages with matching keywords in the descriptions were extracted by the search query. Screening of the descriptions identified 45 packages that were then tested in R. Methods for inappropriate settings (N=11), absent variable selection (N=5), not applicable to the predefined testing data (N=4) or other reasons (N=11) were excluded. Six of these remaining 14 methods were based on mixed effects models (buildmer, rpql, splmm, alqrfe, plsmmLasso, glmmLasso), five on generalized estimating equations (sgee, LassoGEE, geeVerse, PGEE, pgee.mixed), two methods were built on Bayesian frameworks (sparsereg, spikeSlabGAM) and one package was modelling time series (midasml). All implementations were able to process continuous outcomes, while only four supported binary outcomes. A total of N=8 frequentist methods with ‘ready-to-use’ default settings were considered in the simulation study.The packages buildmer and pslmmLasso consistently demonstrated an eTPR exceeding 80% while maintaining the eFDR under 20%, across various signal-to-noise settings. By comparison, all other methods underperformed in jointly evaluating both performance metrics. splmm achieved similar eFDR but yielded lower eTPR, whereas geeVerse showed an opposite trend.Discussion: The majority of the available statistical software is based on frequentist techniques, while Bayesian procedures represent a minority. Alternative concepts like tree-based methods are notably absent. There was no evidence of superiority for modern selection techniques such as regularized regression (pslmmLasso) over traditional approaches like stepwise regression (buildmer) for feature selection in longitudinal data.Future analysis will include Bayesian methods in the simulation study, and provide a more comprehensive examination of the results.Conclusion: A variety of statistical software is available for supervised feature selection in longitudinal biomedical data. Among these, methods based on mixed-effects models appear to outperform generalized estimating equations.The authors declare that they have no competing interests.The authors declare that an ethics committee vote is not required.The contribution has already been presented as a poster at 46th Annual Conference of the International Society for Clinical Biostatistics (ISCB): Systematic review and real life-oriented evaluation on methods for feature selection in longitudinal biomedical data Buch G Wild PS Investigating selection strategies for identifying biometrical techniques: a case study on group variable selection methods in R 2023 68. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS). Heilbronn, 17.-21.09.2023 DocAbstr. 303 Buch G, Wild PS. Investigating selection strategies for identifying biometrical techniques: a case study on group variable selection methods in R. In: 68. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS). Heilbronn, 17.-21.09.2023. Düsseldorf: German Medical Science GMS Publishing House; 2023. DocAbstr. 303 DOI: 10.3205/23gmds081 https://doi.org/10.3205/23gmds081 Morris TP White IR Crowther MJ Using simulation studies to evaluate statistical methods 2019 Stat Med 2074-102 Morris TP, White IR, Crowther MJ. Using simulation studies to evaluate statistical methods. Stat Med. 2019;38(11):2074-102. Schulz A Zoller D Nickels S Beutel ME Blettner M Wild PS Simulation of complex data structures for planning of studies with focus on biomarker comparison 2017 BMC Med Res Methodol 90 Schulz A, Zoller D, Nickels S, Beutel ME, Blettner M, Wild PS, et al. Simulation of complex data structures for planning of studies with focus on biomarker comparison. BMC Med Res Methodol. 2017;17(1):90. Fitzmaurice GM Laird NM Ware JH 2011 Applied longitudinal analysis Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis. Second Edition. Hoboken, NJ: Wiley-Interscience; 2011. 0 0 0 0