Text

Introduction: Clinical decision support systems (CDSS) can reduce the burden on medical staff in the intensive care unit (ICU). Projects like the IntelliLung initiative aim to develop AI-based CDSS that support the optimization of ventilator settings while reducing clinicians’ workload. User-friendly interfaces are described as key facilitators for the adoption of CDSS in patient ventilation [1], while low usability is considered a barrier to adopting AI-based CDSS [2]. We report a first formative evaluation iteration of a user interface (UI) of an AI-based CDSS that simultaneously generates recommendations for multiple ventilator parameter settings.

Methods: A mockup of the CDSS was formatively evaluated in two group usability walkthroughs, lasting approximately 35 minutes. Each walkthrough comprised task completion, group discussions guided by predefined questions, and five individual Likert-scale ratings. The evaluation focused on overall interaction with the UI and the individual assessments addressed interface suitability, layout clarity, content understandability, and information completeness.

The results were analyzed descriptively. Notes from the group discussions were organized according to the guiding questions and emerging themes.

Results: In the two group usability walkthroughs, four ICU nurses and four physicians – the primary user groups of the CDSS – evaluated the UI. All participants were employed at the same university hospital in Germany. The CDSS UI contained general patient information, a tabular layout for the recommended ventilator parameter settings, a section explaining the output, and a section for viewing trends of important parameters.

The interface was positively evaluated by both user groups. All participants successfully completed the individual tasks, and the majority gave positive ratings regarding the interface’s information completeness, content clarity, screen layout, and suitability for clinical use. Group discussions revealed potential design enhancements, such as enhancing the affordance of interactive elements and introducing a graphical countdown for the next recommendation.

Discussion: These results suggest that the interface structure effectively supports users in distinguishing CDSS recommendations from supplementary information, thereby facilitating information retrieval. This supports the use of a table-based layout to display current and recommended ventilator settings, as applied in the present study and used – though not previously evaluated with users – in related studies [3].

The current study complements the growing body of technically focused literature on AI for mechanical ventilation [4] and contributes to the limited usability research on AI-based CDSS in the ICU setting [5].

While the small sample size is appropriate for early-stage formative evaluations, it limits the generalizability of the findings. Future studies should deploy more elaborate evaluations suitable for later iterations, with larger participant groups from multiple institutions using more advanced prototypes for realistic and interactive testing in authentic environments to enhance the generalizability of the findings.

Conclusion: This study gathered an initial positive assessment from future users of a CDSS interface designed to support ventilator parameter optimization in the ICU. The applied methodology enabled a pragmatic and efficient evaluation of user perceptions, generating actionable insights for further interface development in subsequent user-centered design iterations.

Acknowledgements: Funding: BMFTR, grant number 01ZZ2002. We thank the IntelliLung project for the cooperation.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.

Literatur

[1] Murali M, Ni M, Karbing DS, Rees SE, Komorowski M, Marshall D, Ramnarayan P, Patel BV. Clinical practice, decision-making, and use of clinical decision support systems in invasive mechanical ventilation: a narrative review. Br J Anaesth. 2024 Jul;133(1):164-177. DOI: 10.1016/j.bja.2024.03.011
[2] Kloka JA, Holtmann SC, Nürenberg-Goloub E, Piekarski F, Zacharowski K, Friedrichson B. Expectations of Anesthesiology and Intensive Care Professionals Toward Artificial Intelligence: Observational Study. JMIR Form Res. 2023 Jun 12;7:e43896. DOI: 10.2196/43896
[3] Rees SE, Spadaro S, Dalla Corte F, Dey N, Brohus JB, Scaramuzzo G, Lodahl D, Winding RR, Volta CA, Karbing DS. Transparent decision support for mechanical ventilation using visualization of clinical preferences. Biomed Eng Online. 2022 Jan 24;21(1):5. DOI: 10.1186/s12938-021-00974-5/
[4] Gallifant J, Zhang J, Del Pilar Arias Lopez M, Zhu T, Camporota L, Celi LA, Formenti F. Artificial intelligence for mechanical ventilation: systematic review of design, reporting standards, and bias. Br J Anaesth. 2022 Feb;128(2):343-351. DOI: 10.1016/j.bja.2021.09.025
[5] Wang L, Zhang Z, Wang D, Cao W, Zhou X, Zhang P, Liu J, Fan X, Tian F. Human-centered design and evaluation of AI-empowered clinical decision support systems: a systematic review. Front Comput Sci. 2023;5:1187299. DOI: 10.3389/fcomp.2023.1187299