Text

Background: Objective Structured Clinical Examinations (OSCEs) are a widely established format in medical education for assessing clinical competence but are often resource-intensive. Virtual reality (VR) in OSCEs may offer a scalable alternative [1]; however, there are concerns that VR competence – the ability to navigate and operate VR systems – affects assessment outcomes [2]. The present study investigated whether general VR competence, among other factors, predicts performance in a VR-based OSCE.

Methods: In a quantitative study, N=68 medical students in the clinical study phase at a German medical faculty completed a standardized VR emergency OSCE scenario, covering diagnostic assessment, decision-making, and initial treatment steps. Clinical competence was scored using structured checklists. Beforehand, participants answered a 20-item single choice declarative knowledge test on emergency medicine and some sociodemographic items (gender, clinical semester, prior clinical experience, gaming experience). VR competence was assessed using a validated VR application consisting of eight levels designed to measure competence in the domains of navigation, UI activation, and object manipulation. Data were analyzed using multiple stepwise linear regression with OSCE performance as the dependent variable and VR competence, prior knowledge, and demographic variables as predictors.

Results: The mean OSCE score was 12.93 (SD=3.94) out of 25.5 points, suggesting moderate difficulty (p=.51). Several OSCE checklist items demonstrated good to very good corrected item-total correlations (r≥.30), indicating that the station was able to differentiate between higher- and lower-performing students. Internal consistency of the OSCE checklist was acceptable (Cronbach’s α=.66), considering the heterogeneous nature of the assessed clinical tasks.

The final regression model explained 10% (adjusted R²=.10) of the variance in OSCE performance, indicating a small effect of basic VR hardware handling skills (Level 1 of VR application) as the only significant predictor (β=.25, p=.050). Prior knowledge and demographics were non-significant and explained little to no variance in OSCE performance. More complex VR interaction skills were not associated with higher OSCE performance.

Discussion: In this study, general VR competence (apart from hardware handling), prior declarative knowledge, and demographics played a minor role for VR OSCE performance. The negligible influence of these factors supports the construct validity of the VR-OSCE as an assessment format for clinical competence. However, the predictive value of hardware handling suggests that some training with the VR equipment prior to taking part in the VR-OSCE is warranted.

Take-home messages: VR-based OSCEs present a suitable format to assess clinical competence. Examinees should receive standardized training in VR hardware handling prior to assessment to ensure equal preconditions.

Literatur