¹Universität Leipzig, Leipzig, Germany
²Institut für Medizinische Informatik, Statistik und Epidemiologie, Universität Leipzig, Leipzig, Germany

Text

Introduction: Hospitals rely on patient feedback to improve healthcare services. Online reviews have become a crucial resource for understanding patient experiences, offering valuable insights into healthcare services [1], [2]. However, analyzing unstructured text reviews at a detailed level remains challenging, especially in under-resourced languages like German. This research presents an approach to perform Aspect-Based Sentiment Analysis (ABSA) on German hospital reviews, enabling the extraction of specific aspects and assessing their sentiment. By providing hospitals with structured insights from patient reviews, this study contributes to enhancing healthcare quality and patient satisfaction.

Methods: We employed a multi-step methodology. First, we build a dataset of German hospital reviews from the online review platform “klinikbewertungen.de”. After collecting the data, we performed exploratory data analysis to understand its structure and characteristics. To facilitate training and evaluation, we manually labeled a subset of the data with aspect terms, their categories (medical service, general support, hospital facilities, doctor engagement, support personnel, clinical nursing team, others) and corresponding sentiment (positive, neutral, negative). The dataset contains 1,011 labeled sentences from 202 reviews, featuring 1,393 aspect terms with 642 positive, 335 neutral and 416 negative sentiment labels. For Aspect Term Extraction (ATE), we utilized pre-trained transformer-based models to find aspects in the sentences [3]. ATE is split into finding aspects and finding aspects with their corresponding categories. Finally, we performed ABSA using state-of-the-art transformer-based models to classify sentiment polarity for each aspect [3].

Results: Different transformer-based models were fine-tuned for ATE and ABSA on the generated dataset. The models were trained with different epochs (5-8, 10, 12). For the first ATE task the best model (dbmdz BERT cased) achieved Macro-F1-scores of 0.8520 [4]. For category-aware ATE the best model (GBERT-base) achieved 0.8218 [5]. The highest-performing ABSA model (Aaron Chibb’s German Sentiment Model) scored F1-scores up to 0.8550 [5].

Discussion: The results demonstrate that transformer-based models can effectively perform ATE and ABSA on German hospital reviews. The high Macro-F1-scores indicate strong model performance across different aspect categories and sentiment polarities. The best ATE models achieved Macro-F1-scores around 0.85, suggesting it successfully identified relevant aspect terms. When category assignment was included, the best performance slightly decreased to 0.82 indicating that the added complexity of categorization posed challenges.

The ABSA task yielded the highest performance, with a Macro-F1-score up to 0.8550. This suggests that sentiment classification on extracted aspects is more straightforward than aspect extraction. The higher score in ABSA is attributed to the availability of sentiment-related linguistic patterns that the model captured during training. However, variations in performance across different training epochs shows that fine-tuning plays a crucial role in optimizing model accuracy.

Conclusion: This study demonstrates the potential of transformer-based models for ABSA in German hospital reviews, helping medical satisfaction departments extract valuable insights from large datasets. By structuring patient feedback, this approach enables hospitals to systematically assess service quality, identify areas for improvement, and enhance patient care. Future work focuses on optimizing model performance, exploring multilingual approaches, and expanding training datasets to further support data-driven healthcare improvements.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.

Literatur

[1] Bansal A, Kumar N. Aspect-Based Sentiment Analysis Using Attribute Extraction of Hospital Reviews. New Gener Comput. 2022;40:941–960. DOI: 10.1007/s00354-021-00141-3
[2] Zhao Y, Zhang L, Zeng C, Lu W, Chen Y, Fan T. Construction of an aspect-level sentiment analysis model for online medical reviews. Inf Process Manag. 2023;60(6):103513. DOI: 10.1016/j.ipm.2023.103513
[3] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A, Kaiser Ł, Polosukhin I. Attention is all you need. Adv Neural Inf Process Syst (NIPS). 2017;31:6000–6010. DOI: 10.48550/arXiv.1706.03762
[4] Aßenmacher M, Corvonato A, Heumann C. Re-Evaluating GermEval17 Using German Pre-Trained Language Models. In: 6th Swiss Text Analytics Conference (SwissText); 2021 Jun 14-16; Brugg, Switzerland. DOI: 10.48550/arXiv.2102.12330
[5] Chan B, Schweter S, Möller T. German’s Next Language Model. In: Scott D, Bel N, Zong C, editors. Proceedings of the 28th International Conference on Computational Linguistics. Barcelona, Spain; 2020. p. 6788–6796. DOI: 10.18653/v1/2020.coling-main.598