25dkou008 10.3205/25dkou008 urn:nbn:de:0183-25dkou0081 Meeting Abstract Schaack Schaack Victor V

Technical University of Munich, Human Centered Computing and Extended Reality Lab, TUM University Hospital, Clinic for Orthopedics and Sports Orthopedics, TUM School of Medicine and Health TUM School of Computation, Information, and Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), Munich, Deutschland

author Hong Hong Xuan-Pu Autumn XPA

Technical University of Munich, Human Centered Computing and Extended Reality Lab, TUM University Hospital, Clinic for Orthopedics and Sports Orthopedics, TUM School of Medicine and Health TUM School of Computation, Information, and Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), Munich, Deutschland

author Micheler Micheler Carina C

Technical University of Munich, TUM School of Medicine and Health, Department for Clinical Medicine, Clinic for Orthopedics and Sports Orthopedics, Munich, Deutschland

author von Eisenhart-Rothe von Eisenhart-Rothe Rüdiger R

Technical University of Munich, TUM School of Medicine and Health, Department for Clinical Medicine, Clinic for Orthopedics and Sports Orthopedics, Munich, Deutschland

author Roth Roth Daniel D

Technical University of Munich, Human Centered Computing and Extended Reality Lab, TUM University Hospital, Clinic for Orthopedics and Sports Orthopedics, TUM School of Medicine and Health TUM School of Computation, Information, and Technology, Munich Institute of Robotics and Machine Intelligence (MIRMI), Munich, Deutschland

author German Medical Science GMS Publishing House

Düsseldorf

610 20251031 engl This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. Dieser Artikel ist ein Open-Access-Artikel und steht unter den Lizenzbedingungen der Creative Commons Attribution 4.0 License (Namensnennung). M0634 008 Deutsche Gesellschaft für Orthopädie und Unfallchirurgie Deutsche Gesellschaft für Orthopädie und Orthopädische Chirurgie Deutsche Gesellschaft für Unfallchirurgie Berufsverband für Orthopädie und Unfallchirurgie Deutscher Kongress für Orthopädie und Unfallchirurgie (DKOU 2025) Poster | Digitalisierung Berlin 20251028 20251031 AB11-4375 TextObjectives and questions: Germany is facing an acute and ever-growing lack of medical personnel. This lack of medical professionals is forcing healthcare institutions to schedule less surgeries than their infrastructure can support. This study’s aim is to support the tasks of scrub nurses and assist the surgical workflow through an automated robotic solution. In this dissemination of our preliminary results, we focus on simulating a pipeline to adapt an existing grasping network to train a digital twin to pick up surgical instruments. The system completes three tasks: Detecting the surgical instrument, grasping it and then delivering it to a specified position.Material and methods: We implemented our approach within NVIDIA’s Omniverse, specifically utilizing IsaacSim and IsaacLab to create a virtual testing environment. A digital-twin of the Franka-Emika-Research-3 robot was employed to simulate grasping tasks. We choose this robot to demonstrate how a general-purpose robot can offer an affordable alternative to dedicated surgical robots. High-fidelity 3D scans of real instruments were used to simulate the tools to be picked up. A RGB-D image is fed into the AnyGrasp network to generate multiple gripper-position-candidates. These positions were filtered to exclude grasp-configurations deviating over five degrees from the surface normal of the workplane, enhancing both the likelihood of successful grasping and preventing premature gripper collision with the workplane.Results: Our preliminary simulation results prove that generalised grasping solutions can be implemented to pick up specialised tools like surgery instruments with moderate success. A small validation study of 9 tools returned an overall success rate of 71%, while the grasping network managed to detect the surgical instrument 90% and grasped it 79% of the time. One failure mode was apparent in our testing where one specific instrument would fail to generate a successful gripper position.Discussion and conclusions: We have successfully adapted a pipeline to pick up surgical instruments common in orthopedic surgery. Our preliminary study proves the value of implementing general-purpose robotics in surgical workflows as cost-effective measures to alleviate pressure during surgical procedures. It is vital to continue this line of enquiry to create solutions that can be deployed in real surgeries.Figure 1 Table 1 11

Table 1: Success rates of each task including the object that consistently failed (Nailing-Insertion-Handle) and without it.

1

1 1 Figure 1: Training of 4 instances in parallel in IsaacSim.

1 0 0