German Congress of Orthopaedics and Traumatology (DKOU 2025)
Preliminary study: Simulation of an adapted gripper pipeline for orthopedic surgery instruments
2Technical University of Munich, TUM School of Medicine and Health, Department for Clinical Medicine, Clinic for Orthopedics and Sports Orthopedics, Munich, Deutschland
Text
Objectives and questions: Germany is facing an acute and ever-growing lack of medical personnel. This lack of medical professionals is forcing healthcare institutions to schedule less surgeries than their infrastructure can support. This study’s aim is to support the tasks of scrub nurses and assist the surgical workflow through an automated robotic solution. In this dissemination of our preliminary results, we focus on simulating a pipeline to adapt an existing grasping network to train a digital twin to pick up surgical instruments. The system completes three tasks: Detecting the surgical instrument, grasping it and then delivering it to a specified position.
Material and methods: We implemented our approach within NVIDIA’s Omniverse, specifically utilizing IsaacSim and IsaacLab to create a virtual testing environment. A digital-twin of the Franka-Emika-Research-3 robot was employed to simulate grasping tasks. We choose this robot to demonstrate how a general-purpose robot can offer an affordable alternative to dedicated surgical robots. High-fidelity 3D scans of real instruments were used to simulate the tools to be picked up. A RGB-D image is fed into the AnyGrasp network to generate multiple gripper-position-candidates. These positions were filtered to exclude grasp-configurations deviating over five degrees from the surface normal of the workplane, enhancing both the likelihood of successful grasping and preventing premature gripper collision with the workplane.
Results: Our preliminary simulation results prove that generalised grasping solutions can be implemented to pick up specialised tools like surgery instruments with moderate success. A small validation study of 9 tools returned an overall success rate of 71%, while the grasping network managed to detect the surgical instrument 90% and grasped it 79% of the time. One failure mode was apparent in our testing where one specific instrument would fail to generate a successful gripper position.
Discussion and conclusions: We have successfully adapted a pipeline to pick up surgical instruments common in orthopedic surgery. Our preliminary study proves the value of implementing general-purpose robotics in surgical workflows as cost-effective measures to alleviate pressure during surgical procedures. It is vital to continue this line of enquiry to create solutions that can be deployed in real surgeries.
Figure 1 [Abb. 1]
Figure 1: Training of 4 instances in parallel in IsaacSim.
Table 1 [Tab. 1]
Table 1: Success rates of each task including the object that consistently failed (Nailing-Insertion-Handle) and without it.
