Research
Ongoing Research at SPARC
ForNeRo
Human-centred Integration of Surgical Robotics in the OR
The introduction of robotic assistance systems into the clinical workflow leads to a significant increase in technical, social and organizational complexity in the operating room. With ForNeRo, we aim to improve the integration of the systems, taking into account the needs and capacities of the OR staff. This will be realized with support from machine learning, simulation, augmented reality and UI technologies. The focus of the investigations is the optimal placement and use of the robotic systems. (https://fornero.ed.tum.de/, Contact person: Dr. Ramy Zeineldin)
Deformable Object Reconstruction
A Machine Learning Based Alternative to Soft Body Registration
In the context of manipulating deformable objects, such as soft tissue organs, a key challenge is to interact with specific parts of these objects, which are often defined in their non-deformed states. Identifying and locating these parts post-deformation requires a reliable method. Traditionally, this is achieved through deformable object registration, which adapts a digital model to the object’s current deformed shape. However, deformable object registration methods are made for specific objects or deformation types, limiting their adaptability.
Our approach represents a departure from conventional strategies by directly generating segmented 3D models from sensor data, eliminating the need for conventional deformable object registration. This approach constructs a new model that accurately represents the object’s deformed state. The primary advantage of our method lies in its generality and adaptability; it is capable of learning to model any type of deformation specified by the user, offering a unified solution for different objects and deformations.
(Contact Person: Pit Henrich)
The Surgeon’s Eye
Enhancing Minimally Invasive Procedures Through Robotic Assistance
Minimally invasive procedures offer patients a multitude of advantages; however, they present heightened challenges to surgeons compared to conventional open procedures. These techniques necessitate the insertion of surgical instruments through small incisions, restricting direct visualization of the operative field. To address this limitation, a minimally invasive camera, commonly known as an endoscope, becomes indispensable. Operated by an assistant, the endoscope complements the surgeon’s manipulation of surgical instruments. Unfortunately, the ad hoc composition of surgical teams often results in suboptimal collaboration between surgeons and assistants.Introducing the “The surgeon’s eye” project: a pioneering endeavor focused on the development of robot-assisted, minimally invasive surgeries. Central to this innovation is a robotic system that guides the endoscope in accordance with the surgeon’s preferences. This is achieved through an articulate robotic endoscope that ensures consistent and steady visualization of the surgical site. Enhanced by advanced machine learning algorithms, the endoscope intelligently prioritizes focal points, providing the surgeon with an optimal view. (Contact person: Paul Scheikl)
Koala MARS: Multi-Agent Robotic Surgery
Transforming Laparoscopic Surgery: Bridging Skill Shortages with Cognitive Surgical Robots
Modern surgery thrives on collaboration between surgeons and their skilled assistants. However, a shortage of experts, especially in rural areas, poses a challenge. We propose a solution: cognitive surgical robots that automate tasks. Our project employs cooperative multi-agent reinforcement learning (cMARL) to train decentralized artificial agents. These agents unite to tackle robot-assisted laparoscopic procedures, reshaping surgical care and overcoming skill shortages. (Contact Person: Paul Scheikl)
JuBot – Staying young with robots
Robotic Assistance for an Active Lifestyle
In an increasingly aging society, robotics can make a key contribution to the support and care of senior citizens. In this way, the independence and health of people should be maintained and promoted even in old age, and employees in care should be supported. The aim of the project, which is funded by the Carl Zeiss Foundation at KIT, is to develop humanoid assistance robots and wearable exoskeletons that provide personalized, learning assistance to help seniors cope with everyday life. To this end, humanoid assistance robots will take over everyday tasks in human-robot apartments and wearable robots (exoskeletons) will support the personal mobility of elderly people and enable targeted training of motor and cognitive skills.
Within the scope of JuBot, the SPARC team investigates soft actuators as an interface between human extremities and assistive robotics. Their investigation hinges on the exploration of “intelligent materials,” including shape memory alloys and electroactive polymers, which serve as integral components for actuator-sensor synergy. (https://www.jubot.kit.edu/)
Finished Projects
CatHERA
Training of Safe Endovascular Interventions
Endovascular interventions, such as the delicate task of dissolving blood clots, necessitate the utmost precision from medical practitioners. Maneuvering catheters and guidewires within the intricate vascular system demands a profound mastery that can only be achieved through dedicated practice.
Our vision is to empower physicians to navigate this complexity with confidence and finesse, all while mitigating the risk of perilous complications like vascular ruptures. The CatHERA project aims to investigate an intelligent phantom training phantom that provides feedback to medical trainees. This phantom harnesses sensor technology to create an immersive platform for teaching and learning these intricate endovascular techniques involving catheters.
KIRuRC
AI-based registration and robotic-assisted surgery
Minimally invasive laparoscopic surgery offers increased patient safety and reduced healthcare costs. However, surgical robots are only used remotely in laparoscopy. In this project, stereovision methods will be further developed to generate safe 3D point clouds from laparoscopic image data. Reinforcement learning will be used to register a deformable anatomical model of the scene onto the point clouds. This registered model will then be used to perform trajectory planning to perform a surgical sub-step of laparoscopic gallbladder removal. This procedure allows the transparent and interpretable representation of the robotic behavior. Human-machine interfaces will be explored to allow the medical user to interact with the robotic AI system and to visualize the data.