Tracking and localizing objects is a central problem in computer-assisted surgery. Optical coherence tomography (OCT) can be employed as an optical tracking system, due to its high spatial and temporal resolution. Recently, 3D convolutional neural networks (CNNs) have shown promising performance for pose estimation of a marker object using single volumetric OCT images. While this approach relied on spatial information only, OCT allows for a temporal stream of OCT image volumes capturing the motion of an object at high volumes rates. In this work, we systematically extend 3D CNNs to 4D spatio-temporal CNNs to evaluate the impact of additional temporal information for marker object tracking. Across various architectures, our results demonstrate that using a stream of OCT volumes and employing 4D spatio-temporal convolutions leads to a 30% lower mean absolute error compared to single volume processing with 3D CNNs.
Checklists are a valuable tool to ensure process quality and quality of care. To ensure proper integration in clinical processes, it would be desirable to generate checklists directly from formal process descriptions. Those checklists could also be used for user interaction in context-aware surgical assist systems. We built a tool to automatically convert Business Process Model and Notation (BPMN) process models to checklists displayed as HTML websites. Gateways representing decisions are mapped to checklist items that trigger dynamic content loading based on the placed checkmark. The usability of the resulting system was positively evaluated regarding comprehensibility and end-user friendliness.
In this study, we propose a method for marker detection in X-ray fluoroscopy sequences based on adaptive thresholding and classification. Adaptive thresholding yields multiple marker candidates. To remove non-marker areas, 24 specific features are extracted from each extracted patch and four supervised classifiers are trained to differentiate non-marker areas from marker areas. Quantitative evaluation was carried out to assess different classifier performance by calculating accuracy, sensitivity, specificity and precision. SVM outperforms other classifiers based on the mean value for accuracy, specificity and precision with 81.56, 91.94 and 84.21%, respectively.
The treatment of cerebro- and cardiovascular diseases requires complex and challenging navigation of a catheter. Previous attempts to automate catheter navigation lack the ability to be generalizable. Methods of Deep Reinforcement Learning show promising results and may be the key to automate catheter navigation through the tortuous vascular tree. This work investigates Deep Reinforcement Learning for guidewire manipulation in a complex and rigid vascular model in 2D. The neural network trained by Deep Deterministic Policy Gradients with Hindsight Experience Replay performs well on the low-level control task, however the high-level control of the path planning must be improved further.
Cardiac diseases manifest in a multitude of interconnected changes in morphology and dynamics. Radiomics approaches are a promising technique to analyze such changes directly from image data. We propose novel features to specifically describe moving cardiac structures, and an interactive 4D visualization method to explore such data. Prototypical tests with an open data set containing different diseases show that our approach can be a fast and useful tool for the 4D analysis of heterogeneous cohort data.
Risk classes defined by MDR and FDA for state-of-the-art surgical robots based on their intended use are not suitable as indicators for their hazard potential. While there is a lack of safety regulation for an increasing degree of automation as well as the degree of invasiveness into the patient’s body, adverse events have increased in the last decade. Thus, an outright identification of hazards as part of the risk analysis over the complete development process and life cycle of a surgical robot is crucial, especially when introducing new technologies. For this reason, we present a comprehensive approach for hazard identification in early phases of development. With this multi-perspective approach, the number of hazards identified can be increased. Furthermore, a generic catalogue of hazards for surgical robots has been established by categorising the results. The catalogue serves as a data pool for risk analyses and holds the potential to reduce hazards through safety measures already in the design process before becoming risks for the patient.
Previous research reported catheter pose-dependent virtual angioscopy images for endovascular aortic repair (EVAR) (phantom studies) without any validation with video images. The goal of our study focused on conducting this validation using a video graphics array (VGA) camera. The spatial relationship between the coordinate system of the virtual camera and the VGA camera was computed with a Hand-Eye calibration so that both cameras produced similar images. A re-projection error of 3.18 pixels for the virtual camera and 2.14 pixels for the VGA camera was obtained with a designed three-dimensional (3D) printed chessboard. Similar images of the vessel (3D printed aorta) were acquired with both cameras except for the different depth. Virtual angioscopy images provide information from inside the vessel that may facilitate the understanding of the tip position of the endovascular tools while performing EVAR.
Dysphagia, the difficulty in swallowing, is one of the most common and, at the same time, most heterogeneous symptom of the upper digestive tract. Due to its lifetime prevalence of about 5%, every 19th person is affected on average, especially with increasing age. Dysphagia occurs in both benign and malignant diseases of the esophagus and the oropharyngeal tract as well as in neuromuscular diseases. Even dysphagia caused by benign diseases can lead to significantly reduced quality of life.
The diagnostics of the actual underlying disease in patients with dysphagia is commonly conducted using a combination of endoscopy, esophageal manometry, functional assessments and radiologic means, e.g. X-ray-fluoroscopy. As these examinations are typically performed in sequential order, it remains to the physicians to combine the relevant information from each modality to form a conclusion. We argue that this is neither an intuitive, nor a standardized form of presenting the findings to the physician. To address this, we propose a novel approach for fusing time-synchronized manometric and X-ray data into a single view to provide a more comprehensive visualization method as a novel means for diagnosing dysphagia.
Injuries to the biliary tree during surgical, endoscopic or invasive radiological diagnostic or therapeutic procedures involving the pancreas, liver or organs of the upper gastrointestinal tract give rise to the need to develop a method for clear discrimination of biliary anatomy from surrounding tissue. Hyperspectral imaging (HSI) is an emerging optical technique in disease diagnosis and image-guided surgery with inherent advantages of being a non-contact, non-invasive, and non-ionizing technique. HSI can produce quantitative diagnostic information about tissue pathology, morphology, and chemical composition. HSI was applied in human liver transplantation and compared to porcine model operations to assess the capability of discriminating biliary anatomy from surrounding biological tissue. Absorbance spectra measured from bile ducts, gall bladder, and liver show a dependence on tissue composition and bile concentration, with agreement between human and porcine datasets. The bile pigment biliverdin and structural proteins collagen and elastin were identified as contributors to the bile duct and gall bladder absorbance spectra.