Predicting Surgical Phases using CNN-NARX Neural Network

Nour Aldeen Jalal 1 , Tamer Abdulbaki Alshirbaji 2 ,  and Knut Möller 3
  • 1 Institute of Technical Medicine, Furtwangen University, Jakob-Kienzle-Straße 17,, Villingen-Schwenningen, Germany
  • 2 Institute of Technical Medicine, Furtwangen University,, Villingen-Schwenningen, Germany
  • 3 Institute of Technical Medicine, Furtwangen University,, Villingen-Schwenningen, Germany


Online recognition of surgical phases is essential to develop systems able to effectively conceive the workflow and provide relevant information to surgical staff during surgical procedures. These systems, known as context-aware system (CAS), are designed to assist surgeons, improve scheduling efficiency of operating rooms (ORs) and surgical team and promote a comprehensive perception and awareness of the OR. State-of-the-art studies for recognizing surgical phases have made use of data from different sources such as videos or binary usage signals from surgical tools. In this work, we propose a deep learning pipeline, namely a convolutional neural network (CNN) and a nonlinear autoregressive network with exogenous inputs (NARX), designed to predict surgical phases from laparoscopic videos. A convolutional neural network (CNN) is used to perform the tool classification task by automatically learning visual features from laparoscopic videos. The output of the CNN, which represents binary usage signals of surgical tools, is provided to a NARX neural network that performs a multistep-ahead predictions of surgical phases. Surgical phase prediction performance of the proposed pipeline was evaluated on a dataset of 80 cholecystectomy videos (Cholec80 dataset). Results show that the NARX model provides a good modelling of the temporal dependencies between surgical phases. However, more input signals are needed to improve the recognition accuracy.

If the inline PDF is not rendering correctly, you can download the PDF file here.


Journal + Issues

Current Directions in Biomedical Engineering is an open access journal and closely related to the journal Biomedical Engineering - Biomedizinische Technik. CDBME is a forum for the exchange of knowledge in the fields of biomedical engineering, medical information technology and biotechnology/bioengineering for medicine and addresses engineers, natural scientists, and clinicians working in research, industry, or clinical practice.