Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Merhof, Dorit

Biomedical Engineering / Biomedizinische Technik

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering and the German Society of Biomaterials

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Habibović, Pamela / Haueisen, Jens / Jahnen-Dechent, Wilhelm / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Leonhardt, Steffen / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Boenick, Ulrich / Jaramaz, Branislav / Kraft, Marc / Lenarz, Thomas / Lenthe, Harry / Lo, Benny / Mainardi, Luca / Micera, Silvestro / Penzel, Thomas / Robitzki, Andrea A. / Schaeffter, Tobias / Snedeker, Jess G. / Sörnmo, Leif / Sugano, Nobuhiko / Werner, Jürgen /

IMPACT FACTOR 2018: 1.007
5-year IMPACT FACTOR: 1.390

CiteScore 2018: 1.24

SCImago Journal Rank (SJR) 2018: 0.282
Source Normalized Impact per Paper (SNIP) 2018: 0.831

See all formats and pricing
More options …
Volume 61, Issue 5


Volume 57 (2012)

Video-based detection of device interaction in the operating room

Max Rockstroh
  • Corresponding author
  • Universität Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstr. 14, D-04103 Leipzig, Germany
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Marco Wittig
  • Universität Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstr. 14, D-04103 Leipzig, Germany
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Stefan Franke
  • Universität Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstr. 14, D-04103 Leipzig, Germany
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
/ Jürgen Meixensberger / Thomas Neumuth
  • Universität Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstr. 14, D-04103 Leipzig, Germany
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2015-12-02 | DOI: https://doi.org/10.1515/bmt-2015-0008


The establishment of modern workflow management technologies requires the integration of dated devices. The extraction of the essential device data and usage time spans is a central requirement for an integrated OR environment. Therefore, methods are required that extract such information from the output provided by older generation devices, namely video signals. We developed a four-level approach for video-based device information extraction. Usually, video streams contain all relevant patient data and device usage information. We propose an approach consisting of defining regions of interest, grabbing video signals, analyzing the signals and storing the data in a centralized and structured location. The analysis considers textual information and graphical visualization. A prototype of the analysis approach was implemented and applied to a neurosurgical case. An evaluation study was conducted to measure the performance of the approach on video recordings of real interventions. Three medical devices were considered: intraoperative ultrasound, neuro-navigation and microscope. Overall, recognition rates for device usage higher than 95% were obtained. The approach is not limited to a single surgical discipline and does not require modification of medical devices. Furthermore, the analysis of microscopic video streams expands the detectable aspects of the surgical workflow beyond the recognition of device usage.

Keywords: computer-assisted surgery; intraoperative monitoring; surgery; surgical process model; workflow


  • [1]

    Agarwal S, Joshi A, Finin T, Yesha Y, Ganous T. A pervasive computing system for the operating room of the future. Mob Netw Appl 2007; 12: 215–228.Google Scholar

  • [2]

    Aggarwal R, Grantcharov T, Moorthy K, et al., An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 2007; 245: 992–999.Google Scholar

  • [3]

    AXIS Communications, AXIS Commun, http://www.axis.com/global/en/products/axis-video-motion-detection/overview (2015). Accessed on 13 November, 2015.

  • [4]

    Bhatia B, Oates T, Xiao Y, Hu P. Real-time identification of operating room state from video. In: Proc 19th Natl Conf Innov Appl Artif Intell – Vol. 2, AAAI Press, 2007; 1761–1766.Google Scholar

  • [5]

    Blum T, Feußner H, Navab N. Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang, T, Navab N, Pluim J, Viergever M, editors. Med Image Comput Comput-Assist Interv – MICCAI 2010, Springer Berlin/Heidelberg, 2010: 400–407.Google Scholar

  • [6]

    Bohn S, Franke S, Burgert O, Meixensberger J, Lindner D. First clinical application of an open standards based OR integration system. Biomed. Eng.-Biomed. Tech. 2011; 56 Supplement 1.Google Scholar

  • [7]

    Bohn S, Franke S, Burgert O, Neumuth T. An integration architecture with centralized control for medical devices in the digital operating room. In: Proc Fifth Int Workshop Syst Archit Comput Assist Interv SACAI 15th Int Conf Med Image Comput Comput Assist Interv, 2012.Google Scholar

  • [8]

    Bouarfa L, Jonker PP, Dankelman J. Discovery of high-level tasks in the operating room. J Biomed Inform 2011; 44: 455–462.Google Scholar

  • [9]

    bytedeco/javacv, GitHub, https://github.com/bytedeco/javacv. (2015). Accessed on 13 November, 2015.

  • [10]

    Cleary K, Chung HY, Mun SK. OR2020 workshop overview: operating room of the future. Int Congr Ser 2004; 1268: 847–852.Google Scholar

  • [11]

    Dosis A, Aggarwal R, Bello F, et al., Synchronized video and motion analysis for the assessment of procedures in the operating theater. Arch Surg Chic Ill 1960 2005; 140: 293–299.Google Scholar

  • [12]

    Dosis A, BelloF, RockallT, et al., ROVIMAS: a software package for assessing surgical skills using the da Vinci telemanipulator system. In: Inf Technol Appl Biomed 2003 4th Int IEEE EMBS Spec Top Conf On, 2003; 326–329.Google Scholar

  • [13]

    DresslerC, Fischer M, Burgert O, Strauß G. Evaluation of a context sensitive system for intra-operative usage of the electronic patient record. Laryngorhinootologie 2011.Google Scholar

  • [14]

    Endress A, Brucker S, Wallwiener D, Aydeniz B, Kurek R, Zubke W. Systems integration in the operating room: the challenge of the decade. Gynecol Surg 2006; 3: 6–11.Google Scholar

  • [15]

    Franke S, Liebmann P, Neumuth T. Connecting workflow management to the OR network: Design and evaluation of a bridge to enable dynamic systems behaviour. Biomed. Eng.-Biomed. Tech. 2012; 57 (Suppl 1).Google Scholar

  • [16]

    Gessat M, Bohn S, Voruganti A, Franke S, Burgert O. TiCoLi: an open software infrastructure for device integration in the digital OR. Int J CARS 2011; 6 (Suppl 1): 226–295.Google Scholar

  • [17]

    Ibach B, Benzko J, Schlichting S, Zimolong A, Radermacher K. Integrating medical devices in the operating room using service-oriented architectures. Biomed. Eng.-Biomed. Tech. 2012; 57: 221–228.Google Scholar

  • [18]

    Lacassagne L, Manzanera A, Dupret A. Motion detection: fast and robust algorithms for embedded systems. 2009, 3265–3268.Google Scholar

  • [19]

    Lalys F, Riffaud L, Bouget D, Jannin P. A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 2012; 59: 966–976.Google Scholar

  • [20]

    Lalys F, Riffaud L, Morandi X, Jannin P. Surgical phases detection from microscope videos by combining SVM and HMM. In: Menze B, Langs G, Tu Z, Criminisi A, editors. Med Comput Vis Recognit Tech Appl Med Imaging, Springer Berlin Heidelberg, 2011: 54–62.Google Scholar

  • [21]

    Lemke HU, Vannier MW. The operating room and the need for an IT infrastructure and standards. Int J Comput Assist Radiol Surg 2006; 1: 117–121.Google Scholar

  • [22]

    Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O. Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 2009; 16: 72–80.Google Scholar

  • [23]

    Neumuth T, Liebmann P, Wiedemann P, Meixensberger J. Surgical workflow management schemata for cataract procedures. Process model-based design and validation of workflow schemata. Methods Inf Med 2012; 51: 371–382.Google Scholar

  • [24]

    OpenCV, OpenCV, http://opencv.org/. (2015). Accessed on 13 November, 2015.

  • [25]

    Panasonic, Panasonic, http://security.panasonic.com/pss/security/products/hd/i-VMD/ (2015). Accessed on 13 November, 2015.

  • [26]

    Piccardi M. Background subtraction techniques: a review. In: 2004 IEEE Int Conf Syst Man Cybern 2004; 4: 3099–3104.Google Scholar

  • [27]

    Ray S. An overview of the tesseract OCR engine. In: Int Conf Conf Doc Anal Recognit 2007; 629–633.Google Scholar

  • [28]

    Rockstroh M, Franke S, Neumuth T. A workflow-driven information source management. Int J Comput Assist Radiol Surg 2013; 8: 189–191.Google Scholar

  • [29]

    Rockstroh M, Franke S, Neumuth T. Requirements for the structured recording of surgical device data in the digital operating room. Int J Comput Assist Radiol Surg 2014; 9: 49–57.Google Scholar

  • [30]

    Speidel S, Zentek T, Sudra G, et al., Recognition of surgical skills using hidden Markov models. In: 2009: 726125– 726125–8.Google Scholar

  • [31]

    Wu J, Lai C, Gupta R, Abosch A, Nelson D. Video-motion detection for objectively quantifying movements in in patients with Parkinson’s disease. Mov Disord 2013; 28: 322.Google Scholar

About the article

Corresponding author: Max Rockstroh, Universität Leipzig, Innovation Center Computer Assisted Surgery, Semmelweisstr. 14, D-04103 Leipzig, Germany, E-mail:

Received: 2015-01-16

Accepted: 2015-10-22

Published Online: 2015-12-02

Published in Print: 2016-10-01

Citation Information: Biomedical Engineering / Biomedizinische Technik, Volume 61, Issue 5, Pages 567–576, ISSN (Online) 1862-278X, ISSN (Print) 0013-5585, DOI: https://doi.org/10.1515/bmt-2015-0008.

Export Citation

©2016 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Maryam Gholinejad, Arjo J. Loeve, and Jenny Dankelman
Minimally Invasive Therapy & Allied Technologies, 2019, Volume 28, Number 2, Page 91

Comments (0)

Please log in or register to comment.
Log in