Skip to content
BY-NC-ND 4.0 license Open Access Published by De Gruyter April 20, 2017

Surgical data science: the new knowledge domain

  • S. Swaroop Vedula and Gregory D. Hager EMAIL logo

Abstract

Healthcare in general, and surgery/interventional care in particular, is evolving through rapid advances in technology and increasing complexity of care, with the goal of maximizing the quality and value of care. Whereas innovations in diagnostic and therapeutic technologies have driven past improvements in the quality of surgical care, future transformation in care will be enabled by data. Conventional methodologies, such as registry studies, are limited in their scope for discovery and research, extent and complexity of data, breadth of analytical techniques, and translation or integration of research findings into patient care. We foresee the emergence of surgical/interventional data science (SDS) as a key element to addressing these limitations and creating a sustainable path toward evidence-based improvement of interventional healthcare pathways. SDS will create tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and use information gained to inform healthcare decisions, guidelines, best practices, policy, and training, thereby improving the safety and quality of healthcare and its value. Data are pervasive throughout the surgical care pathway; thus, SDS can impact various aspects of care, including prevention, diagnosis, intervention, or postoperative recovery. The existing literature already provides preliminary results, suggesting how a data science approach to surgical decision-making could more accurately predict severe complications using complex data from preoperative, intraoperative, and postoperative contexts, how it could support intraoperative decision-making using both existing knowledge and continuous data streams throughout the surgical care pathway, and how it could enable effective collaboration between human care providers and intelligent technologies. In addition, SDS is poised to play a central role in surgical education, for example, through objective assessments, automated virtual coaching, and robot-assisted active learning of surgical skill. However, the potential for transforming surgical care and training through SDS may only be realized through a cultural shift that not only institutionalizes technology to seamlessly capture data but also assimilates individuals with expertise in data science into clinical research teams. Furthermore, collaboration with industry partners from the inception of the discovery process promotes optimal design of data products as well as their efficient translation and commercialization. As surgery continues to evolve through advances in technology that enhance delivery of care, SDS represents a new knowledge domain to engineer surgical care of the future.

Introduction

More than ever, surgery is an indispensable part of modern healthcare. More than 312 million major surgical procedures were performed globally in 2012, an estimated increase of 38.2% since 2004 [1]. An additional 288 million humans in low-resource countries suffer from a surgically treatable condition, and studies estimated that, within this population, 5.6 million deaths could be prevented with quality surgical care [2]. At the same time, a considerable fraction of adverse outcomes in healthcare are associated with complications from surgical interventions [3]. Surgical complications are most often due to errors in judgment or technique or other forms of system and communication errors [4], [5], [6], [7], [8]. Errors lead to complications in care, adverse outcomes for patients, and a substantial impact on the quality and value of healthcare [3].

The surgical profession has consistently promoted a culture of continuous improvement of performance through evaluation of errors and their consequences and enhancements in the safety and quality of care [9], [10]. Historically, the quality of surgical care was assured through training competent surgeons and controlled through systematic monitoring of outcomes of care. More recently, methods such as registry studies have begun to supplement traditional mechanisms such as mortality and morbidity conferences [10]. This reflects a general trend to make surgical care evidence-based and to more carefully monitor the outcomes of surgical care. However, surgical patient registries typically rely on a limited amount of data about patients, which in turn restricts the scope of possible discoveries to improve the quality of care [11]. Similarly, surgical training has evolved through structured residency programs, graduated delegation of responsibility under supervision, and competency assessment before certification for practice. Despite these advances, the acquisition and retention of surgical skill and competency are still largely supported by unstructured, inconsistent feedback and assessed using methods that are subjective, mostly based on manual observation, and thus resource intensive [12], [13].

As these two examples suggest, traditional methodologies to support improvements in the quality and value of surgical care are no longer adequate for several reasons. First, surgery is growing in complexity not only because of multiple comorbidities in typical surgical patients but also due to various technologies and information sources that augment care. In addition, these technologies, together with several other sensors, yield large amounts of complex, unstructured data (e.g. radiographic and video images). Such data describe individuals, instruments, devices, technologies, the environment, and their interactions throughout the patient care pathway. Information from this deluge of data may enable providers to deliver safer and higher-quality care if the data are captured, modeled, and transformed into products that integrate into clinical workflows. However, conventional methods are limited in the nature and scope of questions they address, data that are used to develop analytics, and the extent to which products are integrated into patient care workflows.

The confluence of routine availability of large amounts of complex, unstructured data, the advances in analytical techniques, and the need to maximize the quality and value of surgical care in an increasingly complex system set the stage for data-driven discoveries and insights to further transform surgery through a data science approach. Data science, in general, refers to a “study of generalizable extraction of knowledge from data” [14]. Data science has become an integral part of several scientific disciplines, including clinical medicine. As surgical disciplines continue to improve the quality and value of care through technology and evidence, data science will likewise enable their evolution through new integrative knowledge. However, surgical/interventional data science (SDS) is still in its nascent stages, with a recently held international workshop marking the first initiative to build a global scientific community focused on this area (http://www.surgical-data-science.org; accessed February 13, 2017). This workshop formed the basis for an initial consensus on the definition and scope for SDS along with key clinical applications and anticipated challenges, all of which are described in [15]. The current article expands on those ideas by articulating some of the opportunities and challenges in bringing data science approaches to the practice of surgery.

Aims and relevance of SDS

SDS aims to enable evidence-based improvement of interventional healthcare pathways by creating tools to measure, model, and quantify the pathways or processes within the context of patient health states or outcomes and using the information gained to inform healthcare delivery, decisions, evidence, best practices, policy, and training, thereby improving the quality and value of healthcare [15], [16]. A data science approach to improving value in healthcare shares commonalities across clinical domains that involve providing care through bodily invasion, through natural orifices or artificially inflicted wounds, for either diagnostic or therapeutic purposes. Examples of such clinical domains include all surgical disciplines, radiotherapy, interventional radiology, and interventional cardiology. SDS aims to leverage scientific principles, research methodology, and techniques from several disciplines related to data capture, curation, and analysis, including various branches of engineering, the physical sciences, computer science, epidemiology, and statistics. Although the methods developed and employed in SDS build on existing in other areas of biomedical data science context, the evaluation of human performance and its impact on patient outcomes is a particularly unique and challenging aspect of SDS.

SDS is distinct from other methodologies such as translational research and implementation science, but it is synergistic with them. Translational research aims to promote effective clinical use, i.e. “bench to bedside” transfer, of new knowledge, techniques, or technologies [17]. On the contrary, implementation science refers to the “methods to promote the systematic uptake of clinical research findings and other evidence-based practices into routine practice [18].” SDS shares with these methodologies a common goal to improve the quality and value of healthcare. However, it diverges from the other methodologies in its scope, research questions, and approach. SDS may address research questions related to processes and technologies that span the entire surgical patient care pathway. SDS emphasizes the identification of appropriate data sources, engineering solutions to seamlessly capture data in a busy healthcare environment, techniques to efficiently curate the data, and effective methodologies to analyze complex data and discover actionable insights.

Research scope for SDS

Figure 1 illustrates our conceptual view of how SDS relates to surgical patient care. Whereas concepts shown in Figure 1 may be applicable to healthcare pathways in general, our emphasis in this article is on care that involves intervention through bodily invasion. The surgical care pathway is a complex interplay between various care providers, such as surgeons, nurses, and other personnel providing supportive care, and the healthcare system or environment. Numerous techniques and technologies to measure disease and deliver care serve to augment human performance. Data are pervasive throughout these pathways from various sources such as observations by care providers and measurements of human and technological performance, variables that describe the patient, clinical decisions, interventions, or the process of care. SDS is focused on harvesting these data, curating it, and developing analytics and data products, which in turn improve the quality and value of care. Thus, the research scope for SDS may be considered under a few broad conceptual categories: (1) capture and curation of data, (2) analytics to transform existing quality improvement methodologies, (3) analytics to inform and improve surgical care processes, (4) analytics to enable intelligent collaboration between care providers and technology, and (5) analytics to augment learning and performance of care providers. Although these conceptual categories may be considered essential areas of focus for SDS, some key clinical applications for SDS are discussed in [15] based on consensus within the scientific community. These categories are further described below in the context of examples from existing research efforts, which serve to illustrate the scope for SDS without attempting to exhaustively catalog existing research in this area.

Figure 1: SDS spans the complete patient care pathway with the objective of maximizing the quality and value of care by augmenting care providers and enhancing technology; it involves seamless, efficient capture, secure storage, and efficient curation of data regardless of their complexity as well as analytics to (1) sustain continuous quality improvement, (2) augment the performance of human care providers as well as that of technologies used for diagnosis and therapy, (3) enable intelligent assistance and collaboration between care providers and technology, and (4) support effective and efficient training for care providers.
Figure 1:

SDS spans the complete patient care pathway with the objective of maximizing the quality and value of care by augmenting care providers and enhancing technology; it involves seamless, efficient capture, secure storage, and efficient curation of data regardless of their complexity as well as analytics to (1) sustain continuous quality improvement, (2) augment the performance of human care providers as well as that of technologies used for diagnosis and therapy, (3) enable intelligent assistance and collaboration between care providers and technology, and (4) support effective and efficient training for care providers.

Examples and potential applications

Data acquisition and curation

Data science relies on data; the systematic capture and curation of data are thus a primary focus for research in SDS. Conventional sources capture only a fraction of data available in the clinical context. Historically, the medical chart served as the primary repository of data on the patient and the care they receive. However, the complexity of surgical care and the data that it generates have rapidly outpaced the evolution of the medical chart and its successor, the electronic medical record (EMR). Consequently, the EMR may serve as an effective data repository to support clinical care, but it remains an incomplete record of data generated by the care process itself. Patient surveys are another conventional data source, but they are severely constrained in the nature and extent of data they can capture. A broader data science approach would leverage all available data and enable a learning system to support surgical care.

The efficient acquisition and curation of available data from diverse sources is vital to facilitate a learning healthcare system [19]. Data in the surgical context, as is the case for any healthcare domain, are pervasive, extremely diverse in its sources, formats, structure, consistency with which it is observed, ease with which it may be accessed, and the extent to which it captures information that can support patient care. For example, patient descriptors that are typically captured during clinical history-taking may be recorded within structured fields in a repository or as free text within a caregiver’s narrative. Other data may be as diverse as patients’ responses to structured surveys, numeric observations of variables at specific time-points, or time-series values of hemodynamic variables from intravascular catheters [20]. High-dimensional data may also be obtained from wearable sensors on the behavior of patients, care providers, or instruments and technology used to deliver care, such as tool motion, or images of varying complexity obtained through endoscopic visualization or radiologic investigations. Although these data are readily available, the challenge lies in acquiring it consistently, completely, and without interfering with patient care. This challenge is compounded by heterogeneity in societal and systemic aspects such as availability of resources, cultural perspectives toward research and data in healthcare, and the clinical context. For example, data from the health record may be routinely captured in some countries and clinical contexts but not in others. Thus, solutions to enable systematic data acquisition for SDS should be scalable across diverse settings.

Systems to integrate diverse sensors within and beyond the operating room are needed to facilitate seamless data capture in real time [21]. The heterogeneity in data available within the surgical care context requires a correspondingly diverse array of sensors to capture them. Whereas some sensors are integrated within technology or devices used to provide care, others must be affixed or embedded in the surgical environment to enable data capture. For example, endoscopic video images may be readily captured using routinely available systems, but data on instrument usage require extraneous placement of sensors or tracking systems. Furthermore, existing approaches to capture data in the surgical context target discrete sources within disparate systems as opposed to networks of sensors. Although sensing networks are routinely employed in other scientific disciplines such as environmental studies, agriculture, geology, and oceanography [22], such systems have yet to become a part of surgical care. Although comprehensive systems, including wireless sensor networks and interconnected medical devices, have been prototyped for data capture in the operating room [23], [24] (OR.net; http://www.ornet.org; accessed February 13, 2017), such networks have yet to be developed for deployment at scale.

Finally, curation of heterogeneous data in the surgical context will rely on the adoption of community-wide standards. Such standards are easier to apply within structured research such as clinical trials [25], but a broader consensus is needed to advance SDS. The disparate data in routine surgical care interactions necessitate a broad approach to integration that includes common ontologies [26], [27], industry-wide data standards for technologies that use and emanate data such as imaging [28], consensus on interoperability of data from EMRs and other repositories [29], and a variety of data curation, normalization, and wrangling techniques [30], [31], [32].

The following research questions can inform a systematic approach to conception, design, and development of systems for acquisition and methods for curation of data for SDS:

  1. How may systems be developed to facilitate data capture for SDS that are scalable across settings that may have varying amounts of resources and heterogeneous workflows?

  2. What technological challenges need to be overcome to develop surgical data capture systems that integrate multimodal data from a variety of sensors and devices?

  3. What are consensus standards for how different types of surgical data are curated to create shared databases?

Analytics to transform existing quality improvement methodologies

Traditionally, quality improvement in surgery has relied on the systematic measurement of outcomes through patient registries [11], [33]. Such registries have not only allowed the monitoring of temporal trends in patient outcomes but also enabled analytics to predict patients’ risk of outcomes after surgery, for example, risk of surgical site infections (SSIs) [34], readmission [35], and other surgical complications [33], [36]. However, registry-based predictive analytics either have poor performance during independent validation [34] or have little generalizability across clinical contexts [37], [38] or across data sources [39]. Furthermore, registry-based analytics rely on cross-sectional observations on a limited subset, for example, preoperative patient descriptors, of all available data, while excluding data on intraoperative and postoperative variables that affect patient outcomes [40], [41]. In the context of surgical and other invasive interventional care, quality of care, and consequently quality improvement, is invariably affected by the interventional care process. Thus, capturing data on intraoperative care processes and modeling it is essential to develop valid analytics to support improvement in the quality of surgical care. Finally, current registry-based predictive analytics are also limited in utility because they do not encompass process-level determinants of patient outcomes. A broadly based data science approach to quality improvement through predictive analytics would have to address all these limitations. Thus, the following research questions inform the development of analytics that transform existing quality improvement methodologies:

  1. What are key measures of surgical care quality should be targeted through a data science approach to have a substantial impact on value in care?

  2. How may improvements in surgical care quality be driven using multimodal, complex data, including measures of intraoperative care and performance, and advanced analytics?

  3. How may data science techniques transform patient registries from simple databases into interactive resources, which accept complex multimodal data and provide accurate and individualized predictions, to support surgical care in real time?

Predicting risk of SSIs illustrates how a data science approach may augment decision-making through accurate information. SSIs are routinely anticipated after surgery, and they have a substantial impact on the value of healthcare. In one study using a national representative sample of in-patient admissions in the United States, SSIs led to an estimated $1.6 billion in additional costs from nearly 1 million additional in-patient days [42]. Furthermore, none of the existing quality improvement strategies targeting SSIs were found to be clearly effective [43], [44]. Although predicting patients’ risk of SSI may facilitate more effective preventive care, conventional analytics only yield a low to moderate accuracy (c-statistics of 0.53 and 0.62) [34], [37]. These accuracies are affected by the heterogeneity in patients’ risk of SSI and failure of conventional analytics to leverage intraoperative and postoperative data in addition to preoperative patient descriptors [40], [45]. On the contrary, models that included serial postoperative observations of the wound or laboratory tests, together with corresponding machine learning techniques, achieved up to 90% prediction accuracy [46], [47]. Although these findings have yet to be validated in independent data-sets with sufficient patient heterogeneity, they suggest that the effectiveness of interventions to prevent SSIs could be improved through accurate predictive analytics using new sources of data and approaches to data analytics.

Analytics to inform and improve surgical care processes

Surgical care is a process, so it follows that patient outcomes and value may be improved by optimizing the process and through other process-level interventions [48]. Conventional approaches to improve surgical processes have emphasized individual elements. For example, in the case of SSIs, the Surgical Care Improvement Project (SCIP) evaluated the effectiveness of quantifying and reporting process-level measures of care [49], [50]. Simply capturing and reporting discrete process measures such as the use of prophylactic antibiotics, optimization of blood glucose control, surgical site preparation, or effective management of postoperative fever was not associated with a reduction in the incidence of SSIs [50]. This suggests that data-driven approaches to surgical processes that are overly constrained in their scope, data sources, and analytics may not be effective for improving the quality and value of care.

A data science approach to modeling surgical care processes would take a broad methodological perspective and bring new powerful tools to bear to maximize the quality and value of care. These include analytics to optimize processes and to enable better situational decision-making and context awareness. Although models for surgical care processes will vary in structure and complexity depending on the specific clinical context and target applications, nearly any SDS approach to process improvement will have to address one or more of the following questions:

  1. How may surgical processes be characterized to be consistent with expert knowledge?

  2. What data are required to model surgical processes and how may that data be efficiently captured?

  3. Which aspects of surgical processes are consequential to its outcomes?

  4. How may analytics be developed to model variations in surgical processes and associate them with the quality and value of care?

  5. How may scalable methods be developed to optimize surgical processes, their outcomes, their efficiency, and their resource usage?

  6. How may surgical process models (SPMs) be used to develop data-driven context awareness?

Although the overall surgical care pathway may be considered a process, most existing researches on SPMs have focused on tasks or phases within the intraoperative surgical procedure. SPMs can facilitate process-level analytics to optimize care, enable data-driven context awareness, and support technology to provide intelligent assistance to care providers and decision-makers. Furthermore, process-level variation may be associated with patient outcomes [51]. Thus, modeling variation in processes and contributing factors may facilitate improvements in the quality and value of care. SPMs also will be key to developing cyber-physical systems that seamlessly integrate the dynamics of intraoperative surgical processes with information and assistance through technology and interactions among care providers as well as human-computer interactions. Such integrated systems are possible with full automation of data capture and curation at scale and SPMs that are generalizable across surgical procedures, platforms, and contexts.

The current literature on the methodologies for SPM is extensively discussed elsewhere [52], [53], [54], [55], [56]. Overall, the methodology for SPMs involves specifying an ontology for surgical processes, data acquisition using a variety of sensors and/or observer-based methods including manual annotations about the action, instrument, actor, and anatomical structure being manipulated, modeling of the data using statistical and machine learning methods, and validation of such models [52], [57], [58]. SPMs have been developed for procedures in general surgery, ophthalmology, otolaryngology-head and neck surgery, oral and maxillofacial surgery, neurosurgery, urology, and trauma surgery, among others [54]. However, the broader acceptance and use of SPMs to address the research questions listed above is still lacking.

Analytics to enable intelligent collaboration between care providers and technology

Currently, technology passively enables the provision of surgical care. SDS can transform surgical technologies into an interactive platform that can collaborate with and actively assist providers. Such collaboration may take the form of (1) automated performance of activities during a procedure, (2) guidance in performing activities either by physical interaction with surgical tools or surgeons’ hands or by augmentation of information, or (3) timely and accurate decision support. Context awareness is key to enable intelligent collaboration between care providers and technology within patient care pathways [59]. SDS can enable such advances through analytics for context awareness, i.e. recognizing what surgical activity is being performed on what tissue, or in which context, to achieve which objective and with how much skill. Such awareness then facilitates analytics to determine how technology can assist with tool or tissue manipulation; to continuously evaluate the status of the patient, devices, surgical tools, and environment; to determine what information is necessary for effective decision-making; to deploy analytics to extract the information from data; and to display the information in a way that seamlessly integrates with the patient care workflow. In addition, SDS can facilitate the design and development of new technologies or devices and their integration into patient care workflows.

Several examples illustrate the potential for data analytics to transform surgical technology to support patient care through collaboration with providers. Technology to automate performance of some surgical activities under supervision is in its nascent stages, with the current work focused on automation of simple tasks such as suturing and knot-tying [59], [60]. Enhanced technologies that either guide or inform surgeons include augmented visualization to guide the operating surgeon or robot-assisted navigation [61], [62], [63], [64], process or workflow analyses for resource management [65], context-aware medical devices [66], surgical trajectory planning [67], automated control of environment in the operating room [68], simulation of SPMs [55], and other virtual reality and cognitive tools for surgical planning [69], [70]. Data analytics that enable technology for such context-aware assistance to surgical care providers are reviewed in detail elsewhere [71].

SDS can also support the continuous monitoring of the status of the patient, surgical tools, devices, and environment [72]. Analytics that rely on streaming data allow for continuous monitoring of patient status. For example, mapping high-dimensional data from electroencephalograms into a lower-dimensional space using established methods allows for reliable characterization of conscious and unconscious states in patients [73]. Similarly, features extracted from time-series data streams of measures describing heart rate variability and the electrocardiogram may be used to accurately detect hemodynamic instability before decompensation [20]. Tracking patient motion may be necessary in some contexts that demand surgical precision or when surgery results in extensive surface motion, such as in external eye surgery [74]. At a granular level, tracking changes in tissue during surgery is more challenging with deformable soft tissue [75], [76] than with rigid anatomical structures such as the paranasal sinuses [77]. Beyond the patient, several methods to estimate motion or changes in pose of surgical instruments using video images and/or kinematics have been developed [78], for example, in minimally invasive surgery [79], [80], open surgery [81], microsurgery [82], endoscopy [83], bronchoscopy [84], and laser surgery [85]. Relative positions and interaction between surgical instruments may be recognized and tracked using sensors such as radiofrequency identification [86] or using video images [87], [88].

In addition, SDS can facilitate collaboration between technology and care providers through context-aware support for decision-making throughout the surgical care pathway. Examples of a data science approach to support decision support in surgical care include predicting the need for intensive care after major surgery [89], risk of highly consequential postoperative complications such as SSIs [46], [47], readmission after cardiac surgery [90], and reoperation or death after surgery for aortic aneurysm repair [91]. Predictive analytics in these studies relied on preoperative patient descriptors, intraoperative data, and postoperative variables (e.g. in-hospital step count). There are many opportunities for decision-support analytics using complex data [92]. Examples include machine learning algorithms applied to (1) reflectance spectrometry to predict healing time after burn injury and assist with surgical planning, (2) images to assess tissue flap viability and tissue reconstruction, (3) computed tomographic scans to determine the nature of craniosynostosis, and (4) characteristics of tissue engineering materials to predict success with various tissue grafts. Such analytics represent a distillation of consequential information from otherwise complex data streams to enhance the accuracy of decisions by care providers.

Finally, data-driven context awareness can enhance the timeliness of analytics that support surgical decision-making. Throughout the surgical care pathway, and particularly in the operating room, care providers constantly integrate information from an array of technologies, devices and sensors to make timely and accurate decisions. Analytics using a data science approach can greatly minimize information complexity and improve the accuracy of decisions in such contexts using larger amounts of data than what may be processed by humans. For example, März et al. described a holistic system using heterogeneous data from multiple sources, including literature-based evidence, institution-specific variables, and all patient-specific information before, during, and after surgery [93].

Research on analytics to enable intelligent collaboration between surgical care providers and technology may be informed by the following questions:

  1. What are key target applications where collaboration with context-aware technology can have a significant impact on the safety and quality of surgical care?

  2. How may the impact of enhanced context-aware technology be evaluated in terms of its effect on patient outcomes, care workflows, provider efficiency, and value in care?

Analytics to augment learning of care providers

Care providers must be competent for surgical/interventional care to be effective. To deliver competent surgical and interventional care, providers must acquire both technical and nontechnical skills, including decision-making, situation awareness, communication, and teamwork [94]. Poor technical skill is associated with severe adverse outcomes after surgical intervention such as reoperation, readmission, and death [95], and acquisition of nontechnical skill is associated with a lower incidence of technical errors [96]. Historically, surgical training, assessment, certification, and continuous improvement have remained the cornerstones of ensuring the quality of surgical care [12]. Traditional models of teaching surgical skills through supervision and demonstration as well as evaluation by observation and opinion are rapidly becoming untenable. The conventional approach to teach surgical skill in the operating room is severely constrained by opportunities limited due to safety and resource concerns [12], [13]. Furthermore, subjectivity in evaluations and inconsistency in teaching opportunities and feedback impair trainees’ ability to learn through deliberate practice. Although data on surgical performance are readily available, its utility in training requires systematic capture, curation, and development of valid analytics.

SDS can yield analytics to equip surgeons with timely, efficient, objective, automated assessments and feedback. Methods for objective assessment rely on data captured using sensors placed on surgical tools, the surgeons, or those integrated within technology used to deliver care. Such data may be captured with greater ease in some surgical platforms such as robotic surgery and in the training laboratory, e.g. virtual reality, than in the operating room [97]. In a simple approach to assess technical skill, kinematic data may be used to compute a variety of metrics describing surgical tool motion that are reviewed in detail elsewhere [98]. Alternatively, data are preprocessed and transformed into an intermediate representation using methods to maximize information pertinent to technical skill. Preprocessed or transformed data are then modeled using appropriate machine learning techniques to derive objective measures of surgical skill. Current methodologies and modeling techniques for such objective computer-aided technical skill evaluation (OCASE-T) in the training laboratory and in the operating room are summarized in detail elsewhere [97]. Although nontechnical skills may also be objectively assessed, capturing and curating the necessary data is challenging and modeling it has been less explored than technical skill.

A data science approach can also enable deliberate practice through an automated surgical coaching framework [99]. Comprehensive coaching by expert surgeons has been shown to be effective for technical skill acquisition [100]. In addition to skill assessment, coaching for technical skill involves the detection of errors and providing targeted feedback [8]. Automating coaching relies on data analytics that provide context awareness in the form of surgical activity recognition, objective skill assessment within activity segments, detection of errors in performance, and methods to determine and deliver appropriate targeted feedback. At the level of surgical procedures, modeling surgical processes and deviations from expert models or their association with patient outcomes serve as a form of evaluation [51], [52]. At a more granular level, surgical tasks or phases may be further decomposed into activities such as maneuvers and gestures [101]. Thus, targeted feedback based on OCASE-T and error detection relies on the automatic recognition of surgical activities such as maneuvers and gestures. Current methods to detect surgical activities using tool motion or video images are summarized in detail in Ref. [102]. Automated coaching systems that integrate activity recognition, objective skill assessment, error detection, and targeted feedback have thus far been developed only in virtual reality [99]. Finally, SDS can support automated coaching systems developed to include robot-assisted active learning (RAAL) of technical skill. RAAL involves robotic guidance through haptics, visual, and/or auditory feedback [103], [104]. In the context of RAAL, data analytics model expert motion, continuously assess robot-assisted performance or learning, and detect deviation from expert models to inform robotic guidance provided to the learners.

In summary, the potential for SDS to augment learning in care providers can be realized through research addressing the following questions:

  1. What data are required to objectively assess surgical technical and nontechnical skills?

  2. How may data analytics be developed to support scalable methods for the objective and accurate assessment of surgical technical and nontechnical skills?

  3. How may data analytics augment deliberate practice for technical skill acquisition through granular assessments, detection of errors in performance, provision of targeted qualitative feedback, as well as demonstrations?

  4. To what extent can surgical coaching be automated through data analytics that enable objective assessments, deliberate practice, and context awareness?

  5. What data and analytics are necessary to support intelligent assistive technologies, including robotics, with which surgical trainees actively interact to efficiently and effectively acquire technical and nontechnical skills?

  6. How may the impact of data analytics to support acquisition surgical technical and nontechnical skills be evaluated in terms of their impact on outcomes of patient care?

Future directions and research gaps

Advances in surgical care through SDS may be realized through a data-centric cultural shift. This is possible through technology to enable pervasive, seamless capture of data without interfering with patient care and active engagement of care providers, patients, and researchers. In addition, data analytics should aim to incorporate multimodal data to maximize accuracy and timeliness while emphasizing clarity, quality, and accessibility of information derived from the data to ensure their acceptability. The full potential for SDS may be achieved when context-specific key clinical applications are identified and updated through a dynamic process and engagement among stakeholders across a broad spectrum of scientific disciplines, roles within healthcare, and clinical settings.

Active engagement of relevant clinical and technical stakeholders is essential not only to promote the data-centric cultural shift necessary to sustain SDS but also to identify and prioritize clinical problems that must be solved. Such engagement may be operationalized by supporting research communities focused on SDS and by fostering interaction and collaboration between care providers and researchers in the academia and the industry. Although the scope for SDS may be broadly discussed in terms of data capture and curation, quality improvement, surgical care processes, improving technology to intelligently collaborate with providers, and improving care providers, advances in some areas are more challenging than others depending on context and application. For example, developing intelligent robotic surgical assistants or fully automated surgical coaches for complex surgical procedures will require extensive innovation. On the contrary, analytics to support quality improvement, intelligent decision-support, or objective assessment of surgical skill may be achievable in the short term with moderate effort and with significant impact on patient care and provider training. In another example, surgical techniques such as robotic surgery pose lower barriers for advances through SDS because multiple modes of data are readily available that are much harder to capture with other techniques such as open or laparoscopic surgery. Given that complexities in problems and potential solutions span clinical and technical domains, an actionable agenda for SDS along with key applications may only be generated with input from both clinical and technical stakeholders.

As with any approach to improve patient care and training using healthcare data, SDS is associated with risks from the loss of individual privacy and data confidentiality. Although such risks are associated with any methodological approach using healthcare data, SDS may pose greater risk because of the wide scope and diversity of data that are captured and used. Thus, technology to capture data must adequately address concerns related to privacy, confidentiality, and security within systematic protocol-driven investigations subjected to standard ethics review. Similarly, technology that results from a data science approach and validated through systematic investigation should be vetted through existing regulatory processes as indicated before widespread use in the community.

Uniform standards are required to ensure the utility of heterogeneous data accessible throughout the surgical care pathway and scalability of data products. Although previous efforts have focused on the interoperability of EMRs [29], a data science approach requires uniformity on a wider scale, including ontologies to describe and encode human insights into various aspects of surgical care processes. Such uniformity can facilitate integrating human insights into data-driven technologies through crowdsourcing and other techniques. In addition, methodologies to substitute different types of data can greatly facilitate the relevance and scalability of data products developed through SDS. For example, substituting kinematic data, which may not be easy to capture in all contexts, with more readily accessible video data expands the utility of analytics developed using the former [105].

Analytics based on process models should be scoped to model the entire surgical care pathway. As already noted above, surgery or interventional processes are one part of a more extensive pathway that may include substantial diagnostic, recuperative, and rehabilitative elements. Capturing and relating these data to the intervention itself may involve gathering data from nontraditional sources and new technologies. Creating clinically actionable knowledge from this data will require the application of new methods in data analytics.

Clinical translation of data products and analytics developed through SDS involves engineering transparent, trustable data analytics and systems, systematic cross-validation, and evaluation of effects using established methodologies. Analytics developed through SDS typically rely on complex high-dimensional data that are transformed into features and then modeled. Engineering features that reflect the human understanding of processes and mechanisms underlying the clinical problem can facilitate the acceptability of data products, though sometimes at the cost of overall performance. Although it is hard to explain how accurate predictions are obtained with some analytical techniques, such as neural networks, the features used as input to such models may be engineered to reflect known concepts about the clinical problem. For example, Ahmidi et al. extracted features from unstructured surgical tool motion data that conceptualize expert surgeons’ understanding of the critical task of flap elevation during nasal septoplasty [106]. These features may thus be effective for feedback because they explain why surgeons are assessed to be an expert or a novice.

In addition to the validation of analytics developed through a data science approach, the effect of incorporating the data product or analytics into the process of surgical care needs to be empirically evaluated. Several methodologies exist for the purpose of clinically translating technologies such as cohort studies, interrupted time-series studies, or randomized controlled trials. For example, the anticipated average effect of instituting predictive analytics for decision support or data-driven treatment algorithms on patient outcomes may be evaluated through randomized controlled comparisons. Furthermore, a data science approach to validation and translation within cohort studies emphasizes the identification, measurement, and modeling of factors affecting variation in the effect of integrating data products into patient care pathways. Insights into factors affecting variation in effects across clinical contexts through interrupted time-series studies enable locally optimizing care using globally developed data solutions.

Finally, industry-academia collaborations from inception through commercialization can ensure efficient product development and effective clinical translation of data products through SDS. Several nontraditional industry players such as IBM and Alphabet are joining traditional companies that are beginning to investigate data-enabled products that will support some of the healthcare advances outlined above. Startups are also beginning to field systems for capturing process data in the operating room (e.g. Sigma Surgical; http://www.sigmasurgical.com), to teach cognitive skills for surgery through simulation (e.g. Touch Surgery; http://www.touchsurgery.com), and to develop crowd-sourced data analytics for skill assessment (C-SATS, Inc.; http://www.csats.com). Further enabling and establishing a synergistic ecosystem of industry platforms and standards, which both enable and build on academic basic, applied, and clinical research, will greatly accelerate the progress and adoption of data science within interventional healthcare pathways.

Conclusion

Data will play a central and growing role in achieving quality and value in surgical care. Data will also become increasingly complex for conventional study designs and statistical methodologies. Integrating techniques from multiple scientific disciplines will be necessary to harness data and to discover insights that reshape surgical knowledge. Several examples illustrate the potential for SDS in patient care and surgical training. However, a data-centric cultural shift is necessary to effectively integrate SDS into surgical patient care and training.

Acknowledgments

We acknowledge Dr. Narges Ahmidi for her assistance with editing the figure.

Author Statement

  1. Research funding: Both authors are supported by NIH grant 1R01-DE025265 and internal funds from The Malone Center for Engineering in Healthcare, The Johns Hopkins University. Conflict of interest: Authors state no conflict of interest. Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animals use.

Author Contributions

  1. S. Swaroop Vedula: Conceptualization; Data curation; Funding acquisition; Methodology; Visualization; Writing (original draft); Writing (review and editing). Gregory D. Hager: Conceptualization; Funding acquisition; Methodology; Supervision; Visualization; Writing (original draft); Writing (review and editing).

Publication Funding

  1. The German Society of Surgery funded the article processing charges of this article.

References

[1] Weiser TG, Haynes AB, Molina G, et al. Size and distribution of the global volume of surgery in 2012. Bull World Health Organ 2016;94:201.10.2471/BLT.15.159293Search in Google Scholar

[2] Gupta S, Groen RS, Kyamanywa P, et al. Surgical care needs of low-resource populations: an estimate of the prevalence of surgically treatable conditions and avoidable deaths in 48 countries. Lancet 2015;385:S1.10.1016/S0140-6736(15)60796-6Search in Google Scholar

[3] Institute of Medicine. To err is human: building a safer health system [Internet]; 1999. Available at: https://www.nap.edu/catalog/9728/to-err-is-human-building-a-safer-health-system. Accessed December 28, 2016.Search in Google Scholar

[4] Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery 2003;133:614–621.10.1067/msy.2003.169Search in Google Scholar PubMed

[5] Rogers Jr SO, Gawande AA, Kwaan M, et al. Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery 2006;140:25–33.10.1016/j.surg.2006.01.008Search in Google Scholar PubMed

[6] Regenbogen SE, Greenberg CC, Studdert DM, Lipsitz SR, Zinner MJ, Gawande AA. Patterns of technical error among surgical malpractice claims: an analysis of strategies to prevent injury to surgical patients. Ann Surg 2007;246:705–711.10.1097/SLA.0b013e31815865f8Search in Google Scholar PubMed

[7] Fabri PJ, Zayas-Castro JL. Human error, not communication and systems, underlies surgical complications. Surgery 2008;144:557–565.10.1016/j.surg.2008.06.011Search in Google Scholar PubMed

[8] DaRosa DA, Pugh CM. Error training: missing link in surgical education. Surgery 2012;151:139–145.10.1016/j.surg.2011.08.008Search in Google Scholar PubMed

[9] Krizek TJ. Surgical error: ethical issues of adverse events. Arch Surg 2000;135:1359–1366.10.1001/archsurg.135.11.1359Search in Google Scholar PubMed

[10] Bosk CL. Forgive and Remember: Managing Medical Failure. 2nd ed. Chicago: University of Chicago Press; 2003.10.7208/chicago/9780226924687.001.0001Search in Google Scholar

[11] Lyu H, Cooper M, Patel K, Daniel M, Makary MA. Prevalence and data transparency of national clinical registries in the United States. J Healthc Qual 2016;38:223–234.10.1097/JHQ.0000000000000001Search in Google Scholar PubMed

[12] Bell Jr RH. Why Johnny cannot operate. Surgery 2009;146: 533–542.10.1016/j.surg.2009.06.044Search in Google Scholar PubMed

[13] Snyder RA, Tarpley MJ, Tarpley JL, Davidson M, Brophy C, Dattilo JB. Teaching in the operating room: results of a national survey. J Surg Educ 2012;69:643–649.10.1016/j.jsurg.2012.06.007Search in Google Scholar PubMed

[14] Dhar V. Data science and prediction. Commun ACM 2013;56:64–73.10.1145/2500499Search in Google Scholar

[15] Maier-Hein L, Vedula S, Speidel S, et al. Surgical data science: enabling next-generation surgery. arXiv: 1701.06482 2017. Available at: https://arxiv.org/abs/1701.06482. Accessed February 13, 2017.Search in Google Scholar

[16] Vedula SS, Ishii M, Hager GD. Perspectives on surgical data science. arXiv: 161004276 2016. Available at: http://arxiv.org/abs/1610.04276. Accessed December 28, 2016.Search in Google Scholar

[17] Woolf SH. The meaning of translational research and why it matters. J Am Med Assoc 2008;299:211–213.10.1001/jama.2007.26Search in Google Scholar PubMed

[18] Lobb R, Colditz GA. Implementation science and its application to population health. Annu Rev Public Health 2013;34:235–251.10.1146/annurev-publhealth-031912-114444Search in Google Scholar PubMed PubMed Central

[19] Krumholz HM, Terry SF, Waldstreicher J. Data acquisition, curation, and use for a continuously learning health system. J Am Med Assoc 2016;316:1669–1670.10.1001/jama.2016.12537Search in Google Scholar PubMed

[20] Belle A, Ansari S, Spadafore M, et al. A signal processing approach for detection of hemodynamic instability before decompensation. PLoS One 2016;11:e0148544.10.1371/journal.pone.0148544Search in Google Scholar PubMed PubMed Central

[21] Zhao X. Acquisition, storage and reconstruction of multidimensional surgical information in a digital operation room environment [Internet]. Virginia Commonwealth University; 2005. Available at: http://search.proquest.com/dissertations/docview/305348961. Accessed December 28, 2016.Search in Google Scholar

[22] Puccinelli D, Haenggi M. Wireless sensor networks: applications and challenges of ubiquitous sensing. IEEE Circuits Syst Mag 2005;5:19–31.10.1109/MCAS.2005.1507522Search in Google Scholar

[23] Shnayder V, Chen B, Lorincz K, Fulford-Jones TRF, Welsh M. Sensor networks for medical care 2005. Available at: https://dash.harvard.edu/handle/1/24829604. Accessed December 28, 2016.Search in Google Scholar

[24] Nouei MT, Kamyad AV, Soroush AR, Ghazalbash S. A comprehensive operating room information system using the Kinect sensors and RFID. J Clin Monit Comput 2015;29:251–261.10.1007/s10877-014-9591-5Search in Google Scholar PubMed

[25] Sheehan J, Hirschfeld S, Foster E, et al. Improving the value of clinical research through the use of common data elements. Clin Trials 2016;13:671–676.10.1177/1740774516653238Search in Google Scholar PubMed PubMed Central

[26] Liyanage H, Krause P, De Lusignan S. Using ontologies to improve semantic interoperability in health data. J Innov Health Inform 2015;22:309–315.10.14236/jhi.v22i2.159Search in Google Scholar PubMed

[27] Matney SA, Settergren T, Carrington JM, Richesson RL, Sheide A, Westra BL. Standardizing physiologic assessment data to enable big data analytics. West J Nurs Res 2016;193945916659471. PMID: 27435084.10.1177/0193945916659471Search in Google Scholar PubMed

[28] Wang KC, Kohli M, Carrino JA. Technology standards in imaging: a practical overview. J Am Coll Radiol 2014;11:1251–1259.10.1016/j.jacr.2014.09.014Search in Google Scholar PubMed

[29] Office of the National Coordinator for Health Information Technology, Department of Health and Human Services. Health information technology: initial set of standards, implementation specifications, and certification criteria for electronic health record technology. Interim final rule. Fed Regist 2010;75:2013–2047.Search in Google Scholar

[30] Ohno-Machado L. Structuring text and standardizing data for clinical and population health applications. J Am Med Inform Assoc 2014;21:763–763.10.1136/amiajnl-2014-003171Search in Google Scholar PubMed PubMed Central

[31] Padula WV, Blackshaw L, Brindle CT, Volchenboum SL. An approach to acquiring, normalizing, and managing EHR data from a clinical data repository for studying pressure ulcer outcomes. J Wound Ostomy Cont Nurs Off Publ Wound Ostomy Cont Nurses Soc 2016;43:39–45.10.1097/WON.0000000000000185Search in Google Scholar PubMed

[32] De Veaux RD. Curriculum guidelines for undergraduate programs in data science. Annu Rev Stat Appl 2017;4:15–30.10.1146/annurev-statistics-060116-053930Search in Google Scholar

[33] Bilimoria KY, Liu Y, Paruch JL, et al. Development and evaluation of the universal ACS NSQIP Surgical Risk Calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 2013;217:833–842.e3.10.1016/j.jamcollsurg.2013.07.385Search in Google Scholar PubMed PubMed Central

[34] Mitchell TO, Holihan JL, Askenasy EP, et al. Do risk calculators accurately predict surgical site occurrences? J Surg Res 2016;203:56–63.10.1016/j.jss.2016.03.040Search in Google Scholar PubMed

[35] Fanari Z, Elliott D, Russo CA, Kolm P, Weintraub WS. Predicting readmission risk following coronary artery bypass surgery at the time of admission. Cardiovasc Revasc Med. 2017;18:95–99.10.1016/j.carrev.2016.10.012Search in Google Scholar PubMed PubMed Central

[36] Kraemer K, Cohen ME, Liu Y, et al. Development and evaluation of the American College of Surgeons NSQIP pediatric surgical risk calculator. J Am Coll Surg 2016;223:685–693.10.1016/j.jamcollsurg.2016.08.542Search in Google Scholar PubMed

[37] Bergquist JR, Thiels CA, Etzioni DA, Habermann EB, Cima RR. Failure of colorectal surgical site infection predictive models applied to an independent dataset: do they add value or just confusion? J Am Coll Surg 2016;222:431–438.10.1016/j.jamcollsurg.2015.12.034Search in Google Scholar PubMed

[38] Arce K, Moore EJ, Lohse CM, Reiland MD, Yetzer JG, Ettinger KS. The American College of Surgeons National Surgical Quality Improvement Program Surgical Risk Calculator does not accurately predict risk of 30-day complications among patients undergoing microvascular head and neck reconstruction. J Oral Maxillofac Surg 2016;74:1850–1858.10.1016/j.joms.2016.02.024Search in Google Scholar PubMed

[39] Pasquali SK, He X, Jacobs JP, et al. Measuring hospital performance in congenital heart surgery: administrative versus clinical registry data. Ann Thorac Surg 2015;99:932–938.10.1016/j.athoracsur.2014.10.069Search in Google Scholar PubMed PubMed Central

[40] Bohnen JD, Mavros MN, Ramly EP, et al. Intraoperative adverse events in abdominal surgery: what happens in the operating room does not stay in the operating room. Ann Surg 2016. [Epub ahead of print]. Available at: http://journals.lww.com/annalsofsurgery/Abstract/publishahead/Intraoperative_Adverse_Events_in_Abdominal.96344.aspx. Accessed December 28, 2016.Search in Google Scholar

[41] Morris MS, Graham LA, Richman JS, et al. Postoperative 30-day readmission: time to focus on what happens outside the hospital. Ann Surg 2016;264:621–631.10.1097/SLA.0000000000001855Search in Google Scholar PubMed

[42] de Lissovoy G, Fraeman K, Hutchins V, Murphy D, Song D, Vaughn BB. Surgical site infection: incidence and impact on hospital utilization and treatment costs. Am J Infect Control 2009;37:387–397.10.1016/j.ajic.2008.12.010Search in Google Scholar PubMed

[43] Shojania KG, McDonald KM, Wachter RM, Owens DK, editors. Closing the quality gap: a critical analysis of quality improvement strategies (Vol. 1: series overview and methodology) [Internet]. Rockville, MD: Agency for Healthcare Research and Quality; 2004 (AHRQ Technical Reviews). Available at: http://www.ncbi.nlm.nih.gov/books/NBK43908/. Accessed December 28, 2016.Search in Google Scholar

[44] Garner BH, Anderson DJ. Surgical site infections: an update. Infect Dis Clin North Am 2016;30:909–929.10.1016/j.idc.2016.07.010Search in Google Scholar PubMed

[45] Lawson EH, Hall BL, Ko CY. Risk factors for superficial vs deep/organ-space surgical site infections: implications for quality improvement initiatives. J Am Med Assoc Surg 2013;148:849–858.10.1001/jamasurg.2013.2925Search in Google Scholar PubMed

[46] Soguero-Ruiz C, Fei WME, Jenssen R, et al. Data-driven temporal prediction of surgical site infection. AMIA Annu Symp Proc 2015;2015:1164–1173.Search in Google Scholar

[47] Ke C, Jin Y, Evans H, et al. Prognostics of surgical site infections using dynamic health data. J Biomed Inform.2016;65:22–33.10.1016/j.jbi.2016.10.021Search in Google Scholar PubMed

[48] Loor G, Shumway SJ, McCurry KR, et al. Process improvement in thoracic donor organ procurement: implementation of a donor assessment checklist. Ann Thorac Surg 2016;102:1872–1877.10.1016/j.athoracsur.2016.06.083Search in Google Scholar PubMed

[49] Clancy CM. SCIP: making complications of surgery the exception rather than the rule. AORN J 2008;87:621–624.10.1016/j.aorn.2008.02.007Search in Google Scholar PubMed

[50] Stulberg JJ, Delaney CP, Neuhauser DV, Aron DC, Fu P, Koroukian SM. Adherence to Surgical Care Improvement Project measures and the association with postoperative infections. J Am Med Assoc 2010;303:2479–2485.10.1001/jama.2010.841Search in Google Scholar PubMed

[51] Schumann S, Bühligen U, Neumuth T. Outcome quality assessment by surgical process compliance measures in laparoscopic surgery. Artif Intell Med 2015;63:85–90.10.1016/j.artmed.2014.10.008Search in Google Scholar PubMed

[52] Neumuth D, Loebe F, Herre H, Neumuth T. Modeling surgical processes: a four-level translational approach. Artif Intell Med 2011;51:147–161.10.1016/j.artmed.2010.12.003Search in Google Scholar PubMed

[53] Neumuth T, Loebe F, Jannin P. Similarity metrics for surgical process models. Artif Intell Med 2012;54:15–27.10.1016/j.artmed.2011.10.001Search in Google Scholar PubMed

[54] Lalys F, Jannin P. Surgical process modelling: a review. Int J Comput Assist Radiol Surg 2014;9:495–511.10.1007/s11548-013-0940-5Search in Google Scholar PubMed

[55] Claude G, Gouranton V, Caillaud B, Gibaud B, Arnaldi B, Jannin P. Synthesis and simulation of surgical process models. Stud Health Technol Inform 2016;220:63–70.Search in Google Scholar

[56] Rojas E, Munoz-Gama J, Sepúlveda M, Capurro D. Process mining in healthcare: a literature review. J Biomed Inform 2016;61:224–236.10.1016/j.jbi.2016.04.007Search in Google Scholar PubMed

[57] Katic D, Schuck J, Wekerle A-L, et al. Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy. Int J Comput Assist Radiol Surg 2016;11: 881–888.10.1007/s11548-016-1379-2Search in Google Scholar PubMed

[58] Dergachyova O, Bouget D, Huaulmé A, Morandi X, Jannin P. Automatic data-driven real-time segmentation and recognition of surgical workflow. Int J Comput Assist Radiol Surg 2016;11:1081–1089.10.1007/s11548-016-1371-xSearch in Google Scholar PubMed

[59] Knoll A, Mayer H, Staub C, Bauernschmitt R. Selective automation and skill transfer in medical robotics: a demonstration on surgical knot-tying. Int J Med Robot Comput Assist Surg MRCAS 2012;8:384–397.10.1002/rcs.1419Search in Google Scholar PubMed

[60] Shademan A, Decker RS, Opfermann JD, Leonard S, Krieger A, Kim PCW. Supervised autonomous robotic soft tissue surgery. Sci Transl Med 2016;8:337ra64.10.1126/scitranslmed.aad9398Search in Google Scholar PubMed

[61] Lavallee S, Troccaz J, Gaborit L, Cinquin P, Benabid AL, Hoffmann D. Image guided operating robot: a clinical application in stereotactic neurosurgery. In: Proceedings 1992 IEEE International Conference on Robotics and Automation; 1992:618–624 vol.1.Search in Google Scholar

[62] Novotny PM, Kettler DT, Jordan P, Dupont PE, Nido PJ del, Howe RD. Stereo display of 3D ultrasound images for surgical robot guidance. In: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society; 2006:1509–1512.10.1109/IEMBS.2006.259486Search in Google Scholar PubMed

[63] Su L-M, Vagvolgyi BP, Agarwal R, Reiley CE, Taylor RH, Hager GD. Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 2009;73:896–900.10.1016/j.urology.2008.11.040Search in Google Scholar PubMed

[64] März K, Franz AM, Seitel A, et al. MITK-US: real-time ultrasound support within MITK. Int J Comput Assist Radiol Surg 2014;9:411–420.10.1007/s11548-013-0962-zSearch in Google Scholar PubMed

[65] Maktabi M, Neumuth T. Online time and resource management based on surgical workflow time series analysis. Int J Comput Assist Radiol Surg 2017;12:325–338.10.1007/s11548-016-1474-4Search in Google Scholar PubMed

[66] Franke S, Neumuth T. Rule-based medical device adaptation for the digital operating room. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2015:1733–1736.10.1109/EMBC.2015.7318712Search in Google Scholar PubMed

[67] Dorileo É, Albakri A, Zemiti N, Poignet P. Simplified adaptive path planning for percutaneous needle insertions. In: 2015 IEEE International Conference on Robotics and Automation (ICRA); 2015:1782–1788.10.1109/ICRA.2015.7139429Search in Google Scholar

[68] Choi DG, Yi BJ, Kim W. Automation of surgical illumination system using robot and ultrasonic sensor. In: 2007 International Conference on Mechatronics and Automation; 2007:1062–1066.10.1109/ICMA.2007.4303695Search in Google Scholar

[69] Schoch N, Philipp P, Weller T, et al. Cognitive tools pipeline for assistance of mitral valve surgery. Proc SPIE 2016;9786:978603.10.1117/12.2216059Search in Google Scholar

[70] Navab N, Fellow M, Hennersperger C, Frisch B, Fürst B. Personalized, relevance-based multimodal robotic imaging and augmented reality for computer assisted interventions. Med Image Anal 2016;33:64–71.10.1016/j.media.2016.06.021Search in Google Scholar PubMed

[71] Kassahun Y, Yu B, Tibebu AT, et al. Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int J Comput Assist Radiol Surg 2015;11:553–568.10.1007/s11548-015-1305-zSearch in Google Scholar PubMed

[72] Kenngott HG, Wagner M, Preukschas AA, Müller-Stich BP. Intelligent operating room suite: From passive medical devices to the self-thinking cognitive surgical assistant. Chir Z Alle Geb Oper Medizen 2016;87:1033–1038.10.1007/s00104-016-0308-9Search in Google Scholar PubMed

[73] Mirsadeghi M, Behnam H, Shalbaf R, Moghadam HJ. Characterizing awake and anesthetized states using a dimensionality reduction method. J Med Syst 2016;40:13.10.1007/s10916-015-0382-4Search in Google Scholar PubMed

[74] Haidegger T, Benyo Z, Kazanzides P. Patient motion tracking in the presence of measurement errors. In: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society; 2009:5563–5566.10.1109/IEMBS.2009.5333737Search in Google Scholar PubMed

[75] Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang GZ. Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. Med Image Comput Comput Assist Interv 2005;8:139–146.10.1007/11566489_18Search in Google Scholar PubMed

[76] Mountney P, Lo B, Thiemjarus S, Stoyanov D, Yang GZ. A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery. Med Image Comput Comput Assist Interv 2007;10:34–41.10.1007/978-3-540-75759-7_5Search in Google Scholar PubMed

[77] Otake Y, Leonard S, Reiter A, et al. Rendering-based video-CT registration with physical constraints for image-guided endoscopic sinus surgery. Proc SPIE Int Soc Opt Eng 2015;9415.10.1117/12.2081732Search in Google Scholar PubMed PubMed Central

[78] Bouget D, Allan M, Stoyanov D, Jannin P. Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med Image Anal 2017;35:633–654.10.1016/j.media.2016.09.003Search in Google Scholar PubMed

[79] Voros S, Long J-A, Cinquin P. Automatic localization of laparoscopic instruments for the visual servoing of an endoscopic camera holder. Med Image Comput Comput Assist Interv 2006;9:535–542.10.1007/11866565_66Search in Google Scholar PubMed

[80] Reiter A, Allen PK, Zhao T. Feature classification for tracking articulated surgical tools. Med Image Comput Comput Assist Interv 2012;15:592–600.10.1007/978-3-642-33418-4_73Search in Google Scholar PubMed

[81] Frasier LL, Azari DP, Ma Y, et al. A marker-less technique for measuring kinematics in the operating room. Surgery 2016;160:1400–1413.10.1016/j.surg.2016.05.004Search in Google Scholar PubMed PubMed Central

[82] Alsheakhali M, Eslami A, Roodaki H, Navab N. CRF-based model for instrument detection and pose estimation in retinal microsurgery. Comput Math Methods Med 2016;2016:1067509.10.1155/2016/1067509Search in Google Scholar PubMed PubMed Central

[83] Oda M, Kondo H, Kitasaka T, et al. Robust colonoscope tracking method for colon deformations utilizing coarse-to-fine correspondence findings. Int J Comput Assist Radiol Surg 2017;12:39–50.10.1007/s11548-016-1456-6Search in Google Scholar PubMed

[84] Deligianni F, Chung A, Yang GZ. Predictive camera tracking for bronchoscope simulation with condensation. Med Image Comput Comput Assist Interv 2005;8:910–916.10.1007/11566465_112Search in Google Scholar PubMed

[85] Schoob A, Laves M-H, Kahrs LA, Ortmaier T. Soft tissue motion tracking with application to tablet-based incision planning in laser surgery. Int J Comput Assist Radiol Surg 2016;11:2325–2337.10.1007/s11548-016-1420-5Search in Google Scholar PubMed

[86] Neumuth T, Meißner C. Online recognition of surgical instruments by information fusion. Int J Comput Assist Radiol Surg 2012;7:297–304.10.1007/s11548-011-0662-5Search in Google Scholar PubMed

[87] Richa R, Balicki M, Sznitman R, Meisner E, Taylor R, Hager G. Vision-based proximity detection in retinal surgery. IEEE Trans Biomed Eng 2012;59:2291–2301.10.1109/TBME.2012.2202903Search in Google Scholar PubMed PubMed Central

[88] Rockstroh M, Wittig M, Franke S, Meixensberger J, Neumuth T. Video-based detection of device interaction in the operating room. Biomed Tech (Berl) 2016;61:567–576.10.1515/bmt-2015-0008Search in Google Scholar PubMed

[89] Betten J, Roness AK, Endreseth BH, et al. Assessment of the time-dependent need for stay in a high dependency unit (HDU) after major surgery by using data from an anesthesia information management system. J Clin Monit Comput 2016;30:235–241.10.1007/s10877-015-9707-6Search in Google Scholar PubMed

[90] Takahashi T, Kumamaru M, Jenkins S, Saitoh M, Morisawa T, Matsuda H. In-patient step count predicts re-hospitalization after cardiac surgery. J Cardiol 2015;66:286–291.10.1016/j.jjcc.2015.01.006Search in Google Scholar PubMed

[91] Karthikesalingam A, Attallah O, Ma X, et al. An artificial neural network stratifies the risks of reintervention and mortality after endovascular aneurysm repair: a retrospective observational study. PLoS One 2015;10:e0129024.10.1371/journal.pone.0129024Search in Google Scholar PubMed PubMed Central

[92] Kanevsky J, Corban J, Gaster R, Kanevsky A, Lin S, Gilardino M. Big data and machine learning in plastic surgery: a new frontier in surgical innovation. Plast Reconstr Surg 2016;137:890e–897e.10.1097/PRS.0000000000002088Search in Google Scholar PubMed

[93] März K, Hafezi M, Weller T, et al. Toward knowledge-based liver surgery: holistic information processing for surgical decision support. Int J Comput Assist Radiol Surg 2015; 10:749–759.10.1007/s11548-015-1187-0Search in Google Scholar PubMed

[94] Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 2006;139:140–149.10.1016/j.surg.2005.06.017Search in Google Scholar PubMed

[95] Birkmeyer JD, Finks JF, O’Reilly A, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med 2013;369:1434–1442.10.1056/NEJMsa1300625Search in Google Scholar PubMed

[96] Mishra A, Catchpole K, Dale T, McCulloch P. The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy. Surg Endosc 2007;22:68–73.10.1007/s00464-007-9346-1Search in Google Scholar PubMed

[97] Vedula SS, Ishii M, Hager GD. Objective assessment of surgical technical skill and competency in the operating room. Annu Rev Biomed Eng 2016. Available at: http://www.annualreviews.org/doi/abs/10.1146/annurev-bioeng-071516-044435.10.1146/annurev-bioeng-071516-044435Search in Google Scholar PubMed PubMed Central

[98] Oropesa I, Sánchez-González P, Lamata P, et al. Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res 2011;171:e81–e95.10.1016/j.jss.2011.06.034Search in Google Scholar PubMed

[99] Malpani A. Automated virtual coach for surgical training. PhD dissertation. The Johns Hopkins University, 2017.Search in Google Scholar

[100] Greenberg CC, Ghousseini HN, Pavuluri Quamme SR, Beasley HL, Wiegmann DA. Surgical coaching for individual performance improvement. Ann Surg 2015;261:32–34.10.1097/SLA.0000000000000776Search in Google Scholar PubMed

[101] Vedula SS, Malpani AO, Tao L, et al. Analysis of the structure of surgical activity for a suturing and knot-tying task. PLoS One 2016;11:e0149174.10.1371/journal.pone.0149174Search in Google Scholar PubMed PubMed Central

[102] Ahmidi N, Tao L, Sefati S, et al. A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery. Trans Biomed Eng 2017. doi: 10.1109/TBME.2016.2647680. [Epub ahead of print].10.1109/TBME.2016.2647680Search in Google Scholar PubMed PubMed Central

[103] Vedula SS, Gu B, Olds KC, et al. Robot-assisted active learning for surgical technical skill acquisition: early findings from a comparative study. In: The Hamlyn Symposium on Medical Robotics, London, UK; 2016.Search in Google Scholar

[104] Chen Z, Malpani A, Chalasani P, et al. Virtual fixture assistance for needle passing and knot tying. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2016:2343–2350.10.1109/IROS.2016.7759365Search in Google Scholar

[105] Rupprecht C, Lea C, Tombari F, Navab N, Hager GD. Sensor substitution for video-based action recognition. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2016:5230–5237.10.1109/IROS.2016.7759769Search in Google Scholar

[106] Ahmidi N, Poddar P, Jones JD, et al. Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty. Int J Comput Assist Radiol Surg 2015;10:981–991.10.1007/s11548-015-1194-1Search in Google Scholar PubMed


Supplementary Material:

The article (DOI: https://doi.org/10.1515/iss-2017-0004) offers reviewer assessments as supplementary material.


Received: 2016-12-30
Accepted: 2017-03-23
Published Online: 2017-04-20

©2017 Vedula S.S., Hager G.D., published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

Downloaded on 19.3.2024 from https://www.degruyter.com/document/doi/10.1515/iss-2017-0004/html
Scroll to top button