Abstract
In this paper we present results to the problem of an adequate and compact symbolic representation of morphological features of anatomical structures that serve as surgical landmarks for automated assistance in endoscopic surgery using the General Formal Ontology (GFO) as a formal framework. For this purpose, we employed a translation from this first-order logic representation to a more compact description logic based formalism with the associated benefits, such as the availability of decidable reasoning procedures, for the purpose of automated landmark recognition in a hybrid symbolic/subsymbolic AI approach.
Introduction
The surgical navigation process in minimally-invasive endoscopic surgery is time- and resource- constrained. In the project COMPASS[1] a new markerless technology for immersive assistance in minimally-invasive and microscopic interventions is developed. It is based on Endsley’s theory of situation awareness [1] and its goal is to convert navigation systems into autonomous surgical actors.
In Endsley’s seminal paper situation awareness is defined as a state of knowledge about a relevant part of a dynamic situation in an observer’s environment. In the case of computer-assisted surgery, with the system fulfilling the role of the observer and the environment being the patient’s anatomy as perceived through the endoscope’s camera, the computer has to perform that perception task.
In previous work, a purely statistical Machine Learning approach based on image analysis using neural networks has been taken [2] but turned out to underperform in its task to recognize anatomical landmarks. In this paper, we focus on a novel approach, which combines subsymbolic with symbolic AI and is founded in cognition theory.
We present our results, which consist of an approach to the problem of a compact symbolic representation of morphological features of anatomical structures that serve as surgical landmarks. They are founded in the General Formal Ontology (GFO) and the onto-axiomatic method. In order to achieve this, we use a translation from the first-order logic (FOL) representation used by GFO to a more compact representation using description logics (DL), which is a family of decidable fragments of FOL. The benefits of a DL representation are the availability of decidable reasoning procedures, which can be then be used for mapping identified spatial arrangements of morphological features to an existing symbolic model of anatomy in order to identify landmarks.
The remainder of the paper is structured as follows: In Section 2 we outline GFO, its associated onto-axiomatic method, and methods for representing mereotopological situations using description logics as well as the cognitive theory of human object recognition the approach is based on. In Section 3 we present our results obtained from the analysis of the problem domain and an adequate representation. Section 4 concludes with a discussion of the results.
Materials and methods
We use the General Formal Ontology (GFO) as a framework for a formal account of the problem domain of spatial arrangements of geometric shapes to identify anatomical structures. The particular problem addressed in this paper is the representation of the morphology of anatomical landmarks for the purpose of automated object recognition. Our approach is based on the theory of recognition by components by Biederman [3]. The goal is to achieve a formal representation in a machine processable format that can be incorporated in the COMPASS system and used by automated reasoning services.
In the COMPASS project a Machine Learning component is trained to recognize geons in stereoscopic images coming from an endoscopic camera. The ML subsystem outputs a list of recognized geometrical objects with information about their shape, location in the scene, and dimension. From these concrete measures we calculate qualitative descriptions, such as relative positions of the objects (e.g. a cube behind a sphere) and topological relations (such as partial or complete intersection or tangential touching). We match these descriptions to a model of the human anatomy, which extends the Foundational Model of Anatomy (FMA)[2] expressed in W3C OWL.[3]
Image recognition by components
Biederman theorizes that object recognition in humans is a process in which objects are represented in the brain as a spatial arrangement of basic geometric objects such as cones, cylinders and ellipsoids, which he refers to as geons [3]. Surprisingly, only 36 different geon types are necessary in order to represent every possible object by their combination. Geons are described qualitatively in terms of their distinguishing features, ignoring exact measurements, which makes them robust against transformations such as warping and bending. Additionally, geons are theorized to be robust against change of point of view and partial blocking by other objects. Figure 1 shows a selection of geons.
Spatial configurations of GEONs are used in human object recognition according to Biederman’s recognition-by-components theory.[4]
General Formal Ontology
In what follows we outline the basic categories of GFO and introduce the entities relevant to the domain of space time.
The term entity refers to anything that has a mode of existence. Entities are classified into categories and individuals. Individuals are divided into concrete, abstract, and space time individuals. The axiomatization of space in GFO is inspired by Franz Brentano’s idea of space time. Space is defined in terms of spatial regions that are bordered by spatial boundaries. Both spatial regions and spatial boundaries are space entities. Spatial regions may have any dimensionality greater than 1. The codimension between the boundary and its associated spatial region is always 1. Spatial boundaries are dependent entities and can only exist in conjunction with a higher-dimensional space entity. When two spatial regions touch each other at their borders, their boundaries coincide but remain distinct.
These are the four fundamental axioms defining Brentano space in GFO:
A detailed account of space in GFO is given in [4].
Modal logic of space
Modal logics are propositional logics extended by two additional operators
Briefly, a proposition is necessarily true in a world that is only connected (by an accessibility relation) to worlds in which the proposition holds. It is possibly true in a world that is connected to at least one world in which the proposition holds. Conversely, a proposition variable with a modal operator may be interpreted as an expression the type of which is a set of possible worlds. Then
Modal logic of space [6] interprets the modal operators topologically: each model (then called topo-model) represents a topological space and each modal formula denotes a region of it. Under Kripke semantics possible worlds correspond to points in the topological space and an accessibility relation connects two such points iff all arbirarily small neighborhoods of these points are locally connected. The formula
Mapping from first-order to a compact description logic representation
Schild discovered a fundamental correspondence relation between modal logic and description logic [7], which we use for a transformation from one to the other. In total the translation process involves two steps: first a translation from the first-order GFO representation to modal space logic and second from the latter to description logic.
Mapping from GFO space to modal space logic
In the following we restrict our treatment to three-dimensional space regions with two-dimensional boundaries in accordance with the COMPASS use case. The mapping is trivially applicable to a different number of dimensions.
GFO space regions have a direct correspondence to regions in topological spaces. Thus, a GFO space region x can be mapped to a modal space region A, and a boundary of A is accordingly:
A (A is a space region)
Modal space logic is less expressive than first-order logic in that it cannot express coinciding distinct boundaries. As a consequence, axiom B3 cannot be directly translated to a modal formula, since in B3 it is assumed that x and y are distinct. To work around this problem, we introduce the artificial concept of Coincidence and an n-ary predicate has_participants that connects a coincidence instance to an arbitrary number of boundaries. Using this concept we define a mapping of coincidence in GFO and modal space logic as follows:
is extended by
which is translated to
Spatial parthood is translated to set inclusion:
Mapping from modal space logic to description logic
Due to the above-mentioned correspondence we can map the above modal logic formulae to description logic formulae in a straightforward fashion. Let A and B be concept names, i an individual name and r a relation serving as our accessibility relation. There is one accessibility relation for each topological space; since we assume everything of importance shares the same topological space, we only need one such relation.
A (A is a space region).
Note that in D3. we instantiate the concept expression
Results
With the translation described in Section 2 it is possible to represent all possible mereotopological constellations of arbitrarily many geometrical objects in a single description logic formula and perform reasoning on them.
Figure 2 shows a number of typical constellations. The description logic formulae for formalizing the constellations are as follows:
Different mereotopological configurations of two or more objects.
Discussion
We have presented a solution to the problem of a compact symbolic representation of mereotopological constellations of objects in space as a subproblem for automatically identifying anatomical structures in 3D endoscopic imagery.
The proposed representation formalism has been put to use in the form of W3C OWL ontologies and tableau-based reasoners. In conjunction with other modeling problems, such as dynamic situations (e.g. change of anatomy due to surgical interventions) or epistemic situations (a model of a systems knowledge about a surgeon’s awareness of a situations) [8] the system is able to complement subsymbolic AI approaches, such as neural networks.
Acknowledgments
This work has been partially supported by the German Federal Ministry of Education and Research (BMBF) by a grant for the COMPASS project under the KMU Innovative program.
Research funding: None declared.
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
Competing interests: Authors state no conflict of interest.
References
1. Endsley, MR. Toward a theory of situation awareness in dynamic systems. Hum Factors 1995;37:32–64. https://doi.org/10.1518/001872095779049543.Search in Google Scholar
2. Siemoleit, S, Uciteli, A, Bieck, R, Heinrich, H. Processual reasoning over sequences of situations in endoscopic surgery. In: Proceedings of the 62nd annual meeting of the German Association of Medical Informatics, Biometry and Epidemiology, GMDS 2017, volume 243 of Studies in Health Technology and Informatics, IOS Press; 2017, pp. 222–6.Search in Google Scholar
3. Biederman, I. Recognition-by-components: a theory of human image understanding. Psychol Rev 1987;94:115–47. https://doi.org/10.1037/0033-295x.94.2.115.Search in Google Scholar
4. Baumann, R, Frank, L, Heinrich, H. Towards an ontology of space for GFO. In: Formal ontology in information systems – proceedings of the 9th international conference, FOIS 2016, Annecy, France, July 6–9, 2016; 2016, pp. 53–66.Search in Google Scholar
5. Blackburn, P, Johan, F, van Benthem, AK, Frank, W. Handbook of modal logic. In: Studies in logic and practical reasoning. New York, NY, USA: Elsevier Science Inc.; 2006, vol 3.Search in Google Scholar
6. Van Benthem, J, Bezhanishvili, G. Modal logics of space. In: Aiello, M, Pratt-Hartmann, I, Van Benthem, J, editors. Handbook of spatial logics. Dordrecht: Springer Netherlands; 2007, pp. 217–98.10.1007/978-1-4020-5587-4_5Search in Google Scholar
7. Schild, K. A correspondence theory for terminological logics: preliminary report. In: Proceedings of the 12th international joint conference on artificial intelligence, Sydney, Australia, August 24–30, 1991; 1991, pp. 466–71.Search in Google Scholar
8. Schäfermeier, R, Adrian, P, Heinrich, H. Ontology design patterns for representing context in ontologies using aspect orientation. In: Proceedings of the 10th workshop on Ontology Design and Patterns (WOP 2019) co-located with 18th International Semantic Web Conference (ISWC 2019), Auckland, New Zealand, October 27, 2019, volume 2459 of CEUR workshop proceedings. CEUR-WS.org; 2019. pp. 32–46.10.3233/SSW210014Search in Google Scholar
© 2020 Ralph Schäfermeier et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.
