Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Evaluation of an algorithm to choose between competing models of respiratory mechanics

Jörn Kretschmer / Axel Riedlinger / Knut Möller
Published Online: 2015-09-12 | DOI: https://doi.org/10.1515/cdbme-2015-0103

Abstract

Model based decision support helps in optimizing therapy settings for individual patients while providing additional insight into a patient’s disease state through the identified model parameters. Using multiple models with different simulation focus and complexity allows adapting decision support to the current clinical situation and the available data. A previously presented set of numerical criteria allows selecting the best model based on fit quality, model complexity, and how well the parameter values are defined by the presented data. To systematically evaluate those criteria in an algorithm we have created insilico data sets using four different respiratory mechanics models with three different parameter settings each. Each of those artificial patients was ventilated with three different manoeuvres and the resulting data was used to identify the same models used to create the data. The selection algorithm was then presented with the results to select the best model. Not considering determinateness of the identified model parameters, the algorithm chose the same model that was used to create the data in 78%, a more complex model in 5% and a less complex model in 18% of all cases. When including the determinateness of model parameters in the decision process, the algorithm chose the same model in 42% of the cases and a less complex model in 56% of all cases. In 2% of the presented cases, no model complied with the required criteria.

Keywords: Medical decision support; physiological modelling; respiratory mechanics; model selection

1 Introduction

Mechanical ventilation is a routinely applied therapy that can be life-saving in patients that are not able to maintain sufficient oxygenation and carbon dioxide removal. However, the application of mechanical ventilation bears the risk of ventilator induced lung injury (VILI) if ventilator settings are set inappropriately [1]. Mathematical models can be individualized to reflect a patient’s pathophysiological characteristics and can thus be implemented in medical decision support systems where they are used to predict patient reaction to changes in therapy settings. To enable a decision support system to adapt to changes in the clinical scenario it needs to be provided with a model that is able to simulate various pathologies. Individualization of such complex models is usually challenging as the available data at the bedside might not be sufficient to robustly identify all model parameters. Providing multiple models differing in complexity and simulation focus is thus advantageous. Various models are described in literature [26] comprising various pathologies, a different numbers of model parameters and are thus of different mathematical complexity. Based on the current clinical question and the available data to identify the models, the decision support system should then be able to select the model that fits the current situation best. This decision should be based on numerical criterias describing the eligibility of the models to choose from.

We have previously presented multiple criteria that allow evaluating models based on fit quality, model complexity and the determinateness of the model parameters [6] and have presented preliminary results on ten mechanically ventilated patients [7]. However, a systematic evaluation of the algorithms performance has not been presented so far and should thus be described below.

2 Methods

2.1 Selection algorithm

The presented algorithm employs three numerical criteria describing the ability of a model to reproduce a given data set. The criteria are described in short below.

Coefficient of Determination (CD): CD describes how well a model is able to reproduce a given data set with 1 denoting perfect agreement and 0 showing that there is no correlation between model results and the data. CD is calculated as [2]:

CD=1SSEi=1n(ydataiyda¯ta)2(1)

with SSE denoting the summed squared differences between simulated and measured results (ydata). ydāta is the mean ofydata, n is the number of samples.

Corrected Akaike Information Criterion (AICc): The AICc allows identifying the model that has the maximum likelihood of describing a given data set while trading off between goodness-of-fit and the number of parameters that are adaptable. Lower values are preferred. AICc identifies which is the best among a given set models. The calculated values are thus not absolute and provide no information about how well a model reproduces the given data.

The AICc is calculated as [8, 9]:

AICc=2m+n[ln(2πSSEn+1)]+2m(m+1)nm1(2)

with m denoting the number of model parameters.

Confidence Interval (CI): The confidence intervals provide information about how well each model parameter is defined by the data. Large CI indicate that the parameter values are highly uncertain. CI were determined using the nlparci function in MATLAB (R2012a, The Mathworks, Natick, USA) which returns a range in which the parameter values are with a probability of 95%. All CI were normalized to their respective parameter value (relative CI, rCI).

Model selection algorithm. ydata – recorded patient data, ydatasim – model simulated data.
Figure 1

Model selection algorithm. ydata – recorded patient data, ydatasim – model simulated data.

Figure 1 shows a flow diagram of the selection algorithm. CD values serve as primary selection parameter, as the model with highest CD and all models within 2% of that CD are selected for the next stage. From the selected models, the model with lowest AICc is then selected and the rCI of all parameters is checked. If one is above 20%, the model is eliminated and the model with next best AICc is checked. If no model with sufficient CI is found, the algorithm selects the remaining models with CD 0.8 and ranks them again with respect to AICc. If CI is critical for all those models as well, model selection fails and no model is selected.

2.2 Artificial patient data and evaluation

We have used four models of respiratory mechanics with different complexity and a different number of identifi-able parameters. The applied models are a trivial model of first order with one resistance and one compliance (FOM) comprising two parameters (resistance R and compliance C), a viscoelastic model (VEM) comprising four parameters (resistances R1 and R2, compliances C1 and C2) [10], a model to simulate pressure dependent alveolar recruitment (PRM) with five parameters (R, C, threshold opening pressure TOP, number of opened alveoli at the onset of inspiration NOpen, overdistension parameter K) [11], and a model to simulate both viscoelastic behaviour and recruitment (PRVEM) comprising seven parameters (R1, R2, C1, C2, TOP, NOpen, K) [12].

Three different parameter sets for each model were created to simulate healthy patients and different stages of lung disease. Each of those 12 artificial patients was ventilated with three different ventilation manoeuvres, each with three different flow rates. The applied ventilation manoeuvres were:

  • Low-Flow (LF): low and constant inspiration flow of 30 to 40 ml/s for 50 seconds. This manoeuvre is applied to determine quasi-static pressure-volume dependencies.

  • Dynamic-Slice (DS): high and constant inspiration flow of 350 to 500 ml/s for 1.5s. This manoeuvre is applied to determine dynamic lung properties.

  • Static Compliance Automated Single Step (SCASS): volume controlled ventilation cycle with inspiration flows of 500 to 600 ml/s for 1s and a prolonged inspira-tory pause of 5s. SCASS is applied to determine equilibration processes in the lung tissue under a quasi-static pressure-volume relation.

In total, 108 data sets were created and modified by adding white noise with an amplitude of 5% of the respective data point. Each of the four respiratory mechanics models was then fitted to each of the data sets using a hierarchical identification approach [10, 12]. It uses the identified parameter values of less complex models as initial guesses when identifying the more complex models. It thus does not require any initial guesses and ensures optimal identification outcome. The model results were then fed to the presented algorithm to select the model that fits the given data best.

3 Results

Figure 2 shows simulated response to the described manoeuvres for all four models compared to artificial patient data created using the PRVEM. In LF results, the PRVEM had best CD and AICc, however CI was critical in two parameters, leading to the algorithm to select VEM. In DS data, the PRVEM again had best CD and AICc, however CI was critical in five parameters, leading to FOM being the selected model. The SCASS results showed highest CD and lowest AICc in VEM, thus this being the selected model.

Tables 1 and 2 show which model was selected compared to which model was used to create the data. Table 1 shows results when CI of the model parameters is not included in the selection process, while Table 2 shows results when CI is included as a criteria.

When omitting the CI criteria, the same model as used to create the data was selected by the algorithm in 84 of 108 cases. In 5 cases, a more complex model was selected due to a lower AICc and in 19 cases a less complex model was selected due to the same reason.

When including the CI criteria, the same model was selected in 45 out of the 108 cases. A more complex model was never selected, while a less complex model was selected in 61 cases (8 cases because of lower AICc, 53 cases because CI was critical in other models). No selection at all was conducted in 2 cases, all due to critical CI values in every model with CD 0.8.

4 Discussion

Using multiple models in decision support allows adaption to a change in the clinical scenario and to the patient data available at the bedside. In cases where detailed measurements are available, complex models can be identified, while simple models are applied if less information is available. The presented algorithm should enable the decision support system to select the model that fits the given data best with the smallest number of parameters while ensuring the correctness of the identified parameter values.

Exemplary results for model simulated results fitted to artificial patient data.
Figure 2

Exemplary results for model simulated results fitted to artificial patient data.

When omitting the CI criteria, i.e. the algorithm does not check if the parameter values of a model are suffi-ciently defined by the data, the algorithm selected the correct model, i.e. the model that was used to create the data, in 78% of all cases. In 18% of the cases, a less complex model was selected, i.e. the data could be reproduced with sufficient accuracy while using less model parameters. In data created with the PRM, five cases led to the algorithm to select the PRVEM, i.e. a more complex model. In those cases, parameter identification failed to find the correct PRM parameters thus leading to the simulation results to slightly deviate from the presented data. Here, identification procedure should be improved.

When including the CI criteria, the algorithm was able to select the correct model in 42% of all cases. In 56% of the cases, a less complex model was selected either due to having a comparable accuracy using less parameters (7% total) or due to CI being critical in the higher ranked models (49% total). CD being below 0.8 in all but one model or in all models was never the case, leading to the assumption that the identification routine is robust. No model was selected in 2% of the cases, all due to CI being critical in every model with CD 0.8. Thus, except for those two cases, all models are able to reproduce the presented data with acceptable accuracy.

Table 1

Results of model selection when CI is not evaluated.

Table 2

Results of model selection when CI is evaluated.

The presented algorithm thus allows selecting the model that is able to reproduce the recorded patient data with sufficient accuracy with the lowest number of parameters possible. Both tests (with CI criteria and without it) showed that the algorithm is able to find the model that was used to create the data or a model that fits the data with comparable accuracy while being less complex in an acceptable number of cases (95% of cases when CI is neglected, 49% with CI criteria). Additionally, the algorithm ensures that the identified parameters may be trusted by checking if the information contained in the recorded data is sufficient to define the parameter values. Here, results showed that in more complex models (PRM and PRVEM), CI is almost always critical in at least one parameter, leading to a less complex or no model being selected. We conclude that identifying models of higher complexity it may be insufficient to apply only one manoeuvre. Future work is planned to investigate which manoeuvres prove to be beneficial for the identification of which parameter in those models.

References

  • [1]

    A. S. Slutsky, ”Lung injury caused by mechanical ventilation,” Chest, vol. 116, pp. 9S-15S, 1999. Google Scholar

  • [2]

    J. H. T. Bates, Lung mechanics - An inverse modeling approach. Cambridge: Cambridge University Press, 2009. Google Scholar

  • [3]

    K. G. Hickling, ”The Pressure-Volume Curve Is Greatly Modified by Recruitment. A Mathematical Model of ARDS Lungs,” Am J Respir Crit Care Med, vol. 158, pp. 194-202, July 1, 1998 1998.Google Scholar

  • [4]

    C. Schranz, et al., ”Structural Identifiability and Practical Applicability of an Alveolar Recruitment Model for ARDS Patients,” IEEE Trans Biomed Eng., vol. 59, pp. 3396-3404, Dec 2012.Google Scholar

  • [5]

    C. Schranz, et al., ”Iterative integral parameter identification of a respiratory mechanics model,” Biomed Eng Online, vol. 11, p. 38, 2012.Google Scholar

  • [6]

    C. Schranz, et al., ”Selection criteria for competing models of respiratory mechanics,” Biomed Tech, vol. 58 (Suppl. 1), 2013. Google Scholar

  • [7]

    S. Basavaraju, et al., ”Kriterien zur automatisierten Modellselektion in Modellen der menschlichen Atemmechanik,” presented at the DMtS, Dresden, Germany, 2014. 

  • [8]

    H. Akaike, ”A new look at the statistical model identification,” IEEE Trans Automat Control, vol. 19, pp. 716-723, 1974. Google Scholar

  • [9]

    K. P. Burnham and D. R. Anderson, Model selection and multimodel interference: A practical information-theroretic approach, 2 ed. New York: Springer, 2002. 

  • [10]

    C. Schranz, et al., ”Hierarchical parameter identification in models of respiratory mechanics,” IEEE Trans Biomed Eng, vol. 58, pp. 3234-3241, Nov 2011. Google Scholar

  • [11]

    C. Schranz, et al., ”Identifiability Analysis of a Pressure-Depending Alveolar Recruitment Model,” in 8th IFAC Symposium on Biological and Medical Systems, Budapest, Hungary, 2012. 

  • [12]

    C. Schranz, et al., ”Hierarchical Individualization of a Recruitment Model with a Viscoelastic Component for ARDS Patients,” in Engineering in Medicine and Biology Society, EMBC, 2013 Annual International Conference of the IEEE, Osaka, Japan, 2013, pp. 5220-5223. 

About the article

Jörn Kretschmer: Institute of Technical Medicine, Furtwangen University, Villingen-Schwenningen, Germany, phone: +49 7720 307 4370, fax: +49 7720 307 4616


Published Online: 2015-09-12

Published in Print: 2015-09-01


Author’s Statement

Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent has been obtained from all individuals included in this study. Ethical approval: The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance the tenets of the Helsinki Declaration, and has been approved by the authors’ institutional review board or equivalent committee.


Citation Information: Current Directions in Biomedical Engineering, Volume 1, Issue 1, Pages 428–432, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2015-0103.

Export Citation

© 2015 by Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
J. Kretschmer, C. Bibiano, P. Stehle, and K. Möller
Journal of Biomedical Science and Engineering, 2016, Volume 09, Number 10, Page 90

Comments (0)

Please log in or register to comment.
Log in