Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Current Directions in Biomedical Engineering

Joint Journal of the German Society for Biomedical Engineering in VDE and the Austrian and Swiss Societies for Biomedical Engineering

Editor-in-Chief: Dössel, Olaf

Editorial Board: Augat, Peter / Buzug, Thorsten M. / Haueisen, Jens / Jockenhoevel, Stefan / Knaup-Gregori, Petra / Kraft, Marc / Lenarz, Thomas / Leonhardt, Steffen / Malberg, Hagen / Penzel, Thomas / Plank, Gernot / Radermacher, Klaus M. / Schkommodau, Erik / Stieglitz, Thomas / Urban, Gerald A.


CiteScore 2018: 0.47

Source Normalized Impact per Paper (SNIP) 2018: 0.377

Open Access
Online
ISSN
2364-5504
See all formats and pricing
More options …

Improvement of a subviral particle tracker by the use of a LAP-Kalman-algorithm

Andreas Rausch / Dennis Müller / Thomas Schanze
Published Online: 2016-09-30 | DOI: https://doi.org/10.1515/cdbme-2016-0092

Abstract

Automated detection and tracking of subviral particles is a promising method to obtain insights in complicated virus-cell interactions. This paper describes the implementation of a linear assignment problem solver and a Kalman-filter in an existing particle tracking algorithm. Two different simulated image sequences are used for the evaluation of the algorithms. Tracking and detection results of the new implemented solver are compared to the results of the original algorithm. The improved algorithm is able to improve the results by closing gaps in the particle tracks.

Keywords: fluorescence microscopy; simulation; subviral particles; tracking

1 Introduction

The development of a reliable particle tracking algorithm for subivral particles is a great challenge. Difficulties in tracking subviral particles are poor spatial resolution, low contrast of the fluorescent images and effects like auto-fluorescent background, high object density, complex motion and varying shapes of the particles [1]. In coooperation with the virological institute of the Philipps-University Marburg, Germany, an algorithm (referred to as “Kienzle-algorithm”) based on expert-knowledge has been developed [2]. This algorithm enables the detection and tracking of particles with a low false-positive and a high true-positive rate. However, it is not able to fill gaps frequently present in the particle tracks [3]. To improve the tracking results by closing these gaps, i.e. combining related tracklets, a linear assignment problem (LAP) solver combined with a Kalman filter has been added to the algorithm. This paper describes the development of the LAP-Kalman-algorithm and the comparison of tracking results before and after its implementation.

2 Methods

The Kienzle-algorithm is divided into three steps. In the first step, possible candidates for subviral particles are detected and other objects are eliminated. The second step consists of the characterization of these objects. All objects are compared to a set of templates to enable a reliable linking of the tracks during the last step.

The original and the improved Kienzle-algorithm were used to analyze two different simulated image sequences with known ground-truth. The simulated image sequences contain 40, respectively 35 particles tracks with different lengths. However, not all tracks could be detected by the Kienzle-algorithm. Some tracks showed gaps, furthermore the beginnings or ends of some tracks could not be located. The simulated image sequence projection along with the interrupted tracks are shown in Figure 1.

Maximum intensity projection of a simulated sequence with subviral particle tracks, detected by the Kienzle algrorithm: The magnified areas in the red and the blue box show the tracked positions of two subviral particles over time (green line). These areas will be refered to in Figure 4 A, B and C. The red arrows mark the gaps, interrupting the tracks. The light green “tub” enclosing the track is the original track of the particle.
Figure 1

Maximum intensity projection of a simulated sequence with subviral particle tracks, detected by the Kienzle algrorithm: The magnified areas in the red and the blue box show the tracked positions of two subviral particles over time (green line). These areas will be refered to in Figure 4 A, B and C. The red arrows mark the gaps, interrupting the tracks. The light green “tub” enclosing the track is the original track of the particle.

Normally, the Kienzle-algorithm is filtering its results by using a treshold for assessing the object similarity. The LAP-Kalman-algorithm is working with the unfiltered raw data, resulting in a higher number of false-positive and true-positive hits. The idea to improve the tracking results was to close the gaps by post-processing the data solving the LAP combined with forward/backward-Kalman-filtering of the tracklets.

A LAP-algorithm is a method to solve an optimziation problem with respect to a cost function. Therefore the expenses of connecting the single tracklets to each other must be defined. Examples for these costs are distance in space and time between tracklets, differences in their average speed and travel direction, the variation of their size and the predicted next positions of the subviral particles. Distance, speed, size and direction differences are quite simple to determine pieces of information and have been already delivered by the Kienzle-tracking algorithm. So for the first step of the algorithm these simple costs are put into the LAP solver, which chooses the tracklet connections causing the lowest global costs. Its decisions are constrained by a threshold. For solving the task several algorithms were analysed and we decided to work with the auction-algorithm [4]. To estimate the behaviour of the subviral particles in the periods where no detection took place (gaps) is a complex task. To handle this problem we implemented a Kalman filter [5]. This filter is able to learn the physical behaviours of the subviral particles to estimate upcoming positions by feeding it with previous measurements. However, these measurements are affected by any kinds of interferences like statistical noise and measurement uncertainties. So the tracking algorithm can take benefit by two aspects of the Kalman filter. On the one hand it allows to smooth noisy tracks and on the other hand it is able to predict the next particle locations even if the particle is not visible at all. With sufficent information about the moving subviral particle it is, in an optimal case, possible to interpolate the missing track data with exact position estimations. We worked with a linear Kalman filter and chose constant velocity as motion model. However, in many applications, e.g. GPS, a Kalman filter is implemented. But GPS navigation or online tracking of objects do not allow to use future data for Kalman filtering, thus they only can process data from the past for updating the Kalmans predictions. In this application the Kienzle algorithm is doing the entire tracking task first so the Kalman filter is able to access all the time related positions of the virus simultaneously. As the Kalman predictions are getting more reliable the more measurement points it is fed with, it is quite instable in the beginning and is getting statistically better towards the end of each track. To compensate this effect the filter is processing each track dataset in both directions (for- and backward). The benefit of this method is illustrated in Figure 2.

Kalman forward and backward standard deviations: shown are the standard deviations of the estimated positions in y-direction minus the true ones, related to the position numbers one image per second). The blue line represents the standard deviations of the Kalman-forward-predictions und the green line of the backwarded ones. The red dots show the position with missing data.
Figure 2

Kalman forward and backward standard deviations: shown are the standard deviations of the estimated positions in y-direction minus the true ones, related to the position numbers one image per second). The blue line represents the standard deviations of the Kalman-forward-predictions und the green line of the backwarded ones. The red dots show the position with missing data.

Up to this point the solution of the LAP algorithm was only related on data from the Kienzle-algorithm (distance, speed, size and direction), fed into the LAP solver, followed by interpolating and smoothing the connected tracks by the Kalman filter. This already allowed to connect the tracklets obviously belonging together, as only small gaps appear between them. In contrast to this we found that in some cases tracks are seperated by huge gaps as well in space as in time. Often these gaps occur because of cell structures superposing the subviral particle tracks. In this case not only the gap between the two track parts is often enormous, also some tracks of other particles could cross the track of interest. The attempt to connect these parts by using the LAP algorithm for the previously mentioned basic cost parameters would propably fail in involving structures of a wrong track into the connection. Again we are benefiting of the Kalman algorithm ability to predict the positions of a non observable subviral particle. So in this second step of the presented algorithm each track is extended into the future and the past to anticipate the probable progression of the subviral particle track, by using the Kalman filter prediction. With the help of those predicted past and future extensions of the tracks it is now possible to estimate which track parts belong together, even if they are seperated by enormous gaps and superimposed by other structures. Again the LAP solver concludes related on these extended tracks which track belong together, by solving for the minimal distances between the estimated and the actual position of the subviral particle. Finally the tracks again are processed by the Kalman filter. So the output of the algorithm are complete smoothed and interpolated tracks. The whole algorithm sequence can be reviewed in the structure chart in Figure 3.

Structure chart: Boxes and ellipses represent algorithms and data, respectively. First the input for the LAP solver is obtained from the Kienzle tracking algorithm, like distance in space and time. Relating on these data the LAP algorithm is connecting the most likely track parts. Next the Kalman algorithm is interpolating and smoothing these connected tracks. The Kalman filter is estimating the previous and following position of the subviral particle. Secondly the LAP is using this new amount of information to associate more difficult tracklets with each other. Again the Kalman filter delivers smoothed and interpolated complete tracks as result. The dashed arrows symbolize only once executed dataflows and processes.
Figure 3

Structure chart: Boxes and ellipses represent algorithms and data, respectively. First the input for the LAP solver is obtained from the Kienzle tracking algorithm, like distance in space and time. Relating on these data the LAP algorithm is connecting the most likely track parts. Next the Kalman algorithm is interpolating and smoothing these connected tracks. The Kalman filter is estimating the previous and following position of the subviral particle. Secondly the LAP is using this new amount of information to associate more difficult tracklets with each other. Again the Kalman filter delivers smoothed and interpolated complete tracks as result. The dashed arrows symbolize only once executed dataflows and processes.

3 Results

In this sections the results of the single algorithm steps, the runtime performance and the improvements of the Kienzle tracker by the presented LAP-Kalman-algorithm will be presented. The execution of the LAP-solver is resulting in a list with the IDs of the connected tracklets. For a first impression of how the results could look like we linearly connected these tracklets. These connections are symbolized by red lines in Figure 4A on the example of one subviral particle track. The Kalman filter helps the LAP solver to connect far seperated tracklets with each other. Additionally it is able to smooth and to interpolate interrupted and noisy tracks. A Kalman filtering of one interrupted track in for- and backward direction is shown in Figure 4B. One completely reconstructed track can be seen in Figure 4C. In case of the presented simulated sequences the algorithm had to process 3060, respectively 3852 tracklets. In real image sequences the amount of data can vary widely, depending on the number of subviral particles. As the cost-matrix is growing quadratically with the amount of detected tracklets, the duration of running the whole algorithm also is increasing exponetially. So developing the algorithm amongst other aspects we focussed on optimizing the performance. The time consumption of executing the whole algorithm with different datasets is shown in Figure 5 (evaluated on a i7-3770 @ 3,4 Ghz, 24 GB RAM system). For evaluating the algorithm we used the described simulated datasets with 40 tracks from subviral structures in the first sequence and 35 in the second one. Using the Kienzle tracker 37, respectively 33 of these tracks were found, but were interrupted by 58 or 52 gaps, so these tracks are seperated into 95 or rather 85 tracklets. In addition to these subviral tracks, we resort to the fact that we were working on the unfiltered dataset, a certain amount of tracks from other structures were found by the Kienzle-algorithm, resulting in alltogether 3060, respectively 4852 tracklets. Processing these “raw” data with the LAP-Kalman-algorithm reduced the amount of gaps and tracklets dramatically, by not affecting the number of found true positive tracks. The number of gaps could be reduced to three or five. At the same time the averall amount of detected tracklets decreases to 819 or 1383. So also some tracklets of cell structures are connected, which reduces the effort to do a selection of the true positive hits afterwards. However, no additional false positives were produced by connecting subviral partical tracks with some of other cell structures. The number of detected subviral tracks was determined by counting manually. Figure 6 quantifies the results.

LAP and Kalman results: The interrupted track parts (green) have simply been reconnected by linear interpolation (red) related on the solution for the LAP problem (A). The Kalman filter is interpolating and smoothing tracklets (green) in for- and backward direction (yellow and blue). A weightened mean track is calculated (red line). The right connections are produced, even if the tracklets are interrupted by some other structure (“dirt”) (B). The algorithm (red) is reproducing the interrupted tracklets (green) (C).
Figure 4

LAP and Kalman results: The interrupted track parts (green) have simply been reconnected by linear interpolation (red) related on the solution for the LAP problem (A). The Kalman filter is interpolating and smoothing tracklets (green) in for- and backward direction (yellow and blue). A weightened mean track is calculated (red line). The right connections are produced, even if the tracklets are interrupted by some other structure (“dirt”) (B). The algorithm (red) is reproducing the interrupted tracklets (green) (C).

Time consumption: The expired time by running the whole algorithm is depending on the amount of detected tracklets.
Figure 5

Time consumption: The expired time by running the whole algorithm is depending on the amount of detected tracklets.

Results: Presented in absolute amounts. While reducing the number of gaps and thus the amount of tracklets, the number of found subviral particle tracks does not change.
Figure 6

Results: Presented in absolute amounts. While reducing the number of gaps and thus the amount of tracklets, the number of found subviral particle tracks does not change.

4 Discussion

A linear assignment problem solver and a Kalman-filter have been implemented to improve an existing particle tracking algorithm. The new algorithm is working in two main steps. First the LAP solver is connecting the obviously related tracks with each other by using parameters like distance between tracks, size of the subviral particles, etc. Afterwards the Kalman-filter is interpolating and smoothing these achieved completed tracks by processing them in for- and backward direction. In the second step the LAP-solver suggests tracklets which belong together. This time its results are based on predictions from the Kalman filter for the previous and following positions of a subviral particle. At the end the new tracks are smoothed and interpolated. The tracking results have been introduced and compared. The LAP-Kalman-algorithm is able to close gaps in the particle tracks and smoothes noisy track data. Despite on the fact that we were working on the raw data, no tracks which don’t belong together were connected mistakenly. A way to improve the stability and quality of results could be to process the presented LAP-Kalman algorithm as often as desired within an iterative loop. So after each iteration more tracklets are connected with each other and the cost parameters can be updated for even more reliable results. Also a dynamic weighting of the particular cost parameters, related on their increasing accuracy within each iteration, is planned. Furthermore the whole algorithm will be applied to more datasets for a more profound evaluation and to analyze influences on the LAP-Kalman-algorithm with and without iterative approach.

Acknowledgement

The authors would like to thank the virological institute of the Philipps-University, Marburg for providing fluorescence images.

Author’s Statement

Parts of this work have been presented at “Workshop Biosignalverarbeitung 201”, Berlin, 7–8 April 2016 [6]. Research funding: The author state no funding involved. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animals use

References

  • [1]

    Godinez WJ, Lampe M, Wörz S, Müller B, Eils R, Rohr L. Deterministic and probabilistic approaches for tracking virus particles in timelapse fluorescence microscopy image sequences. Med Image Anal. 2009;13:325–42. Google Scholar

  • [2]

    Kienzle C, Schudt G, Becker S, Schanze T. Multiple subviral particle in fluorescence microscopy sequences. BVMed; 2012. Google Scholar

  • [3]

    Müller D, Schanze T. Comparison of different algorithms for automated detection and tracking of fluorescence-labeled subviral particles. BMT; 2015. Google Scholar

  • [4]

    Bertsekas DP. Auction algorithms. Encyclopedia of Optimization; 2001. Google Scholar

  • [5]

    Welch G, Bishop G. An introduction to the Kalman filter. University of North Carolina at Chapel Hill; 2001. Google Scholar

  • [6]

    Rausch A, Müller D, Schanze T. Improvement of a subviral particle tracking algorithm by the use of a Kalman filter. BSV; 2016. Google Scholar

About the article

Published Online: 2016-09-30

Published in Print: 2016-09-01


Citation Information: Current Directions in Biomedical Engineering, Volume 2, Issue 1, Pages 415–418, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2016-0092.

Export Citation

©2016 Andreas Rausch et al., licensee De Gruyter.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0

Comments (0)

Please log in or register to comment.
Log in