The development of a reliable particle tracking algorithm for subivral particles is a great challenge. Difficulties in tracking subviral particles are poor spatial resolution, low contrast of the fluorescent images and effects like auto-fluorescent background, high object density, complex motion and varying shapes of the particles . In coooperation with the virological institute of the Philipps-University Marburg, Germany, an algorithm (referred to as “Kienzle-algorithm”) based on expert-knowledge has been developed . This algorithm enables the detection and tracking of particles with a low false-positive and a high true-positive rate. However, it is not able to fill gaps frequently present in the particle tracks . To improve the tracking results by closing these gaps, i.e. combining related tracklets, a linear assignment problem (LAP) solver combined with a Kalman filter has been added to the algorithm. This paper describes the development of the LAP-Kalman-algorithm and the comparison of tracking results before and after its implementation.
The Kienzle-algorithm is divided into three steps. In the first step, possible candidates for subviral particles are detected and other objects are eliminated. The second step consists of the characterization of these objects. All objects are compared to a set of templates to enable a reliable linking of the tracks during the last step.
The original and the improved Kienzle-algorithm were used to analyze two different simulated image sequences with known ground-truth. The simulated image sequences contain 40, respectively 35 particles tracks with different lengths. However, not all tracks could be detected by the Kienzle-algorithm. Some tracks showed gaps, furthermore the beginnings or ends of some tracks could not be located. The simulated image sequence projection along with the interrupted tracks are shown in Figure 1.
Normally, the Kienzle-algorithm is filtering its results by using a treshold for assessing the object similarity. The LAP-Kalman-algorithm is working with the unfiltered raw data, resulting in a higher number of false-positive and true-positive hits. The idea to improve the tracking results was to close the gaps by post-processing the data solving the LAP combined with forward/backward-Kalman-filtering of the tracklets.
A LAP-algorithm is a method to solve an optimziation problem with respect to a cost function. Therefore the expenses of connecting the single tracklets to each other must be defined. Examples for these costs are distance in space and time between tracklets, differences in their average speed and travel direction, the variation of their size and the predicted next positions of the subviral particles. Distance, speed, size and direction differences are quite simple to determine pieces of information and have been already delivered by the Kienzle-tracking algorithm. So for the first step of the algorithm these simple costs are put into the LAP solver, which chooses the tracklet connections causing the lowest global costs. Its decisions are constrained by a threshold. For solving the task several algorithms were analysed and we decided to work with the auction-algorithm . To estimate the behaviour of the subviral particles in the periods where no detection took place (gaps) is a complex task. To handle this problem we implemented a Kalman filter . This filter is able to learn the physical behaviours of the subviral particles to estimate upcoming positions by feeding it with previous measurements. However, these measurements are affected by any kinds of interferences like statistical noise and measurement uncertainties. So the tracking algorithm can take benefit by two aspects of the Kalman filter. On the one hand it allows to smooth noisy tracks and on the other hand it is able to predict the next particle locations even if the particle is not visible at all. With sufficent information about the moving subviral particle it is, in an optimal case, possible to interpolate the missing track data with exact position estimations. We worked with a linear Kalman filter and chose constant velocity as motion model. However, in many applications, e.g. GPS, a Kalman filter is implemented. But GPS navigation or online tracking of objects do not allow to use future data for Kalman filtering, thus they only can process data from the past for updating the Kalmans predictions. In this application the Kienzle algorithm is doing the entire tracking task first so the Kalman filter is able to access all the time related positions of the virus simultaneously. As the Kalman predictions are getting more reliable the more measurement points it is fed with, it is quite instable in the beginning and is getting statistically better towards the end of each track. To compensate this effect the filter is processing each track dataset in both directions (for- and backward). The benefit of this method is illustrated in Figure 2.
Up to this point the solution of the LAP algorithm was only related on data from the Kienzle-algorithm (distance, speed, size and direction), fed into the LAP solver, followed by interpolating and smoothing the connected tracks by the Kalman filter. This already allowed to connect the tracklets obviously belonging together, as only small gaps appear between them. In contrast to this we found that in some cases tracks are seperated by huge gaps as well in space as in time. Often these gaps occur because of cell structures superposing the subviral particle tracks. In this case not only the gap between the two track parts is often enormous, also some tracks of other particles could cross the track of interest. The attempt to connect these parts by using the LAP algorithm for the previously mentioned basic cost parameters would propably fail in involving structures of a wrong track into the connection. Again we are benefiting of the Kalman algorithm ability to predict the positions of a non observable subviral particle. So in this second step of the presented algorithm each track is extended into the future and the past to anticipate the probable progression of the subviral particle track, by using the Kalman filter prediction. With the help of those predicted past and future extensions of the tracks it is now possible to estimate which track parts belong together, even if they are seperated by enormous gaps and superimposed by other structures. Again the LAP solver concludes related on these extended tracks which track belong together, by solving for the minimal distances between the estimated and the actual position of the subviral particle. Finally the tracks again are processed by the Kalman filter. So the output of the algorithm are complete smoothed and interpolated tracks. The whole algorithm sequence can be reviewed in the structure chart in Figure 3.
In this sections the results of the single algorithm steps, the runtime performance and the improvements of the Kienzle tracker by the presented LAP-Kalman-algorithm will be presented. The execution of the LAP-solver is resulting in a list with the IDs of the connected tracklets. For a first impression of how the results could look like we linearly connected these tracklets. These connections are symbolized by red lines in Figure 4A on the example of one subviral particle track. The Kalman filter helps the LAP solver to connect far seperated tracklets with each other. Additionally it is able to smooth and to interpolate interrupted and noisy tracks. A Kalman filtering of one interrupted track in for- and backward direction is shown in Figure 4B. One completely reconstructed track can be seen in Figure 4C. In case of the presented simulated sequences the algorithm had to process 3060, respectively 3852 tracklets. In real image sequences the amount of data can vary widely, depending on the number of subviral particles. As the cost-matrix is growing quadratically with the amount of detected tracklets, the duration of running the whole algorithm also is increasing exponetially. So developing the algorithm amongst other aspects we focussed on optimizing the performance. The time consumption of executing the whole algorithm with different datasets is shown in Figure 5 (evaluated on a i7-3770 @ 3,4 Ghz, 24 GB RAM system). For evaluating the algorithm we used the described simulated datasets with 40 tracks from subviral structures in the first sequence and 35 in the second one. Using the Kienzle tracker 37, respectively 33 of these tracks were found, but were interrupted by 58 or 52 gaps, so these tracks are seperated into 95 or rather 85 tracklets. In addition to these subviral tracks, we resort to the fact that we were working on the unfiltered dataset, a certain amount of tracks from other structures were found by the Kienzle-algorithm, resulting in alltogether 3060, respectively 4852 tracklets. Processing these “raw” data with the LAP-Kalman-algorithm reduced the amount of gaps and tracklets dramatically, by not affecting the number of found true positive tracks. The number of gaps could be reduced to three or five. At the same time the averall amount of detected tracklets decreases to 819 or 1383. So also some tracklets of cell structures are connected, which reduces the effort to do a selection of the true positive hits afterwards. However, no additional false positives were produced by connecting subviral partical tracks with some of other cell structures. The number of detected subviral tracks was determined by counting manually. Figure 6 quantifies the results.
A linear assignment problem solver and a Kalman-filter have been implemented to improve an existing particle tracking algorithm. The new algorithm is working in two main steps. First the LAP solver is connecting the obviously related tracks with each other by using parameters like distance between tracks, size of the subviral particles, etc. Afterwards the Kalman-filter is interpolating and smoothing these achieved completed tracks by processing them in for- and backward direction. In the second step the LAP-solver suggests tracklets which belong together. This time its results are based on predictions from the Kalman filter for the previous and following positions of a subviral particle. At the end the new tracks are smoothed and interpolated. The tracking results have been introduced and compared. The LAP-Kalman-algorithm is able to close gaps in the particle tracks and smoothes noisy track data. Despite on the fact that we were working on the raw data, no tracks which don’t belong together were connected mistakenly. A way to improve the stability and quality of results could be to process the presented LAP-Kalman algorithm as often as desired within an iterative loop. So after each iteration more tracklets are connected with each other and the cost parameters can be updated for even more reliable results. Also a dynamic weighting of the particular cost parameters, related on their increasing accuracy within each iteration, is planned. Furthermore the whole algorithm will be applied to more datasets for a more profound evaluation and to analyze influences on the LAP-Kalman-algorithm with and without iterative approach.
The authors would like to thank the virological institute of the Philipps-University, Marburg for providing fluorescence images.
Parts of this work have been presented at “Workshop Biosignalverarbeitung 201”, Berlin, 7–8 April 2016 . Research funding: The author state no funding involved. Conflict of interest: Authors state no conflict of interest. Material and Methods: Informed consent: Informed consent is not applicable. Ethical approval: The conducted research is not related to either human or animals use
Godinez WJ, Lampe M, Wörz S, Müller B, Eils R, Rohr L. Deterministic and probabilistic approaches for tracking virus particles in timelapse fluorescence microscopy image sequences. Med Image Anal. 2009;13:325–42. Google Scholar
Kienzle C, Schudt G, Becker S, Schanze T. Multiple subviral particle in fluorescence microscopy sequences. BVMed; 2012. Google Scholar
Müller D, Schanze T. Comparison of different algorithms for automated detection and tracking of fluorescence-labeled subviral particles. BMT; 2015. Google Scholar
Bertsekas DP. Auction algorithms. Encyclopedia of Optimization; 2001. Google Scholar
Welch G, Bishop G. An introduction to the Kalman filter. University of North Carolina at Chapel Hill; 2001. Google Scholar
Rausch A, Müller D, Schanze T. Improvement of a subviral particle tracking algorithm by the use of a Kalman filter. BSV; 2016. Google Scholar
About the article
Published Online: 2016-09-30
Published in Print: 2016-09-01
Citation Information: Current Directions in Biomedical Engineering, Volume 2, Issue 1, Pages 415–418, ISSN (Online) 2364-5504, DOI: https://doi.org/10.1515/cdbme-2016-0092.
©2016 Andreas Rausch et al., licensee De Gruyter.. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0