Skip to content
Licensed Unlicensed Requires Authentication Published by Oldenbourg Wissenschaftsverlag May 7, 2021

Fusion of sequential LiDAR measurements for semantic segmentation of multi-layer grid maps

Fusion von Sequentiellen LiDAR Messungen für die Semantische Segmentierung von Multi-Layer Grid Maps
Frank Bieder

Frank Bieder received a Master‘s degree in electrical engineering and information technology from the Karlsruhe Institute of Technology, where he is currently pursuing a Ph. D. degree in computer vision and machine learning at the Institute of Measurement and Control Systems. Since 2019, he is a research scientist at the Mobile Perception Systems Department, FZI Research Center for Information Technology and a Doctoral Researcher at the Karlsruhe School of Optics & Photonics (KSOP).

EMAIL logo
, Sascha Wirges

Sascha Wirges received a Master‘s degree in electrical engineering from the Karlsruhe Institute of Technology in 2015, where he is currently pursuing a Ph. D. degree with the Institute of Measurement and Control Systems. His research is focused on traffic scene perception from top-view grid maps for automated driving applications.

, Sven Richter

Sven Richter received a Master‘s degree in mathematics from the University of Kaiserslautern in 2017. He is currently pursuing a Ph. D. degree with the Institute of Measurement and Control Systems at the Karlsruhe Institute of Technology. His research interests include sensor data fusion and environment perception.

and Christoph Stiller

Christoph Stiller received a Diploma degree in electrical engineering and a Ph. D. degree from RWTH Aachen University, Germany, in 1988 and 1994, respectively. He held a Postdoctoral position with INRS, Montreal, QC, Canada. In 1995, he joined the Research of Robert Bosch GmbH in Germany. Since 2001, he has been a full Professor with the Karlsruhe Institute of Technology, Germany and since 2009 director at FZI Research Center for Information Technology in Karlsruhe, Germany. He is spokesperson of the focus project ‘Cooperative Interacting Automobiles’ of the German Science Foundation DFG. His research interests include perception and planning for automated vehicles.

From the journal tm - Technisches Messen

Abstract

In this work, we improve the semantic segmentation of multi-layer top-view grid maps in the context of LiDAR-based perception for autonomous vehicles. To achieve this goal, we fuse sequential information from multiple consecutive LiDAR measurements with respect to the driven trajectory of an autonomous vehicle. By doing so, we enrich the multi-layer grid maps which are subsequently used as the input of a neural network. Our approach can be used for LiDAR-only 360 ° surround view semantic scene segmentation while being suitable for real-time critical systems. We evaluate the benefit of fusing sequential information using a dense semantic ground truth and discuss the effect on different classes.

Zusammenfassung

Im Rahmen dieser Arbeit verbessern wir die semantische Segmentierung von mehrschichtigen Top-View Rasterkarten im Kontext der LiDAR-basierten Umfeldwahrnehmung für das automatisierte Fahren. Um dieses Ziel zu erreichen, fusionieren wir sequenzielle Informationen aus mehreren aufeinanderfolgenden LiDAR-Messungen unter Berücksichtigung der gefahrenen Strecke eines autonomen Fahrzeugs. Hierdurch reichern wir die mehrschichtigen Rasterkarten mit den Informationen aus vergangenen Zeitschritten an und verwenden diese anschließend als Input eines neuronalen Netzes. Unser Ansatz kann für die semantische Segmentierung der 360 ° Rundumsicht verwendet werden und ist gleichzeitig für echtzeitkritische Systeme geeignet. Wir evaluieren den Nutzen der Fusion von sequenziellen Informationen auf Basis einer dichten, semantischen Ground Truth. Anschließend werden die Auswirkungen auf verschiedene semantische Klassen diskutiert.

About the authors

Frank Bieder

Frank Bieder received a Master‘s degree in electrical engineering and information technology from the Karlsruhe Institute of Technology, where he is currently pursuing a Ph. D. degree in computer vision and machine learning at the Institute of Measurement and Control Systems. Since 2019, he is a research scientist at the Mobile Perception Systems Department, FZI Research Center for Information Technology and a Doctoral Researcher at the Karlsruhe School of Optics & Photonics (KSOP).

Sascha Wirges

Sascha Wirges received a Master‘s degree in electrical engineering from the Karlsruhe Institute of Technology in 2015, where he is currently pursuing a Ph. D. degree with the Institute of Measurement and Control Systems. His research is focused on traffic scene perception from top-view grid maps for automated driving applications.

Sven Richter

Sven Richter received a Master‘s degree in mathematics from the University of Kaiserslautern in 2017. He is currently pursuing a Ph. D. degree with the Institute of Measurement and Control Systems at the Karlsruhe Institute of Technology. His research interests include sensor data fusion and environment perception.

Christoph Stiller

Christoph Stiller received a Diploma degree in electrical engineering and a Ph. D. degree from RWTH Aachen University, Germany, in 1988 and 1994, respectively. He held a Postdoctoral position with INRS, Montreal, QC, Canada. In 1995, he joined the Research of Robert Bosch GmbH in Germany. Since 2001, he has been a full Professor with the Karlsruhe Institute of Technology, Germany and since 2009 director at FZI Research Center for Information Technology in Karlsruhe, Germany. He is spokesperson of the focus project ‘Cooperative Interacting Automobiles’ of the German Science Foundation DFG. His research interests include perception and planning for automated vehicles.

Acknowledgment

The authors thank Daimler AG for the fruitful collaboration and the support for this work. In addition, they like to express their gratitude to Johannes Janosovits, who made contributions in the development of the underlying framework.

References

1. J. Behley, M. Garbade, A. Milioto, et al.“SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences”. In: Proc. of the IEEE/CVF International Conf. on Computer Vision (ICCV). 201910.1109/ICCV.2019.00939Search in Google Scholar

2. F. Bieder, S. Wirges, J. Janosovits, et al.“Exploiting Multi-Layer Grid Maps for Surround-View Semantic Segmentation of Sparse LiDAR Data”. In: IEEE Intelligent Vehicles Symposium, Proceedings. 202010.1109/IV47402.2020.9304848Search in Google Scholar

3. L. C. Chen, G. Papandreou, I. Kokkinos, et al.“DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2018)10.1109/TPAMI.2017.2699184Search in Google Scholar PubMed

4. M. Everingham, S. M. Eslami, L. Van Gool, et al.“The Pascal Visual Object Classes Challenge: A Retrospective”. In: International Journal of Computer Vision (2014)10.1007/s11263-014-0733-5Search in Google Scholar

5. A. Geiger, P. Lenz, and R. Urtasun. “Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite”. In: Conference on Computer Vision and Pattern Recognition (CVPR). 201210.1109/CVPR.2012.6248074Search in Google Scholar

6. GoogleResearch. “TensorFlow: Large-scale machine learning on heterogeneous systems”. In: Google Research (2015)Search in Google Scholar

7. A. Krizhevsky, I. Sutskever, and G. E. Hinton. “ImageNet classification with deep convolutional neural networks”. In: Communications of the ACM (2017)10.1145/3065386Search in Google Scholar

8. D. Nuss, T. Yuan, G. Krehl, et al.“Fusion of laser and radar sensor data with a sequential Monte Carlo Bayesian occupancy filter”. In: IEEE Intelligent Vehicles Symposium, Proceedings. 201510.1109/IVS.2015.7225827Search in Google Scholar

9. S. Richter, S. Wirges, H. Königshof, et al.“Fusion of range measurements and semantic estimates in an evidential framework”. In: tm - Technisches Messen (2019)10.1515/teme-2019-0052Search in Google Scholar

10. S. Wirges, C. Stiller, and F. Hartenbach. “Evidential Occupancy Grid Map Augmentation using Deep Learning”. In: IEEE Intelligent Vehicles Symposium, Proceedings. 201810.1109/IVS.2018.8500635Search in Google Scholar

11. S. Wirges, Y. Yang, S. Richter, et al.“Learned Enrichment of Top-View Grid Maps Improves Object Detection”. In: IEEE Conference on Intelligent Transportation Systems (ITSC), Proceedings. 202010.1109/ITSC45102.2020.9294330Search in Google Scholar

Received: 2021-02-15
Accepted: 2021-04-19
Published Online: 2021-05-07
Published in Print: 2021-06-26

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 10.12.2022 from https://www.degruyter.com/document/doi/10.1515/teme-2021-0026/html
Scroll Up Arrow