Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter January 25, 2017

Optical lens-shift design for increasing spatial resolution of 3D ToF cameras

  • Henrik Lietz

    Henrik Lietz studied Optical System Technologies at the University of Applied Sciences Ravensburg-Weingarten and received his MSc degree in 2013. Since then, he has been working as a researcher primarily in the field of 3D camera technology and technical optics. He is currently a PhD student at the Ilmenau University of Technology.

    EMAIL logo
    , M. Muneeb Hassan

    M. Muneeb Hassan completed his MSc in Mechatronics from the University of Applied Sciences Ravensburg-Weingarten in 2015. He is currently pursuing his doctorate from the University of Brescia, Italy, and is working as a researcher at the University of Applied Sciences Ravensburg-Weingarten in the field of marker-less motion capture with 3D cameras.

    and Jörg Eberhardt

    Jörg Eberhardt studied Information Science majoring in Machine Vision and Machine Learning at the Polytechnic of Konstanz in 1993. After graduating, he carried out further studies and research in the field of 3D scanners and 3D modeling at Coventry University/GB and was awarded a PhD in 1998. From 1999 to 2013, he worked as senior engineer for well-known companies in the field of machine vision where he was responsible for research and development. In 2013, Jörg Eberhardt became an endowed professor for optical technologies at the University of Applied Sciences Weingarten. His research areas are 3D technologies, camera technologies, and machine vision. Jörg Eberhardt is the co-founder of the company Corpus-e, the leading experts in the field of 3D scanning of human body parts.

Abstract

Sensor resolution of 3D time-of-flight (ToF) outdoor-capable cameras is strongly limited because of its large pixel dimensions. Computational imaging permits enhancement of the optical system’s resolving power without changing physical sensor properties. Super-resolution (SR) algorithms superimpose several sub-pixel-shifted low-resolution (LR) images to overcome the system’s limited spatial sampling rate. In this paper, we propose a novel opto-mechanical system to implement sub-pixel shifts by moving an optical lens. This method is more flexible in terms of implementing SR techniques than current sensor-shift approaches. In addition, we describe a SR observation model that has been optimized for the use of LR 3D ToF cameras. A state-of-the-art iteratively reweighted minimization algorithm executes the SR process. It is proven that our method achieves nearly the same resolution increase as if the pixel area would be halved physically. Resolution enhancement is measured objectively for amplitude images of a static object scene.

About the authors

Henrik Lietz

Henrik Lietz studied Optical System Technologies at the University of Applied Sciences Ravensburg-Weingarten and received his MSc degree in 2013. Since then, he has been working as a researcher primarily in the field of 3D camera technology and technical optics. He is currently a PhD student at the Ilmenau University of Technology.

M. Muneeb Hassan

M. Muneeb Hassan completed his MSc in Mechatronics from the University of Applied Sciences Ravensburg-Weingarten in 2015. He is currently pursuing his doctorate from the University of Brescia, Italy, and is working as a researcher at the University of Applied Sciences Ravensburg-Weingarten in the field of marker-less motion capture with 3D cameras.

Jörg Eberhardt

Jörg Eberhardt studied Information Science majoring in Machine Vision and Machine Learning at the Polytechnic of Konstanz in 1993. After graduating, he carried out further studies and research in the field of 3D scanners and 3D modeling at Coventry University/GB and was awarded a PhD in 1998. From 1999 to 2013, he worked as senior engineer for well-known companies in the field of machine vision where he was responsible for research and development. In 2013, Jörg Eberhardt became an endowed professor for optical technologies at the University of Applied Sciences Weingarten. His research areas are 3D technologies, camera technologies, and machine vision. Jörg Eberhardt is the co-founder of the company Corpus-e, the leading experts in the field of 3D scanning of human body parts.

Acknowledgments

This work has been funded by ‘Innovative Projekte 2014’. The authors would like to express their gratitude toward the state of Baden-Württemberg and ifm electronic GmbH for their support during the course of this research.

References

[1] S. Foix, G. Alenya and C. Torras, IEEE Sens. J. 11, 1917–1926 (2011).10.1109/JSEN.2010.2101060Search in Google Scholar

[2] J. D. van Ouwerkerk, Image Vision Comput 24, 1039–1052 (2006).10.1016/j.imavis.2006.02.026Search in Google Scholar

[3] J. Tian and K.-K. Ma, Signal Image Video P. 5, 329–342 (2011).10.1007/s11760-010-0204-6Search in Google Scholar

[4] K. Nasrollahi and T. B. Moeslund, Mach Vision Appl. 25, 1423–1468 (2014).10.1007/s00138-014-0623-4Search in Google Scholar

[5] M. Ben-Ezra, A. Zomet and S. K. Nayar, CVPR 2004, 135–142 (2004).Search in Google Scholar

[6] T. Köhler, X. Huang, F. Schebesch, A. Aichert, A. Maier, et al., IEEE Trans. Comput. Imaging 2, 42–58 (2016).10.1109/TCI.2016.2516909Search in Google Scholar

[7] P. D. Burns and D. Williams, Proc. SPIE 6808, Image Quality and System Performance V 6808 (2008).Search in Google Scholar

[8] H. Lietz and J. Eberhardt, Proc. SPIE 9628, Optical Systems Design 2015: Optical Fabrication, Testing, and Metrology V (9628H) (2015).Search in Google Scholar

[9] R. Lange, P. Seitz, A. Biber and R. Schwarte, Proc. SPIE 3823, Laser Metrology and Inspection (3823) (1999).Search in Google Scholar

[10] R. Lange and P. Seitz, IEEE J. Quantum Elect. 37, 390–397 (2001).10.1109/3.910448Search in Google Scholar

[11] H. G. Heinol, Dissertation (2001).Search in Google Scholar

[12] R. Lange, Dissertation (2000).Search in Google Scholar

[13] T. Möller, H. Kraft, J. Frey, M. Albrecht and R. Lange, Proceedings of the 1st Range Imaging Research Day at ETH, 3-906467 (2005).Search in Google Scholar

[14] T. Köhler, J. Jordan, A. Maier and J. Hornegger, Proceedings of the British Machine Vision Conference (BMVC), 143.1–143.12 (2015).Search in Google Scholar

[15] S. C. Park, M. K. Park and M. G. Kang, IEEE Signal Proc., 20, 21–36 (2003).10.1109/MSP.2003.1203207Search in Google Scholar

[16] Tsai and T. Huang, Advances in Computer Vision and Image Processing, vol. 1, no. 2, JAI Press Inc., 317–339 (1984).Search in Google Scholar

[17] P. Vandewalle, S. Süsstrunk and M. Vetterli, EURASIP J. Adv. Sig. Pr. 2006, 1–15 (2006).Search in Google Scholar

[18] B. Huhle, T. Schairer, P. Jenke and W. Straßer, Comput. Vis. Image Understand. 114, 1336–1345 (2010).10.1016/j.cviu.2009.11.004Search in Google Scholar

[19] A. Colaco, A. Kirmani, G. A. Howland, J. C. Howell and V. K. Goyal, 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 96–102 (2012).Search in Google Scholar

[20] L. Xiao, F. Heide, M. O’Toole, A. Kolb, M.B. Hullin, K. Kutulakos, W. Heidrich. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2376–2384 (2015).Search in Google Scholar

[21] S. Schuon, C. Theobalt, J. Davis and S. Thrun, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 343–350 (2009).Search in Google Scholar

[22] S. Schuon, C. Theobalt, J. Davis and S. Thrun, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 1–7 (2008).Search in Google Scholar

[23] S. Farsiu, M. D. Robinson, M. Elad and P. Milanfar, IEEE Trans. Image Process. 13, 1327–1344 (2004).10.1109/TIP.2004.834669Search in Google Scholar PubMed

[24] ifm electronic gmbh, O3M151 data sheet, Available at: http://www.ifm.com/products/de/ds/O3M151.htm, Accessed: 24/Nov/2016.Search in Google Scholar

[25] L. f. Mustererkennung and F.-A.-U. Erlangen-Nürnberg, Multi-Frame Super-Resolution Toolbox, Available at: https://www5.cs.fau.de/research/software/multi-frame-super-resolution-toolbox/, Accessed: 26/Nov/2016.Search in Google Scholar

[26] M. Lindner and A. Kolb, Proc. SPIE 6764, Intelligent Robots and Computer Vision XXV: Algorithms, Techniques, and Active Vision (6764W) (2007).Search in Google Scholar

[27] S. A. Gudmundsson, H. Aanaes and R. Larsen, Environmental Effects on Measurement Uncertainties of Time-of-Flight Cameras, Available at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.133.4032.Search in Google Scholar

[28] X. Li, Y. Hu, X. Gao, D. Tao, and B. Ning, Signal Process. 90, 405–414 (2010).10.1016/j.sigpro.2009.05.028Search in Google Scholar

[29] sfrmat3: slanted-edge analysis for digital cameras and scanners, Available at: http://losburns.com/imaging/software/SFRedge/index.htm, Accessed: 26/Nov/2016.Search in Google Scholar

[30] P. D. Burns, sfrmat 2.0 User’s Guide, Available at: http://read.pudn.com/downloads108/sourcecode/math/448322/sfrmat2_guide.pdf.Search in Google Scholar

[31] G. D. Boreman, Modulation Transfer Function in Optical and Electro-Optical Systems. Bellingham, Washington, USA: SPIE – The International Society for Optical Engineering, 2001.10.1117/3.419857Search in Google Scholar

Received: 2016-12-1
Accepted: 2016-12-22
Published Online: 2017-1-25
Published in Print: 2017-2-1

©2017 THOSS Media & De Gruyter

Downloaded on 7.2.2023 from https://www.degruyter.com/document/doi/10.1515/aot-2016-0063/html
Scroll Up Arrow