Accessible Requires Authentication Published by De Gruyter January 25, 2017

Optical lens-shift design for increasing spatial resolution of 3D ToF cameras

Henrik Lietz, M. Muneeb Hassan and Jörg Eberhardt

Abstract

Sensor resolution of 3D time-of-flight (ToF) outdoor-capable cameras is strongly limited because of its large pixel dimensions. Computational imaging permits enhancement of the optical system’s resolving power without changing physical sensor properties. Super-resolution (SR) algorithms superimpose several sub-pixel-shifted low-resolution (LR) images to overcome the system’s limited spatial sampling rate. In this paper, we propose a novel opto-mechanical system to implement sub-pixel shifts by moving an optical lens. This method is more flexible in terms of implementing SR techniques than current sensor-shift approaches. In addition, we describe a SR observation model that has been optimized for the use of LR 3D ToF cameras. A state-of-the-art iteratively reweighted minimization algorithm executes the SR process. It is proven that our method achieves nearly the same resolution increase as if the pixel area would be halved physically. Resolution enhancement is measured objectively for amplitude images of a static object scene.

Acknowledgments

This work has been funded by ‘Innovative Projekte 2014’. The authors would like to express their gratitude toward the state of Baden-Württemberg and ifm electronic GmbH for their support during the course of this research.

References

[1] S. Foix, G. Alenya and C. Torras, IEEE Sens. J. 11, 1917–1926 (2011). Search in Google Scholar

[2] J. D. van Ouwerkerk, Image Vision Comput 24, 1039–1052 (2006). Search in Google Scholar

[3] J. Tian and K.-K. Ma, Signal Image Video P. 5, 329–342 (2011). Search in Google Scholar

[4] K. Nasrollahi and T. B. Moeslund, Mach Vision Appl. 25, 1423–1468 (2014). Search in Google Scholar

[5] M. Ben-Ezra, A. Zomet and S. K. Nayar, CVPR 2004, 135–142 (2004). Search in Google Scholar

[6] T. Köhler, X. Huang, F. Schebesch, A. Aichert, A. Maier, et al., IEEE Trans. Comput. Imaging 2, 42–58 (2016). Search in Google Scholar

[7] P. D. Burns and D. Williams, Proc. SPIE 6808, Image Quality and System Performance V 6808 (2008). Search in Google Scholar

[8] H. Lietz and J. Eberhardt, Proc. SPIE 9628, Optical Systems Design 2015: Optical Fabrication, Testing, and Metrology V (9628H) (2015). Search in Google Scholar

[9] R. Lange, P. Seitz, A. Biber and R. Schwarte, Proc. SPIE 3823, Laser Metrology and Inspection (3823) (1999). Search in Google Scholar

[10] R. Lange and P. Seitz, IEEE J. Quantum Elect. 37, 390–397 (2001). Search in Google Scholar

[11] H. G. Heinol, Dissertation (2001). Search in Google Scholar

[12] R. Lange, Dissertation (2000). Search in Google Scholar

[13] T. Möller, H. Kraft, J. Frey, M. Albrecht and R. Lange, Proceedings of the 1st Range Imaging Research Day at ETH, 3-906467 (2005). Search in Google Scholar

[14] T. Köhler, J. Jordan, A. Maier and J. Hornegger, Proceedings of the British Machine Vision Conference (BMVC), 143.1–143.12 (2015). Search in Google Scholar

[15] S. C. Park, M. K. Park and M. G. Kang, IEEE Signal Proc., 20, 21–36 (2003). Search in Google Scholar

[16] Tsai and T. Huang, Advances in Computer Vision and Image Processing, vol. 1, no. 2, JAI Press Inc., 317–339 (1984). Search in Google Scholar

[17] P. Vandewalle, S. Süsstrunk and M. Vetterli, EURASIP J. Adv. Sig. Pr. 2006, 1–15 (2006). Search in Google Scholar

[18] B. Huhle, T. Schairer, P. Jenke and W. Straßer, Comput. Vis. Image Understand. 114, 1336–1345 (2010). Search in Google Scholar

[19] A. Colaco, A. Kirmani, G. A. Howland, J. C. Howell and V. K. Goyal, 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 96–102 (2012). Search in Google Scholar

[20] L. Xiao, F. Heide, M. O’Toole, A. Kolb, M.B. Hullin, K. Kutulakos, W. Heidrich. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2376–2384 (2015). Search in Google Scholar

[21] S. Schuon, C. Theobalt, J. Davis and S. Thrun, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 343–350 (2009). Search in Google Scholar

[22] S. Schuon, C. Theobalt, J. Davis and S. Thrun, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops), 1–7 (2008). Search in Google Scholar

[23] S. Farsiu, M. D. Robinson, M. Elad and P. Milanfar, IEEE Trans. Image Process. 13, 1327–1344 (2004). Search in Google Scholar

[24] ifm electronic gmbh, O3M151 data sheet, Available at: http://www.ifm.com/products/de/ds/O3M151.htm, Accessed: 24/Nov/2016. Search in Google Scholar

[25] L. f. Mustererkennung and F.-A.-U. Erlangen-Nürnberg, Multi-Frame Super-Resolution Toolbox, Available at: https://www5.cs.fau.de/research/software/multi-frame-super-resolution-toolbox/, Accessed: 26/Nov/2016. Search in Google Scholar

[26] M. Lindner and A. Kolb, Proc. SPIE 6764, Intelligent Robots and Computer Vision XXV: Algorithms, Techniques, and Active Vision (6764W) (2007). Search in Google Scholar

[27] S. A. Gudmundsson, H. Aanaes and R. Larsen, Environmental Effects on Measurement Uncertainties of Time-of-Flight Cameras, Available at: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.133.4032. Search in Google Scholar

[28] X. Li, Y. Hu, X. Gao, D. Tao, and B. Ning, Signal Process. 90, 405–414 (2010). Search in Google Scholar

[29] sfrmat3: slanted-edge analysis for digital cameras and scanners, Available at: http://losburns.com/imaging/software/SFRedge/index.htm, Accessed: 26/Nov/2016. Search in Google Scholar

[30] P. D. Burns, sfrmat 2.0 User’s Guide, Available at: http://read.pudn.com/downloads108/sourcecode/math/448322/sfrmat2_guide.pdf. Search in Google Scholar

[31] G. D. Boreman, Modulation Transfer Function in Optical and Electro-Optical Systems. Bellingham, Washington, USA: SPIE – The International Society for Optical Engineering, 2001. Search in Google Scholar

Received: 2016-12-1
Accepted: 2016-12-22
Published Online: 2017-1-25
Published in Print: 2017-2-1

©2017 THOSS Media & De Gruyter