Accessible Requires Authentication Published by Oldenbourg Wissenschaftsverlag April 28, 2021

Model-independent light field reconstruction using a generic camera calibration

Modellunabhängige Lichtfeld-Rekonstruktion mithilfe einer generischen Kamerakalibrierung
David Uhlig ORCID logo and Michael Heizmann
From the journal tm - Technisches Messen


Sophisticated and highly specialized optical measuring devices are becoming increasingly important for high-precision manufacturing and environment perception. In particular, light field cameras are experiencing an ever-increasing interest in research and industry as they enable a variety of new measurement methods. Unfortunately, due to their complex structure, their calibration is very difficult and usually precisely tailored to the particular type of light field camera. To overcome these difficulties, we present a method that decodes a light field from the raw data of any light field imaging system without knowing and modeling the internal optical elements. We calibrate the camera using a precise generic calibration method and transform the obtained ray set into an equivalent light field representation. Finally, we reconstruct a rectified light field from the irregularly sampled data and in addition we derive the geometric ray properties as intrinsic camera parameters. Experimental results validate the method by showing that both the information of the observed scene and the geometric structure of the light field are preserved by an adequate rectification and calibration.


Anspruchsvolle und hochspezialisierte optische Messgeräte werden für die hochpräzise Fertigung und Umfelderkennung immer wichtiger. Insbesondere Lichtfeldkameras erfahren ein immer größeres Interesse in Forschung und Industrie, da sie eine Vielzahl von neuen Messmethoden ermöglichen. Leider ist ihre Kalibrierung aufgrund ihres komplexen Aufbaus sehr schwierig und meist genau auf den jeweiligen Lichtfeldkamera-Typ zugeschnitten. Um diese Schwierigkeiten zu überwinden, stellen wir eine Methode vor, die ein Lichtfeld aus den Rohdaten eines beliebigen Lichtfeldaufnahmesystem decodiert, ohne die internen optischen Elemente zu kennen und zu modellieren. Wir kalibrieren die Kamera mit einer präzisen generischen Kalibrierungsmethode und transformieren das erhaltene Strahlenset in eine äquivalente Lichtfelddarstellung. Schließlich rekonstruieren wir ein rektifiziertes Lichtfeld aus den unregelmäßig abgetasteten Daten und leiten darüber hinaus die geometrischen Eigenschaften der Strahlen als intrinsische Kameraparameter ab. Experimentelle Ergebnisse validieren die Methode, indem sie zeigen, dass sowohl die Informationen der beobachteten Szene als auch die geometrische Struktur des Lichtfeldes durch eine adäquate Rektifizierung und Kalibrierung erhalten bleiben.


1. A. Bartoli and P. Sturm. The 3D Line Motion Matrix and Alignment of Line Reconstructions. International Journal of Computer Vision, 57(3):159–178, 2004. Search in Google Scholar

2. F. Bergamasco, A. Albarelli, E. Rodola, and A. Torsello. Can a Fully Unconstrained Imaging Model Be Applied Effectively to Central Cameras? In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013, pages 1391–1398, Piscataway, NJ, 2013. IEEE. Search in Google Scholar

3. Y. Bok, H.-G. Jeon, and I. S. Kweon. Geometric Calibration of Micro-Lens-Based Light Field Cameras Using Line Features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(2):287–300, 2017. Search in Google Scholar

4. D. G. Dansereau, O. Pizarro, and S. B. Williams. Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, pages 1027–1034, 2013. Search in Google Scholar

5. T. Georgiev and A. Lumsdaine, editors. The multifocus plenoptic camera. International Society for Optics and Photonics, 2012. Search in Google Scholar

6. M. D. Grossberg and S. K. Nayar. The Raxel Imaging Model and Ray-Based Calibration. International Journal of Computer Vision, 61(2):119–137, 2005. Search in Google Scholar

7. I. Ihrke, J. Restrepo, and L. Mignard-Debise. Principles of Light Field Imaging: Briefly revisiting 25 years of research. IEEE Signal Processing Magazine, 33(5):59–69, 2016. Search in Google Scholar

8. A. Lumsdaine and T. Georgiev, editors. The focused plenoptic camera. IEEE, 2009. Search in Google Scholar

9. M. Schambach and F. P. León. Microlens Array Grid Estimation, Light Field Decoding, and Calibration. IEEE Transactions on Computational Imaging, 6:591–603, 2020. Search in Google Scholar

10. M. Schambach and M. Heizmann. A Multispectral Light Field Dataset and Framework for Light Field Deep Learning. IEEE Access, 8:193492–193502, 2020. Search in Google Scholar

11. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan. Light field photography with a hand-held plenoptic camera. Computer Science Technical Report CSTR, 2(11):1–11, 2005. Search in Google Scholar

12. O. Johannsen, A. Sulc, and B. Goldluecke. On Linear Structure from Motion for Light Field Cameras. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 720–728, 2015. Search in Google Scholar

13. D. Uhlig and M. Heizmann. A Calibration Method for the Generalized Imaging Model with Uncertain Calibration Target Coordinates. In Proceedings of the Asian Conference on Computer Vision (ACCV), 2020. Search in Google Scholar

14. W. v. d. Hodge and D. Pedoe. Methods of Algebraic Geometry. Cambridge University Press, Cambridge, 1994. Search in Google Scholar

15. S. van der Jeught and J. J. Dirckx. Real-time structured light profilometry: a review. Optics and Lasers in Engineering, 87:18–31, 2016. Search in Google Scholar

16. S. Wanner and B. Goldluecke. Variational light field analysis for disparity estimation and super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(3):606–619, 2014. Search in Google Scholar

17. S. Werling, M. Mai, M. Heizmann, and J. Beyerer. Inspection of specular and partially specular surfaces. Metrology and Measurement Systems, 16(3):415–431, 2009. Search in Google Scholar

18. Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, 2000. Search in Google Scholar

Received: 2021-02-15
Accepted: 2021-04-19
Published Online: 2021-04-28
Published in Print: 2021-06-26

© 2021 Walter de Gruyter GmbH, Berlin/Boston