Skip to content
Licensed Unlicensed Requires Authentication Published by Oldenbourg Wissenschaftsverlag July 16, 2021

Sub-micron pupillometry for optical EEG measurements

Sub-Mikrometer Pupillometrie für optische EEG-Messungen
Niels-Ole Rohweder

Niels-Ole Rohweder is a PhD student at the Institute for Electrical Information Technology and the Simulation Science Center of Clausthal University of Technology. He studied Physics and Robotics at the University of Hamburg, and received his Master degree in 2019, working on nonclassical light and laser interferometry. His research interests include the fields of Human-Machine-Interaction and optical sensor systems.

EMAIL logo
, Jan Gertheiss

Jan Gertheiss received his PhD in Statistics from Ludwig Maximilians University, Munich in 2011. After spending some time as a postdoctoral researcher at the Department of Statistics at North Carolina State University, he took over a position as a professor of Biometrics and Bioinformatics at the Department of Animal Sciences at Georg August University, Göttingen in 2012. From 2016 until 2018, he worked as a Statistics professor at Clausthal University of Technology. In 2019, Jan Gertheiss moved to Helmut Schmidt University, Hamburg, where he holds a professorship in Statistics and Data Science as part of the faculty of Economics and Social Sciences. His research interests include statistical and machine learning as well as feature selection for functional, categorical and high-dimensional data.

and Christian Rembe

Prof. Dr. Christian Rembe is professor of applied metrology at Clausthal University of Technology. He completed his physics studies in 1994. He accomplished his diploma thesis at the Institute of Quantum Optics at the University of Hannover. Then, he became a doctoral researcher at the University of Ulm, Germany, where he completed his doctorate in engineering with honors in 1999. For his research in the field of measurement technology for microsystems, he received the doctoral award of the Ulm University Society and the Research Award for Applied Sciences of Baden-Württemberg in 1999. After a two-year research stay as a Feodor Lynen Fellow at the University of California, Berkeley, he moved to industry in 2001 as head of the Optics Development Department at Polytec GmbH. During this time, he received the AMA Innovation Award twice. In 2015, he was appointed to the professorship for applied metrology at the Clausthal University of Technology, where his main fields of work are vibration measurement technology, optical metrology, microsensors and production metrology.

From the journal tm - Technisches Messen

Abstract

Recent research indicates that a direct correlation exists between brain activity and oscillations of the pupil. A publication by Park and Whang shows measurements of excitations in the frequency range below 1 Hz. A similar correlation for frequencies between 1 Hz and 40 Hz has not yet been clarified. In order to evaluate small oscillations, a pupillometer with a spatial resolution of 1 µm is required, exceeding the specifications of existing systems. In this paper, we present a setup able to measure with such a resolution. We consider noise sources, and identify the quantisation noise due to finite pixel sizes as the fundamental noise source. We present a model to describe the quantisation noise, and show that our algorithm to measure the pupil diameter achieves a sub-pixel resolution of about half a pixel of the image or 12 µm. We further consider the processing gains from transforming the diameter time series into frequency space, and subsequently show that we can achieve a sub-micron resolution when measuring pupil oscillations, surpassing established pupillometry systems. This setup could allow for the development of a functional optical, fully-remote electroencephalograph (EEG). Such a device could be a valuable sensor in many areas of AI-based human-machine-interaction.

Zusammenfassung

Aktuelle Forschungsergebnisse deuten auf eine direkte Korrelation zwischen Gehirnaktivität und Pupillenoszillationen hin. Eine Veröffentlichung von Park und Whang untersucht Schwingungen im Frequenzbereich unter 1 Hz; eine ähnliche Korrelation bei Frequenzen zwischen 1 und 40 Hz ist noch ungeklärt. Zum Nachweis dieser Oszillationen wird jedoch ein Pupillometer mit einer Auflösung von 1 µm benötigt, was die Spezifikationen bestehender Systeme überschreitet. In diesem Beitrag präsentieren wir einen Aufbau, welcher mit einer solchen Auflösung messen kann. Wir betrachten mögliche Rauschquellen, und identifizieren das Quantisierungsrauschen, welches aufgrund der endlichen Pixelgröße der Kamera entsteht, als den fundamentalen Rauschbeitrag. Wir präsentieren ein theoretisches Modell, um das Quantisierungsrauschen der Kamera zu beschreiben, und zeigen, dass unser Algorithmus zur Bestimmung des Pupillendurchmessers eine Subpixel-Auflösung von ca. einem halben Bildpixel oder 12 µm erzielt. Für die Zeitreihen der Durchmesser betrachten wir den Verarbeitungsgewinn durch die Transformation in den Frequenzraum. Damit zeigen wir schließlich, wie eine sub-Mikrometer Auflösung bei der Messung von Pupillenoszillationen erreicht werden kann, und etablierte Pupillometriesysteme übertroffen werden. Dieser Aufbau könnte es ermöglichen, einen funktionsfähigen optischen, vollständig drahtlosen Elektroenzephalograph (EEG) zu entwickeln. Ein solches Gerät könnte ein wertvoller Sensor in vielen Bereichen KI-gestützter Mensch-Maschine-Interaktion sein.

About the authors

Niels-Ole Rohweder

Niels-Ole Rohweder is a PhD student at the Institute for Electrical Information Technology and the Simulation Science Center of Clausthal University of Technology. He studied Physics and Robotics at the University of Hamburg, and received his Master degree in 2019, working on nonclassical light and laser interferometry. His research interests include the fields of Human-Machine-Interaction and optical sensor systems.

Jan Gertheiss

Jan Gertheiss received his PhD in Statistics from Ludwig Maximilians University, Munich in 2011. After spending some time as a postdoctoral researcher at the Department of Statistics at North Carolina State University, he took over a position as a professor of Biometrics and Bioinformatics at the Department of Animal Sciences at Georg August University, Göttingen in 2012. From 2016 until 2018, he worked as a Statistics professor at Clausthal University of Technology. In 2019, Jan Gertheiss moved to Helmut Schmidt University, Hamburg, where he holds a professorship in Statistics and Data Science as part of the faculty of Economics and Social Sciences. His research interests include statistical and machine learning as well as feature selection for functional, categorical and high-dimensional data.

Christian Rembe

Prof. Dr. Christian Rembe is professor of applied metrology at Clausthal University of Technology. He completed his physics studies in 1994. He accomplished his diploma thesis at the Institute of Quantum Optics at the University of Hannover. Then, he became a doctoral researcher at the University of Ulm, Germany, where he completed his doctorate in engineering with honors in 1999. For his research in the field of measurement technology for microsystems, he received the doctoral award of the Ulm University Society and the Research Award for Applied Sciences of Baden-Württemberg in 1999. After a two-year research stay as a Feodor Lynen Fellow at the University of California, Berkeley, he moved to industry in 2001 as head of the Optics Development Department at Polytec GmbH. During this time, he received the AMA Innovation Award twice. In 2015, he was appointed to the professorship for applied metrology at the Clausthal University of Technology, where his main fields of work are vibration measurement technology, optical metrology, microsensors and production metrology.

References

1. L. G. Yeo et al., Mobile EEG-based situation awareness recognition for air traffic controllers, in: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, 2017.10.1109/SMC.2017.8123090Search in Google Scholar

2. B. Allison, I. Sugiarto, B. Graimann, A. Gräser, Display optimization in SSVEP BCIs, in: Computer-Human Interaction, 2008, pp. 2–5.Search in Google Scholar

3. F. Lotte et al., A review of classification algorithms for EEG-based brain–computer interfaces: a 10 year update, Journal of neural engineering, 15(3):031005, 2018.10.1088/1741-2552/aab2f2Search in Google Scholar PubMed

4. C. Berka, D. J. Levendowski, M. M. Cvetinovic, M. M. Petrovic, G. Davis, M. N. Lumicao, V. T. Zivkovic, M. V. Popovic, R. Olmsted, Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset, International Journal of Human-Computer Interaction, 17(2):151–170, 2004.10.1207/s15327590ijhc1702_3Search in Google Scholar

5. V. Mihajlović, B. Grundlehner, R. Vullers, J. Penders, Wearable, wireless EEG solutions in daily life applications: What are we missing? IEEE J. Biomed. Health Inform., 19(1):6–21, 2015.10.1109/JBHI.2014.2328317Search in Google Scholar PubMed

6. P. A. Freer, G. K. Freer, Non-contact body and head based monitoring of brain electrical activity, U.S. Patent Application No. 15/497,734, 2017.Search in Google Scholar

7. S. Park, M. Whang, Infrared camera-based non-contact measurement of brain activity from pupillary rhythms, Frontiers in physiology, 9:1400, 2018.10.3389/fphys.2018.01400Search in Google Scholar PubMed PubMed Central

8. N.-O. Rohweder, C. Rembe, J. Gertheiss, Towards a remote EEG for use in robotic sensors, in: Forum Bildverarbeitung 2020, KIT Scientific Publishing, 2020, pp. 225–237.Search in Google Scholar

9. B. Alhaji, J. Beecken, R. Ehlers, J. Gertheiss, F. Merz, J. P. Müller, M. Prilla, A. Rausch, A. Reinhardt, D. Reinhardt, C. Rembe, N.-O. Rohweder, Engineering Human–Machine Teams for Trusted Collaboration, Big Data and Cognitive Computing, 4(4):35, 2020.10.3390/bdcc4040035Search in Google Scholar

10. K. Tekin, M. A. Sekeroglu, H. Kiziltoprak, S. Doguizi, M. Inanc, P. Yilmazbas, Static and dynamic pupillometry data of healthy individuals, Clinical and Experimental Optometry, 101(5):659–665, 2018.10.1111/cxo.12659Search in Google Scholar PubMed

11. L. Stark, Stability, Oscillations, and Noise in the Human Pupil Servomechanism, Proc. IRE, 47(11):1925–1939, 1959.10.1109/JRPROC.1959.287206Search in Google Scholar

12. M. Naber, G. A. Alvarez, K. Nakayama, Tracking the allocation of attention using human pupillary oscillations, Frontiers in Psychology, 4:919, 2013.10.3389/fpsyg.2013.00919Search in Google Scholar PubMed PubMed Central

13. P. A. Barrionuevo, N. Nicandro, J. J. McAnany, A. J. Zele, P. Gamlin, D. Cao, Assessing Rod, Cone, and Melanopsin Contributions to Human Pupil Flicker Responses, Investigative Ophthalmology & Visual Science, 55(2):719–727, 2014.10.1167/iovs.13-13252Search in Google Scholar PubMed PubMed Central

14. W. Nowak, A. Żarowska, E. Szul-Pietrzak, M. Misiuk-Hojło, System and measurement method for binocular pupillometry to study pupil size variability, Biomed Eng Online, 13:69, 2014.10.1186/1475-925X-13-69Search in Google Scholar PubMed PubMed Central

15. A. Hachol, W. Szczepanowska-Nowak, H. Kasprzak, I. Zawojska, A. Dudzinski, R. Kinasz, D. Wygledowska-Promienska, Measurement of pupil reactivity using fast pupillometry, Physiol. Meas., 28(1):61–72, 2007.10.1088/0967-3334/28/1/006Search in Google Scholar PubMed

16. A. B. Roig, M. Morales, J. Espinosa, J. Perez, D. Mas, C. Illueca, Pupil detection and tracking for analysis of fixational eye micromovements, Optik, 123(1):11–15, 2012.10.1016/j.ijleo.2010.10.049Search in Google Scholar

17. S. Meimon, J. Jarosz, G. Chenegros, C. Petit, J.-M. Conan, M. Paques, High frequency pupillometry, in: Ophthalmic Technologies XXIV, 2015, p. 89300F.10.1117/12.2042917Search in Google Scholar

18. G. Borghini, V. Hazan, Listening Effort During Sentence Processing Is Increased for Non-native Listeners: A Pupillometry Study, Front. Neurosci., 12:152, 2018.10.3389/fnins.2018.00152Search in Google Scholar PubMed PubMed Central

19. F. Maqsood, Effects of varying light conditions and refractive error on pupil size, Cogent Medicine, 4(1):1338824, 2017.10.1080/2331205X.2017.1338824Search in Google Scholar

20. S.-C. Zhang, Z.-Q. Liu, A robust, real-time ellipse detector, Pattern Recognition, 38(2):273–287, 2005.10.1016/j.patcog.2004.03.014Search in Google Scholar

21. A. M. Fernandes, Discussion on paper “A Robust Real-Time Ellipse Detector” by Zhang and Liu, Pattern Recognition, 44(2):488–489, 2011.10.1016/j.patcog.2010.08.031Search in Google Scholar

22. O. Lowenstein, R. Feinberg, I. E. Loewenfeld, Pupillary movements during acute and chronic fatigue, Invest Ophthalmol, 2:138–157, 1963.Search in Google Scholar

23. W. R. Bennett, Spectra of Quantized Signals, Bell System Technical Journal, 27:446–471, 1948.10.1002/j.1538-7305.1948.tb01340.xSearch in Google Scholar

24. W. Kester, Taking the mystery out of the infamous formula, “snr = 6.02 n + 1.76 db,” and why you should care, Analog Devices, MT-001, 2008.Search in Google Scholar

25. N. Schuster, Messen mit Subpixel-Genauigkeit: Legende und Wirklichkeit, Optik & Photonik, 2(3):49–52, 2007.10.1002/opph.201190271Search in Google Scholar

26. F.-B. Vialatte, M. Maurice, J. Dauwels, A. Cichocki, Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives, Progress in Neurobiology, 90(4):418–438, 2010.10.1016/j.pneurobio.2009.11.005Search in Google Scholar PubMed

Received: 2021-02-19
Accepted: 2021-06-29
Published Online: 2021-07-16
Published in Print: 2021-08-27

© 2021 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 9.12.2022 from https://www.degruyter.com/document/doi/10.1515/teme-2021-0030/html
Scroll Up Arrow