Direct-imaging DOEs for high-NA multi-spot confocal microscopy

Zheng Li 1 , 2 , Miro Taphanel 1 , 2 , Thomas Längle 2 , and Jürgen Beyerer 1 , 2
  • 1 Lehrstuhl für Interaktive Echtzeitsysteme (IES), Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany
  • 2 Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB, Karlsruhe, Germany
Zheng Li
  • Corresponding author
  • Lehrstuhl für Interaktive Echtzeitsysteme (IES), Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany
  • Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB, Karlsruhe, Germany
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Miro Taphanel
  • Lehrstuhl für Interaktive Echtzeitsysteme (IES), Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany
  • Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB, Karlsruhe, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Thomas Längle
  • Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB, Karlsruhe, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Jürgen Beyerer
  • Lehrstuhl für Interaktive Echtzeitsysteme (IES), Karlsruher Institut für Technologie (KIT), Karlsruhe, Germany
  • Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB, Karlsruhe, Germany
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

Diffractive lens arrays with overlapping apertures can produce spots with high numerical apertures (NAs). Such diffractive optical elements (DOEs) can replace high-NA objectives and measure a large area with high resolution in transmission microscopes. However, in reflection microscopes for surface measurements, the axial resolution is still limited by the objectives. Direct-imaging DOEs are proposed to solve the problem. They can perform high-NA multi-spot measurement in reflection configurations in both lateral and axial directions. Experiments demonstrate a lateral non-vanishing contrast up to 1448 lp/mm and an axial response on a plane mirror with a full width at half maximum (FWHM) of 2.24 μm.

1 Introduction

Traditional confocal microscopy has an unavoidable tradeoff between resolution and field-of-views (FOVs). Objectives with high NAs usually have high magnifications as well, which lead to very limited FOVs. Thus, only a very small area can be measured by these objectives at once. Although there exist high-NA objectives with large FOVs like lithography lenses, they are extremely expensive and difficult to manufacture, which makes them impractical for application in microscopy [9].

In previous researches, the diffractive lens arrays with overlapping apertures are proposed to solve the problem [2, 8]. Unlike the traditional micro lens arrays, they release the restriction between the pitches and the NAs of the micro lenses. They can produce high-NA spots in a dense grid for illumination. Combined with low-NA objectives with large FOVs in the imaging path, they can perform highresolution confocal scanning over a large area. These DOEs are only used in transmission microscopes at first. Later they are modified to be used in reflection microscopes as well [4], which become suitable for opaque surface measurement or fluorescence microscopy. However, for plane surfaces, the axial resolution of such configurations with high-NA illumination and low-NA imaging is still fundamentally limited by the low-NA objective, which leads to limitations in surface measurement.

To overcome the limitations, Direct-imaging DOEs are proposed to improve the axial resolution while maintaining the lateral resolution in multi-spot confocal microscopy [5]. They act exactly as high-NA finite-conjugate objectives, but they can cover a large area under certain arrangements and significantly reduce the cost by mass-production lithography. They are designed by superposition of a diffractive lens array which produces aspot array by plane-wave illumination, and another lens array which images the spots to an intermediate image or directly onto the camera sensor. In the following sections, the design processes of the Direct-imaging DOEs are introduced, and experiments are carried out to demonstrate the capabilities of them for surface measurement. Experiments of measuring a resolution target and a plane mirror show both, high lateral and the axial resolution of the proposed DOEs.

2 Design and simulation

The design of such DOEs starts from building a unit element by the principle of the so-called multi-functional DOEs [1]. By simply adding different field distributions generated from different target light patterns together, all the target patterns can be generated simultaneously with one single DOE. The construction of the unit element is shown in Fig. 1.

Fig. 1
Fig. 1

Design of the unit element by superposition of two field distributions.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

The unit element is composed of two lenses, one is the illumination lens which focuses the plane wave illumination into a spot, and the other one is the imaging lens which images the spot to an intermediate spot above. The field distribution of the unit element uunit is expressed as

uunit=u1+Wu2,

where u1 is the field distribution of the illumination lens, u2 is the field distribution of the imaging lens, and W is a constant ratio which needs to be optimized later.

u1 and u2 are calculated by diffraction propagation with Rayleigh-Sommerfeld integral [6, 7], since in high-NA applications, traditional diffraction simulation methods like the Fresnel approximation are no longer valid. For example, u1 is obtained by propagating uspot through a certain working distance d, which is expressed as

u1(r)=RS(uspot,d)=uspot(r)eik|rr|d|rr|2dxdy,

where k = 2π/λ is the wavenumber, λ is the wavelength, r = (x, y, z) is the coordinate on the DOE plane at z = d if we assume the plane of uspot is at z = 0 in Fig. 1, Σ denotes the surface on the boundary, i.e. the plane of uspot and the semi-infinite sphere above it, r' = (x', y', z') is the coordinate on Σ. Similarly, u2 can be calculated as

u2(r)=RS(uspot',d)¯/RS(uspot,d),

where uspot'is the designed intermediate spot, i.e. an airy disc, while d' is the designed distance from the DOE plane to the intermediate image shown in Fig. 1, and the complex conjugate is necessary to reverse the divergent wave into a convergent wave in order to form the intermediate spot.

By the above-mentioned process, a unit element is obtained, which can produce a focused spot by planewave illumination and then image the spot back to an intermediate spot above by itself. Afterwards, the unit element is replicated and overlapped with a certain pitch to create the total field distribution uD of the DOEs, which is shown in Fig. 2. Then the phase of the field distribution is extracted and binarized as the following equation shows

Fig. 2
Fig. 2

Design of the Direct-imaging DOEs by overlapping and binarization.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

ϕD=mod(arg(uD)+Bπ,2)π,

where B is the binarization factor. Then the binarized phase is again examined by simulation and the parameters such as W and B are optimized iteratively to achieve the highest signal-to-noise ratio in the intermediate image.

In this way, the Direct-imaging DOEs with different number of spots, pitches and working distances are designed. Later, according to the design, a binary DOE prototype is produced by e-beam lithography by Fraunhofer IOF. The prototype is tested and the results are shown in the following section.

3 Experimental Results

An experiment setup for testing the DOE prototype is built and shown in Fig. 3. The prototype is designed for the wavelength of 785 nm. A 785 nm fiber-coupled diode laser from Thorlabs is used as the light source. Its light is reflected by a beam splitter and collimated by a 5× objective with an NA of 0.15. The collimated light, which can be considered as plane wave, illuminates the DOEs. The produced spots are measured. For one DOE pattern which produces a 5×5 spot array with a pitch of 100 μm, the measured lateral FWHM of the central spot is 0.57 μm and the axial FWHM is 2.61 μm, which roughly corresponds to an NA of 0.68 [3]. Fig. 4 shows the measured spot size.

Fig. 3
Fig. 3

Setup of the DOE-based confocal microscope.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

Fig. 4
Fig. 4

Size of the produced central spot in the 5×5 spot array with 100 μm pitch. (a) Lateral size of the spot, FWHM = 0.57 μm. (b) Axial size of the spot, FWHM = 2.61 μm.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

Afterwards, a test sample is placed underneath. The produced illumination spots are imaged by the DOE itself to an intermediate image. The magnification of the micro lens elements in the DOE is about 16×. Then the intermediate image is imaged by the 5× objective and a 1× tube lens onto the camera sensor, which is shown as the schematic in Fig. 3. In such a configuration, the DOE acts exactly as a high-NA finite-conjugate objective and it can provide high resolution in both lateral and axial directions for surface measurement.

A high-resolution calibration target from Newport is laterally scanned with the 5×5 spot array produced by the above-mentioned DOE. The scanning step is 0.2 μm. The results are shown in Fig. 5. There are some stitching artifacts due to the non-orthogonality of the two axes of the piezo stage, and the stripes are possibly due to the power drifting and interference of the laser. Nevertheless, compared to the wide-field image in Fig. 6, which is taken by the 5× objective alone, it is clear that the DOE significantly increases the lateral resolution up to a non-vanishing contrast at 1448 lp/mm.

Fig. 5
Fig. 5

Confocal scanning image of the resolution target by a Direct-imaging DOE with 5×5 spot array and 100 μm pitch.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

Fig. 6
Fig. 6

Wide-field image of the resolution target by a 5× objective with 0.15 NA.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

Furthermore, a plane mirror is placed underneath the DOEs and it is scanned axially to test the axial resolution of the setup for surface measurement. The confocal axial response of the central spot in the 5×5 spot array with 100 μm pitch is shown in Fig. 7(a). A FWHM of 2.24 μm is measured. However, the signal will become noisy with an increasing number of spots due to cross-talk among the micro lenses in the DOEs [4]. On the one hand, for a single unit element, when the illumination lens projects the spots, the imaging lens also produces an out-of-focus spot around the original one. The blurred spot will cause noise in the intermediate image. On the other hand, the spot produced by the unit element will not only be imaged by itself, but also it will be imaged by the adjacent elements and eventually become part of the background noise in the intermediate image. Thus, further increase of the number of spots will reduce the signal-to-noise ratio and make the spots in the intermediate image indistinguishable [5]. By changing the spot arrangement, for example, into a line, increasing the pitch of the array, or decreasing the NAs of the DOEs, the cross-talk can be reduced. For example, the axial response of the central spot in a 25×25 spot array with 0.5 NA and 200 μm pitch is measured in Fig. 7(b), which shows that with a larger pitch the spot can still be distinguished in such a large grid.

Fig. 7
Fig. 7

Confocal axial response of a plane mirror. (a) Central spot in the 5×5 spot array with 0.68 NA and 100 μm pitch, FWHM = 2.24 μm. (b) Central spot in the 25×25 spot array with 0.5 NA and 200 μm pitch, FWHM = 3.46 μm.

Citation: tm - Technisches Messen 87, s1; 10.1515/teme-2020-0017

4 Conclusion

Microscope objectives have unavoidable trade-offs between resolution and FOVs. In previous researches, DOEs with overlapping apertures are proposed to replace high-NA objectives in confocal microscopy to increase the FOVs and reduce the cost while maintaining high lateral resolution. However, for surface measurement, the axial resolution is still limited by the imaging objectives. In this work, the idea of multi-functional DOEs is used to overcome this limit and to increase the axial resolution of the DOEs for opaque surface measurements. The proposed Directimaging DOEs have the same discerning capability as traditional high-NA objectives in both lateral and axial directions.

A prototype of the DOEs is manufactured and tested. Experiments show that spots with NA = 0.68 can be produced by the designed DOEs. Measurement of a resolution target demonstrates a lateral resolution up to 1448 lp/mm of the setup. And a confocal axial response by a plane mirror with a FWHM of 2.24 μm is measured. However, with the increase in the number of spots in the array, the image will suffer from severe disturbances, which leads to a poor signal-to-noise ratio. By simulation, 1D arrangement of the spots can reduce the disturbances and the setup can be used as as a line scanning device [4]. Having a sparser spot array also relieves the problem, which is demonstrated in the experiment by a 25×25 spot array with 0.5 NA.

In the future, application of the DOEs in fluorescence microscopy for high-resolution large-area measurement will be investigated and new possibilities to combine the DOEs with structured illumination and interference will be explored.

References

  • [1]

    E. Dai, C. Zhou, P. Xi und L. Liu. Multifunctional doublelayered diffractive optical element. Optics letters, 28(17): 1513–1515, 2003.

    • Crossref
    • Export Citation
  • [2]

    B. Hulsken, D. Vossen und S. Stallinga. High NA diffractive array illuminators and application in a multi-spot scanning microscope. Journal of the European Optical Society-Rapid publications, 7, 2012.

  • [3]

    H. Kirshner, D. Sage und M. Unser. 3D PSF models for fluorescence microscopy in ImageJ. In Proceedings of the Twelfth International Conference on Methods and Applications of Fluorescence Spectroscopy, Imaging and Probes (MAF’11), S. 154, Strasbourg, French Republic, September 11-14, 2011.

  • [4]

    Z. Li. Application of diffractive optical elements in confocal microscopy. In M. Taphanel und J. Beyerer, editors, Proceedings of the 2018 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. KIT Scientific Publishing, Karlsruhe, 2019.

  • [5]

    Z. Li, M. Taphanel, T. Längle und J. Beyerer. Application of DOE in confocal microscopy for surface measurement. In M. Rosenberger, P.-G. Dittrich und B. Zagar, editors, IMEKO Joint TC1 - TC2 International Symposium on Photonics and Education in Measurement Science, Band 11144, S. 254 – 261. International Society for Optics and Photonics, SPIE, 2019.

  • [6]

    A. Sommerfeld. Mathematische Theorie der Diffraction. Mathematische Annalen, 47(2):317–374, 1896.

    • Crossref
    • Export Citation
  • [7]

    A. Sommerfeld. Mathematical Theory of Diffraction. Birkhäuser Boston, Boston, MA, 2004. ISBN 978-08176-8196-8. 10.1007/978-0-8176-8196-8_2. URL https://doi.org/10.1007/978-0-8176-8196-8_2

  • [8]

    T. Stenau und K.-H. Brenner. Diffractive lenses with overlapping aperture a new tool in scanning microscopy. In Imaging Systems and Applications, S. IT1F–1. Optical Society of America, 2016.

  • [9]

    G. Zheng. Fourier Ptychographic Imaging: A Matlab Tutorial. Morgan & Claypool Publishers, 2016.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • [1]

    E. Dai, C. Zhou, P. Xi und L. Liu. Multifunctional doublelayered diffractive optical element. Optics letters, 28(17): 1513–1515, 2003.

    • Crossref
    • Export Citation
  • [2]

    B. Hulsken, D. Vossen und S. Stallinga. High NA diffractive array illuminators and application in a multi-spot scanning microscope. Journal of the European Optical Society-Rapid publications, 7, 2012.

  • [3]

    H. Kirshner, D. Sage und M. Unser. 3D PSF models for fluorescence microscopy in ImageJ. In Proceedings of the Twelfth International Conference on Methods and Applications of Fluorescence Spectroscopy, Imaging and Probes (MAF’11), S. 154, Strasbourg, French Republic, September 11-14, 2011.

  • [4]

    Z. Li. Application of diffractive optical elements in confocal microscopy. In M. Taphanel und J. Beyerer, editors, Proceedings of the 2018 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. KIT Scientific Publishing, Karlsruhe, 2019.

  • [5]

    Z. Li, M. Taphanel, T. Längle und J. Beyerer. Application of DOE in confocal microscopy for surface measurement. In M. Rosenberger, P.-G. Dittrich und B. Zagar, editors, IMEKO Joint TC1 - TC2 International Symposium on Photonics and Education in Measurement Science, Band 11144, S. 254 – 261. International Society for Optics and Photonics, SPIE, 2019.

  • [6]

    A. Sommerfeld. Mathematische Theorie der Diffraction. Mathematische Annalen, 47(2):317–374, 1896.

    • Crossref
    • Export Citation
  • [7]

    A. Sommerfeld. Mathematical Theory of Diffraction. Birkhäuser Boston, Boston, MA, 2004. ISBN 978-08176-8196-8. 10.1007/978-0-8176-8196-8_2. URL https://doi.org/10.1007/978-0-8176-8196-8_2

  • [8]

    T. Stenau und K.-H. Brenner. Diffractive lenses with overlapping aperture a new tool in scanning microscopy. In Imaging Systems and Applications, S. IT1F–1. Optical Society of America, 2016.

  • [9]

    G. Zheng. Fourier Ptychographic Imaging: A Matlab Tutorial. Morgan & Claypool Publishers, 2016.

FREE ACCESS

Journal + Issues

Search

  • View in gallery

    Design of the unit element by superposition of two field distributions.

  • View in gallery

    Design of the Direct-imaging DOEs by overlapping and binarization.

  • View in gallery

    Setup of the DOE-based confocal microscope.

  • View in gallery

    Size of the produced central spot in the 5×5 spot array with 100 μm pitch. (a) Lateral size of the spot, FWHM = 0.57 μm. (b) Axial size of the spot, FWHM = 2.61 μm.

  • View in gallery

    Confocal scanning image of the resolution target by a Direct-imaging DOE with 5×5 spot array and 100 μm pitch.

  • View in gallery

    Wide-field image of the resolution target by a 5× objective with 0.15 NA.

  • View in gallery

    Confocal axial response of a plane mirror. (a) Central spot in the 5×5 spot array with 0.68 NA and 100 μm pitch, FWHM = 2.24 μm. (b) Central spot in the 25×25 spot array with 0.5 NA and 200 μm pitch, FWHM = 3.46 μm.