It is well-known that a Fourier optical system can be used to perform specific computing tasks, such as image differentiation, with a superior speed and power consumption in comparison with digital computers, despite bulky optical components that are often required. Recently, there has been a surge in the interest to design much more compact nanophotonic structures, such as dielectric and metallic thin films, photonic crystals, and metasurfaces with a tailored angle-dependent (nonlocal) optical response, to directly perform image differentiation without additional lenses for Fourier transformation. Here, we present a straightforward platform, a multilayer dielectric thin film optical filter, fabricated using mature wafer-scale thin film deposition technique, with an optimized nonlocal optical response, for isotropic image differentiation in transmission mode for arbitrary input polarization. The proposed thin film filter may be conveniently coated at various transparent surfaces and inserted in machine vision or microscopy systems for enhanced, real-time image processing.
Using spatial differentiation of an image, one can selectively amplify the high-spatial-frequency component of the image, leading to enhancements of the edge information . Since edges typically contain the most important geometric features of an image, edge detection has been widely adopted for image processing towards applications in feature classification , target recognition , and data compression  in a variety of imaging settings.
Conventionally, image differentiation is performed digitally with a limited computation speed and large power consumption that may fail to meet the ever-increasing demand for real-time image processing towards emerging applications such as autonomous driving . Alternatively, it has long been known that image differentiation can be done optically by filtering out the low wavevector component of the impinging light using an amplitude mask placed at the center of the Fourier plane of a 4-f imaging system , thus allowing the massively parallel processing of an image with minimal power consumption. However, the Fourier filter system may require bulky lenses for the Fourier transformation as well as precise positioning of an amplitude mask at the Fourier plane of the 4-f system.
Recently, a Green’s function approach was proposed that may directly perform spatial filtering of an image without Fourier transformation, by leveraging a tailored angle-dependent (nonlocal) response of an optical filter . This approach allows the potential miniaturization of the image differentiation system and does not require precision positioning of the optical filter, thus has attracted much attention in recent years [8, 9]. For instance, a multilayer thin film composed of metamaterial of permittivity ranging from −2.12 to 13.85 was theoretically proposed to realize a Green’s function kernel , leading to 2nd-order spatial differentiation . Later, several groups proposed to perform image differentiation using single- or multi-layer thin film made of more readily available dielectric or metallic materials, albeit they may be direction- , ,  or polarization-sensitive , , , , , , , , , . Some proposals may only work in the reflection mode , , , , , , , [17, 20, 21], leading to challenges to integrate them with conventional imaging systems in a compact form factor. Moreover, image differentiation was demonstrated using nanostructured gratings , , , photonic crystals , and metasurfaces , , , , , , , with some designs capable of two-dimensional (2D) transmissive image differentiation and being polarization insensitive [31, 32]. However, the fabrication of large-area photonic crystal or metasurface samples with subwavelength features generally requires costly, time-consuming, or unconventional lithography processes that may hinder their imminent applications.
Here, we design and experimentally demonstrate a multilayer thin film optical filter for image differentiation. The filter is designed to operate in the transmission mode to facilitate convenient integration with conventional machine vision or microscopy imaging systems. Due to its rotational symmetry, the planar thin film structure naturally possesses an isotropic optical response for 2D image formation. In addition, the angle-dependent response of the filter can be optimized for both s- and p-polarized light to allow image differentiation for arbitrary polarization. More importantly, since the proposed structure is simply made of alternating dielectric layers, it can be manufactured at the wafer-scale on a variety of transparent surfaces with a relatively low cost, leveraging widely available, mature thin film deposition techniques.
2 Results and discussions
The basic principle of thin film-based image differentiation is schematically shown in Figure 1. For an object (here, a logo of Tsinghua University) illuminated by a coherent beam, the scattered light field is modulated by the multilayer thin film with an angle/wavenumber-dependent optical transfer function , following the function , where and represent the field profiles of the incident and transmitted beams, x and y indicate spatial coordinates, k x and k y denote the wavevector in the x- and y-direction, respectively. To perform the detection of edge information in 2D for both s- and p-polarized light incident along the z-direction, an ideal optical transfer function is given by,
where α s and α p are constant related to the transmission coefficient of the s- and p-polarized incident light, respectively.
It is well-known that a multilayer dielectric thin film can have a strongly angle-dependent transmission resulting from the multiple interference effect [33, 34]; however, it is not intuitive to identify the required thin film parameter to realize the wavenumber-dependent optical transfer function as specified in Eq. (1) from a rigorous physical model. Therefore, we resort to optimization algorithms in conjunction with the transfer matrix method, with fabrication constraints taken into consideration.
Figure 2(a) schematically shows the flow of the iterative optimization process of a multilayer thin film optical filter, which consisting of alternating dielectric films supported on a substrate. The optimization algorithm we chose is the widely used particle swarm optimization (PSO) algorithm , although alternative methods such as the genetic algorithm , adjoint method , or deep learning  may also be used. With the total number of layers and the thickness of each layer set as input variables, the goal of the optimization is to minimize the loss function such that the angle-dependent transmission coefficient of the multilayer film can approach the ideal transfer function. We define the loss function of the optimization as,
where t s and t p represent the designed optical transfer function of s- and p-polarized light, respectively. Here, we further impose an upper limit of the total film thickness of 15 μm and single-layer thickness of 1 μm, to prevent excessive strain and cracking of the fabricated multilayer film.
We choose SiO2 and TiO2 as the alternating dielectric materials due to their transparency in the visible spectrum, their relatively large refractive index contrast (n SiO2 = 1.46 and n TiO2 = 2.33 at a wavelength of 532 nm), as well as the mature thin film deposition techniques. The optimized geometric parameters of a 13-pair SiO2/TiO2 thin film are shown in Figure 2(b), with the calculated transmission coefficient as a function of wavelength and incident angle for s- and p-polarized light shown in Figure 2(c) and (d), respectively. As shown in Figure 2(e), at the designed wavelength of 532 nm, the optimized thin film stack has an angle-dependent transmission coefficient in close agreement with the ideal optical transfer function for both s- and p-polarized light, up to a numerical aperture (NA) of 0.31, which is among the largest of experimentally demonstrated nanophotonic image differentiators [10, 24, 29, 31]. For k x /k 0 smaller than 0.1, the optimized filter response slightly deviates from the ideal one, resulting in background noise in the differentiated image.
It is well known that stronger spatial dispersion can be obtained in photonic crystals made of materials with larger refractive index contrast . A stronger spatial dispersion can consequently provide more degrees of freedom for optimizing the angle-dependent transmittance of a multilayer film. Therefore, to further improve the filter performance, to achieve a larger NA or a reduced background noise, one may resort to material combination with a larger index contrast. For instance, in the near-infrared frequency, one may use the combination of SiO2 and Si (n SiO2 = 1.45 and n Si = 3.64 + 0.0042i at a wavelength of 828 nm ). Figure 2(f) shows the optimized geometric parameters of a 17-layer SiO2/Si thin film for image differentiation with a NA up to 0.6, using a modified ideal optical transfer function . Notably, it has been shown that any optical transfer function may be used for image differentiation as long as they can effectively suppress the low-frequency component of the image and amplify the high-frequency component . Figure 2(g)–(i) show the calculated transmission coefficient for the SiO2/Si multilayer stack as a function of wavelength and incident angle for s- and p-polarized light, respectively, with good agreement with the ideal optical transfer function, including for k x /k 0 close to zero.
Figure 3 shows the simulated image differentiation performance of both the SiO2/TiO2 and the SiO2/Si multilayer stack. We assume rectangles with width ranging from 0.8 to 12 μm as the input object (Figure 3(a) and (c)). Subsequently, differentiated images can be simulated through the convolution of the input object with the designed transfer function of the multilayer thin film filter. Figure 3(b) shown the simulated image modulated by the optimized 13-pair SiO2/TiO2 thin film. A vertical cut of the modulated image, as shown in Figure 3(c), illustrates that the SiO2/TiO2 multilayer thin film can reveal edges of the rectangle in all directions, with some background noise and a spatial resolution of about 2 μm. With an improved filter response, Figure 3(e) shown the simulated image modulated by the optimized 17-layer SiO2/Si thin film. A vertical cut of the modulated image, as shown in Figure 3(f), illustrates that the SiO2/Si multilayer thin film can also reveal edges of the rectangle in all directions, but with much-reduced background noise and a spatial resolution of about 0.8 μm.
Experimentally, based on the SiO2/TiO2 design, we deposit the multilayer thin film on a two-inch-diameter fused silica substrate using ion-assisted deposition available from a commercial service (Beijing Qifenglanda Inc.), with an optical image of the fabricated thin film filter shown in Figure 4(a). To verify the optical response of the fabricated filter, its angle- and polarization-dependent transmittance is measured and superimposed on the designed transmittance curve, as shown in Figure 4(b), showing good agreement.
Subsequently, we built a setup that can perform typical bright-field imaging of a test object (here, a chrome mask of the Tsinghua University logo) with a minimum feature size of about 2 μm, as shown in Figure 4(c). The logo is illuminated by an expanded unpolarized laser beam with a central wavelength of 532 nm and a bandwidth of 4 nm, generated using a supercontinuum laser (YSL SC-PRO-7) coupled with an acoustic-optical tunable filter (YSL AOTF0019). A magnified bright-field image of the test object was then obtained with the combination of an objective lens (NA = 0.42), a tube lens, and a visible camera, as shown in Figure 4(d).
To perform image differentiation, the multilayer thin film filter is simply inserted between the test object and the objective lens, with no need for precise alignment. Figure 4(e) shows the differentiated image that selectively enhances the edge information of the Tsinghua University logo. The filter can differentiate edges with separation down to about 3 µm, evident from the zoom-in differentiated image shown in Figure 4(f) and (g).
In conclusion, we experimentally demonstrate a multilayer thin film optical filter with an optimized wavenumber-dependent optical transfer function for image differentiation. Compared to its alternatives, this proposal can distinctively allow 2D image differentiation for arbitrary incident polarization with a high resolution in the transmission mode. The filter can be readily manufactured at a large scale, leveraging mature technology. The current design is narrowband, yet it can be easily scaled to other wavelengths. One can further design filters for more sophisticated applications, such as space compression, based on a similar concept , , . We envision the thin film optical filter may be conveniently integrated with various existing imaging platforms, for example, coated on the coverslip of a biological sample in microscopy  or in front of an automobile sensing camera for image processing in the physical layer towards a plethora of applications including but not limited to dark field and phase contrast imaging [21, 24, 44], and machine vision.
Funding source: National Natural Science Foundation of Chinadoi.org/10.13039/501100001809
Award Identifier / Grant number: 61975251
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
Research funding: This work was supported by the startup funding provided to Y.Y. by Tsinghua University and by the National Natural Science Foundation of China (Grant No. 61975251).
Conflict of interest statement: The authors declare no competing financial interests.
 J. J. Oram, J. C. McWilliams, and K. D. Stolzenbach, “Gradient-based edge detection and feature classification of sea-surface images of the Southern California Bight,” Remote Sens. Environ., vol. 112, no. 5, pp. 2397–2415, 2008, https://doi.org/10.1016/j.rse.2007.11.010. Search in Google Scholar
 X. Wang, Y. Deng, and H. Duan, “Edge-based target detection for unmanned aerial vehicles using competitive Bird Swarm Algorithm,” Aerosp. Sci. Technol., vol. 78, pp. 708–720, 2018, https://doi.org/10.1016/j.ast.2018.04.047. Search in Google Scholar
 H. S. Hsu and W. H. Tsai, “Moment-preserving edge detection and its application to image data compression,” Opt. Eng., vol. 32, no. 7, pp. 1596–1608, 1993. Search in Google Scholar
 R. C. Gonzalez and R. E. Woods, Digital Image Processing, New Jersey, Pearson prentice Hall, 2008. Search in Google Scholar
 J. W. Goodman, Introduction to Fourier Optics, Greenwoood Village, Roberts & Company Publishers, 2005. Search in Google Scholar
 A. Silva, F. Monticone, G. Castaldi, et al.., “Performing mathematical operations with metamaterials,” Science, vol. 343, no. 6167, pp. 160–163, 2014, https://doi.org/10.1126/science.1242818. Search in Google Scholar
 S. Abdollahramezani, O. Hemmatyar, and A. Adibi, “Meta-optics for spatial optical analog computing,” Nanophotonics, vol. 9, no. 13, pp. 4075–4095, 2020, https://doi.org/10.1515/nanoph-2020-0285. Search in Google Scholar
 F. Zangeneh-Nejad, D. L. Sounas, A. Alù, and R. Fleury, “Analogue computing with metamaterials,” Nat. Rev. Mater., vol. 6, no. 3, pp. 207–225, 2021, https://doi.org/10.1038/s41578-020-00243-2. Search in Google Scholar
 J. Zhang, Q. Ying, and Z. Ruan, “Time response of plasmonic spatial differentiators,” Opt. Lett., vol. 44, no. 18, pp. 4511–4514, 2019, https://doi.org/10.1364/ol.44.004511. Search in Google Scholar
 T. Zhu, Y. Lou, Y. Zhou, et al.., “Generalized spatial differentiation from the spin hall effect of light and its application in image processing of edge detection,” Phys. Rev. Appl., vol. 11, no. 3, p. 034043, 2019, https://doi.org/10.1103/physrevapplied.11.034043. Search in Google Scholar
 A. Youssefi, F. Zangeneh-Nejad, S. Abdollahramezani, and A. Khavasi, “Analog computing by Brewster effect,” Opt. Lett., vol. 41, no. 15, pp. 3467–3470, 2016, https://doi.org/10.1364/ol.41.003467. Search in Google Scholar
 W. Wu, W. Jiang, J. Yang, S. Gong, and Y. Ma, “Multilayered analog optical differentiating device: performance analysis on structural parameters,” Opt. Lett., vol. 42, no. 24, pp. 5270–5273, 2017, https://doi.org/10.1364/ol.42.005270. Search in Google Scholar
 Y. Zhou, R. Chen, W. Chen, R.-P. Chen, and Y. Ma, “Optical analog computing devices designed by deep neural network,” Opt. Commun., vol. 458, p. 124674, 2020, https://doi.org/10.1016/j.optcom.2019.124674. Search in Google Scholar
 T. Zhu, J. Huang, and Z. Ruan, “Optical phase mining by adjustable spatial differentiator,” Adv. Opt. Photonics, vol. 2, no. 1, p. 016001, 2020, https://doi.org/10.1117/1.ap.2.1.016001. Search in Google Scholar
 O. Y. Long, C. Guo, H. Wang, and S. Fan, “Isotropic topological second-order spatial differentiator operating in transmission mode,” Opt. Lett. vol. 46, no. 13, pp. 3247-3250, 2021. https://doi.org/10.1364/ol.430699. Search in Google Scholar
 D. A. Bykov, L. L. Doskolovich, E. A. Bezus, and V. A. Soifer, “Optical computation of the Laplace operator using phase-shifted Bragg grating,” Opt. Express, vol. 22, no. 21, pp. 25084–25092, 2014, https://doi.org/10.1364/oe.22.025084. Search in Google Scholar
 L. Wesemann, E. Panchenko, K. Singh, et al.., “Selective near-perfect absorbing mirror as a spatial frequency filter for optical image processing,” APL Photonics, vol. 4, no. 10, p. 100801, 2019, https://doi.org/10.1063/1.5113650. Search in Google Scholar
 Y. Fang, Y. Lou, and Z. Ruan, “On-grating graphene surface plasmons enabling spatial differentiation in the terahertz region,” Opt. Lett., vol. 42, no. 19, pp. 3840–3843, 2017, https://doi.org/10.1364/ol.42.003840. Search in Google Scholar
 Y. Fang and Z. Ruan, “Optical spatial differentiator for a synthetic three-dimensional optical field,” Opt. Lett., vol. 43, no. 23, pp. 5893–5896, 2018, https://doi.org/10.1364/ol.43.005893. Search in Google Scholar
 L. Wesemann, J. Rickett, J. Song, et al.., “Nanophotonics enhanced coverslip for phase imaging in biology,” Light Sci. Appl., vol. 10, no. 1, p. 98, 2021, https://doi.org/10.1038/s41377-021-00540-7. Search in Google Scholar
 C. Guo, M. Xiao, M. Minkov, Y. Shi, and S. Fan, “Photonic crystal slab Laplace operator for image differentiation,” Optica, vol. 5, no. 3, pp. 251–256, 2018, https://doi.org/10.1364/optica.5.000251. Search in Google Scholar
 H. Kwon, D. Sounas, A. Cordaro, A. Polman, and A. Alù, “Nonlocal metasurfaces for optical signal processing,” Phys. Rev. Lett., vol. 121, no. 17, p. 173004, 2018, https://doi.org/10.1103/physrevlett.121.173004. Search in Google Scholar
 A. Cordaro, H. Kwon, D. Sounas, et al.., “High-index dielectric metasurfaces performing mathematical operations,” Nano Lett., vol. 19, no. 12, pp. 8418–8423, 2019, https://doi.org/10.1021/acs.nanolett.9b02477. Search in Google Scholar
 J. Zhou, H. Qian, C. F. Chen, et al.., “Optical edge detection based on high-efficiency dielectric metasurface,” Proc. Natl. Acad. Sci. USA, vol. 116, no. 23, pp. 11137–11140, 2019, https://doi.org/10.1073/pnas.1820636116. Search in Google Scholar
 Y. Zhou, H. Zheng, I. I. Kravchenko, and J. Valentine, “Flat optics for image differentiation,” Nat. Photonics, vol. 14, no. 5, pp. 316–323, 2020, https://doi.org/10.1038/s41566-020-0591-3. Search in Google Scholar
 J. Zhou, H. Qian, J. Zhao, et al.., “Two-dimensional optical spatial differentiation and high-contrast imaging,” Natl. Sci. Rev., pp. 1–8, 2020, https://doi.org/10.1093/nsr/nwaa176. Search in Google Scholar
 H. Kwon, A. Cordaro, D. Sounas, A. Polman, and A. Alù, “Dual-polarization analog 2D image processing with nonlocal metasurfaces,” ACS Photonics, vol. 7, no. 7, pp. 1799–1805, 2020, https://doi.org/10.1021/acsphotonics.0c00473. Search in Google Scholar
 A. Komar, R. A. Aoni, L. Xu, et al.., “Edge detection with mie-resonant dielectric metasurfaces,” ACS Photonics, vol. 8, no. 3, pp. 864–871, 2021, https://doi.org/10.1021/acsphotonics.0c01874. Search in Google Scholar
 C. J. Laan and H. J. Frankena, “Fast computation method for derivatives of multilayer stack reflectance,” Appl. Opt., vol. 17, no. 4, pp. 538–541, 1978, https://doi.org/10.1364/ao.17.000538. Search in Google Scholar
 K.-O. Peng and M. R. Fonteijne, “Derivatives of transmittance and reflectance for an absorbing multilayer stack,” Appl. Opt., vol. 24, no. 4, pp. 501–503, 1985, https://doi.org/10.1364/ao.24.000501. Search in Google Scholar
 A. P. Engelbrecht, Computational Intelligence: An Introduction, Hoboken, John Wiley & Sons, Ltd., 2007. Search in Google Scholar
 D. E. Goldberg, Genetic Algorithms in Search Optimization and Machine Learning, Boston, Addison-Wesley, 1988. Search in Google Scholar
 J. D. Joannopoulos, S. G. Johnson, J. N. Winn, et al., Photonic Crystals: Molding the Flow of Light, Princeton, Princeton University Press, 2008. Search in Google Scholar
 O. Reshef, M. P. DelMastro, K. K. M. Bearne, et al.., “An optic to replace space and its application towards ultra-thin imaging systems,” Nat. Commun., vol. 12, no. 1, p. 3512, 2021, https://doi.org/10.1038/s41467-021-23358-8. Search in Google Scholar
 A. Chen and F. Monticone, “Dielectric nonlocal metasurfaces for fully solid-state ultrathin optical systems,” ACS Photonics, vol. 8, no. 5, pp. 1439–1447, 2021, https://doi.org/10.1021/acsphotonics.1c00189. Search in Google Scholar
 T. Zhu, J. Huang, and Z. Ruan, “Optical phase mining by adjustable spatial differentiator,” Adv. Photonics, vol. 2, no. 1, p. 016001, 2020. https://doi.org/10.1117/1.ap.2.1.016001. Search in Google Scholar
© 2021 Chunqi Jin and Yuanmu Yang, published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.