The “one-to-many” problem is a typical challenge that faced by many machine learning aided inverse nanophotonics designs where one target optical response can be achieved by many solutions (designs). Although novel training approaches, such as tandem network, and network architecture, such as the mixture density model, have been proposed, the critical problem of solution degeneracy still exists where some possible solutions or solution spaces are discarded or unreachable during the network training process. Here, we report a solution to the “one-to-many” problem by employing a conditional generative adversarial network (cGAN) that enables generating sets of multiple solution groups to a design problem. Using the inverse design of a transmissive Fabry–Pérot-cavity-based color filter as an example, our model demonstrates the capability of generating an average number of 3.58 solution groups for each color. These multiple solutions allow the selection of the best design for each color which results in a record high accuracy with an average index color difference ΔE of 0.44. The capability of identifying multiple solution groups can benefit the design manufacturing to allow more viable designs for fabrication. The capability of our cGAN is verified experimentally by inversely designing the RGB color filters. We envisage this cGAN-based design methodology can be applied to other nanophotonic structures or physical science domains where the identification of multi-solution across a vast parameter space is required.
Retrieval of structural arrangements for a desired optical performance has been the fundamental challenge in nanophotonics inverse design . This challenging task requires searching a range of geometrical parameters in an ample design space. For the past decades, sophisticated methods have been developed to arrive at optimized solutions using heuristics-based approaches such as particle swarm [2, 3], genetic algorithms , , , and gradient-based approaches [7, 8]. Often, the intuition and experience of the designer have been critical in obtaining rational operational principles with the advantage that the performance of such structures can be understood [7, 9], , , . However, such a rational approach can be rather unsatisfactory as human intuition will not always guarantee an optimal design with the highest possible performance.
The recent remarkable development of deep learning (DL) has offered great promise in solving some of the critical challenges faced in nanophotonics [1, 13], , . DL is a subset of machine learning technology inspired by animal brains’ biological neural networks . Through a training process that involves iterative optimization of network parameters via gradient descent , the DL model, which contains a few artificial neuron layers, can learn the non-linear relations between the photonic structure and optical response. In recent years, it has found many successful applications in photonic research such as chiral metamaterials [18, 19], wavefront engineering , , , and structural color , , , , . An exhaustive overview of this field of study can be found in the review literature [14, 15].
In the inverse design of nanophotonic structures, the DL regression models have been utilized to predict the potential solution of optical response and achieved success to some extent under some specific regions and constraints [9, 25, 28], , , . One of the main difficulties for inverse design is the “one-to-many” mapping characteristic where one target optical response can be produced by multiple structures. A direct inversion of the artificial neural network (ANN) to predict the design usually fails as the training will not converge due to the existence of several possible solutions . To circumvent the challenges related to the “one-to-many” mapping, tandem networks architecture is proposed . This type of network consists of two sub-networks: an actual inverse network that takes the optical response and outputs the desired design and a pre-trained forward network that evaluates the outputs and generates optical responses. The tandem architecture allows direct comparison in the physics domain and avoids confusion from different designs that lead to a similar optical response. In previous work, we adopted such an architecture for the inverse design of a Fabry–Pérot-cavity-based structural color with high design accuracy . Despite the success in tackling the convergence problem using this tandem network, the one-to-many problem leaves us with some critical issues. A prominent limitation of the tandem approach resides in the existence of inaccessible solutions  or a dead zone , which means that some possible solutions or solution spaces are discarded or unreachable during the network training process. In consequence, only a singular solution can be identified for each target optical response. The collapse of the network onto a set of singular solutions significantly limits the solution diversity and could even prevent reaching an optimal solution as the selected solution is not necessarily the best performing one. Therefore, developing an approach that enables identifying multiple solutions in different local/global optima can be extremely beneficial in achieving higher inverse design accuracy/performance. In addition, it will also provide more freedom in the actual fabrication process to select designs that fits the manufacturing requirement or facility limitation. For instance, a solution with the smaller dimensions could be selected to realize denser device integration while a large-dimension solution could be more suitable when the available facilities have certain resolution limitations. Furthermore, the performance decay induced by the fabrication error can also be mitigated via manufacturing a device with a less parameter-sensitive solution.
Compared with other works that employed the generative models (e.g., generative adversarial networks (GAN) and variational autoencoder (VAE)) to design the 2D metasurface , , , , , we introduced the GAN model  into the parameter-based nanophotonic inverse design, which provides an approach fully addressing the one-to-many problem. GAN is one of the most popular generative models that have been widely used in computer vision and image synthesis . It consists of two networks, a generator and a discriminator, which contest with each other during the training process to create authentic images . Due to its unique ability to reveal the hidden distribution behind enormous training datasets, GAN has been used by optical scientists to inversely design metasurfaces and diffractive optical devices with a high degree of freedom [22, 43], , , . Here, we investigate the potential of applying GAN in identifying multiple solutions for one target optical response. We argue that the geometrical distribution in the training dataset is a key hidden parameter that can be utilized in GAN to avoid dead zone and identify multiple solutions.
We conduct an inverse design of a transmissive Fabry–Pérot (F–P) cavity-based structural color filter device using GAN as a demonstration. The structural color filter can display colors by selectively transmitting or reflecting the light with a specific wavelength by varying geometry. It has received enormous interest from research communities owing to its potential applications in display, full-color printing, optical encryption and solar cells , , . The reasons for using such a system are threefolds: Firstly, the inverse design of the tri-layer F–P cavity structure (Ag–SiO2–Ag in this work) is a typical one-to-many problem as the transmissive peaks would appear periodically at the same wavelength as the increase of resonant cavity length [50, 51]; Secondly, as a relatively simple structure with the well-established analytical model , the performance of the developed GAN can be effectively investigated and evaluated. Thirdly, the design quality can be precisely quantified by the color difference index ΔE 2000  (denoted as ΔE below), which is calculated from two CIELAB vectors and has a physical meaning in statistics compared to mean squared error (MSE). After training, our GAN produces an average of 3.58 solution groups for each color in the testing set, which covers 93.9% of all ground truths. These multiple solutions allow the selection of the best design for each color which results in the lowest average color difference ΔE of 0.44 that has ever been reported for the inverse design of structural color. The identification of multiple solution groups is verified experimentally by fabricating different multilayer structures to demonstrate the same color.
2 Results and discussion
2.1 Problems of tandem network
We start by elaborating on the critical dead zone problem that the tandem network faces when performing inverse design. Taking the inverse design of a transmissive F–P-cavity-based structural color device as an example, the structure (shown in Figure 1a) has two reflective metal layers (Ag), which are separated by a lossless dielectric (SiO2) and stacks on a quartz substrate. The function of color filter is achieved by multiple round-trip phase delays of electromagnetic waves in the F–P resonator. By tuning the thickness of each layer (d1, d2, and d3, denoted as a single design vector D below), the resonant peak can be controlled to produce different colors. In a typical one-to-many problem, one color, represented by the CIELAB vector Lab, could be produced by several different structures D 1, D 2, and D 3. When this color (Lab) is fed to a tandem network, only one of these solutions will be selected. The other solutions will be discarded to minimize the loss during the training, as illustrated in Figure 1b. This solution degeneracy is due to the inherent contradiction between the one-to-one mapping nature of ANNs and the one-to-many mapping problem.
One potential approach to resolve this contradiction is to introduce a latent vector z. The value of this latent vector can be sampled from a specific distribution (e.g., uniform or normal distribution) and assigned to the Lab (as shown in Figure 1c). Therefore, the one color can be split into three inputs by the variation of this latent vector and generates three one-to-one mappings that the tandem network can handle. However, these three inputs could still degenerate into very similar solutions within one solution group. The cartoon schematic of such degeneracy is illustrated in Figure 1d where the loss versus D landscape is plotted. The three sunken areas represent the three potential solution groups (outlined by blue lines) for this color. Different inputs can still be mapped into one sunken area, resulting in the degeneracy of the solution groups (depicted as the red dots). This behavior will be demonstrated and discussed later in the paper. Therefore, more constraint is required to force the network to search in the entire D space so all three solution groups can be identified (shown in Figure 1e). This additional constraint can be realized by generating a dataset distributed across the entire D space and forcing the network to learn such distribution. This means that alongside the effort to minimize the color difference, the distribution distance between the true thickness and the generator’s predicted thickness should also be minimized during the training process.
2.2 cGAN network design and training
The conditional generative adversarial networks  (cGAN) is a perfect tool for this task. cGAN usually has two parts, a discriminator and a generator. The generator takes a latent vector and the design target (condition) as inputs and outputs the generated data (also known as fake data). The discriminator evaluates the performance and distribution of the fake data and produces losses for backpropagation to shape the fake data close to the real data (training data). Figure 2 illustrates the network architectures used in this paper (details are shown in Figure S1). The generator comprises three sub-networks: Lab upraising layers, z upraising layers, and thickness regressor. The upraising layers are responsible for raising the dimensions of Lab and z from 3 and 2 to 128. The two upraised vectors are concatenated and put into the thickness regressor, which has 256 neurons per hidden layer and a sigmoid function at the end of the output layer, to generate the predicted thicknesses. The discriminator has two parallel networks: evaluator and Lab regressor. The evaluator evaluates the thickness data from the dataset (real) and generator (fake) and outputs two distribution scores. The difference between real and fake scores measures the distribution difference between ground truths and predicted data. The pre-trained Lab regressor takes thickness as input and returns the corresponding color (Lab).
As mentioned above, during the training of cGAN, a reference data (real data) distribution is required so the generator could learn the distribution from these data via the evaluator. The dataset was generated by randomly sampling 50,000 different combinations of layer thicknesses (D) using a uniform distribution. We set the sampling range of 0–50 nm for the Ag films and 0–1000 nm for the SiO2 film. As a thicker Ag layer will significantly reduce the color brightness while too thick SiO2 can cause many resonances in the visible spectrum and induce color degeneration and crosstalk, we believe that the sampling ranges and distribution are sufficient to cover the requested D space for diverse solution groups. The thickness distributions are shown in Figure 3a–c, in which the thicknesses are uniformly divided into 100 groups, and the bars indicate the probability density of different groups. It is worth mentioning that other thickness distributions may also be applied as the reference distribution, but the uniform one is more straightforward for us due to its accessibility in implementation and excellent color coverage. These structural designs were then turned into transmission spectra via the transfer matrix method and converted into HSV (hue, saturation, and value) and CIELAB color models. All generated colors are plotted in the CIE 1931-xy chromaticity diagrams in Figure S2. The discussion of relationships between geometry parameters and the color is provided in Section 2 of Supplementary Materials. The obtained dataset is divided into three groups for training (40,000), validation (5000), and testing (5000).
Before training our cGAN model, we firstly trained a Lab regressor to achieve forward modeling of the color Lab from a structure D. The detailed training process of this Lab regressor is provided in Supplementary Materials (Table S1 and Figure S4). The trained Lab regressor demonstrates an average color difference (ΔE) of 0.19 from the testing set. The significance of ΔE can be classified into five groups: (1) ΔE < 1, no discernable color difference; (2) 1 < ΔE < 2, the difference can be observed by experienced persons; (3) 2 < ΔE < 3.5, the difference can be observed by inexperienced persons; (4) 3.5 < ΔE < 5, clear difference can be noticed; (5) 5 < ΔE, two different colors are observed . In the training of cGAN, both the color and thickness distribution differences are used in the loss function to apply backpropagation. We used Wasserstein distance as the distribution loss and applied spectral normalization to both evaluator and generator to improve the training stability . The updating in the training process was alternatively performed on the generator and evaluator to minimize the hinge adversarial loss , , . Details of the training process and the definitions of loss functions are provided in the Supplementary Materials (Table S2 and Equation S1–S3).
The training loss curves of the generator and evaluator are shown in Figure 3d. The loss of generator demonstrates a clear drop to close to zero, indicating the significant improvement of its predicting capability during the training process. The loss of evaluator compares the fake and real score and remains steadily at ca. 2 during training. To better understand this behavior, we plot the value of real and fake scores as a function of the training epoch in Figure 3e. It can be observed that the real and fake scores are close, indicating a balanced performance improvement for the generator and evaluator. This property is essential in cGAN training to ensure a balanced development of the generator and evaluator, which will be discussed in detail in the following section. Figure 3f displays the evolution curve of the generator. The Lab loss (red line), which is the MSE between prediction and ground truth of Lab, dropped fast in the first ten thousand epochs and then gradually converged to 0.02, meaning that the generator’s prediction accuracy improved quickly then converged. The generator’s fake score (blue line) in Figure 3f fluctuates around zero, which projects that the generator followed the development of the evaluator and had no model collapse during the training process.
2.3 cGAN performance evaluation
After training, we used the testing set to evaluate the performance of the generator. The testing set contains 5000 data with the same uniform distribution as the training set (shown in Figure 4a–c). The trained generator takes the 5000 Lab and latent vectors z sampled from the standard normal distribution as inputs and predicts 5000 inverse designed thicknesses (denoted as D p). The distributions of the three thicknesses in D p are plotted in Figure 4d–f. To quantify the distribution distance between predicted thickness and ground truth, the Jensen–Shannon distance (JSD) is introduced. The corresponding JSD computation process and reference values are provided in Supplementary Materials Section 4 (Figure S5 and Eqs. (S4)–(S8)). The measured JSDs for our case are 0.069 for d 1, 0.067 for d 2, and 0.066 for d 3, reflecting that the predicted thickness and ground truth have great similarity in distribution. This proves that the trained generator is able to produce designs with restricted thickness distribution.
Next, we investigated the capability of the generator in achieving diverse solutions. As mentioned above, the latent vector z is introduced to generate multiple solutions for each Lab. Currently, only one z is sampled for each Lab, hence only one solution is predicted for each color. In order to obtain multiple solutions, we randomly sampled 1000 different z from the standard normal distribution for each Lab and classified the obtained 1000 solutions into different solution groups by the density-based spatial clustering of applications with noise (DBSCAN), an unsupervised machine learning clustering algorithm. Details of the classification process can be found in Section 5 of the Supplementary Materials. Figure 5a and b presents the distributions of solution group numbers when 100 and 1000 z were assigned to each Lab, respectively. The aggregation of solution group numbers moves from 1–4 to 2–5 as the z number grew from 100 to 1,000, showing a significant solution diversity improvement. An average of 3.58 solution groups can be identified for these 5000 Lab when each target color is assigned with 1000 different z, proving the capability of our network in achieving diverse solution groups. Figure 5c plots the solution group number as a function of different z numbers used for each color. It can be observed that the solution group number initially increases sharply with increasing z number and eventually saturates when the z number reaches over 1000 (highlighted by the red dot in Figure 5c). A maximum average solution number of 3.66 was obtained when 2100 z was used for each color (highlighted by the red triangle in Figure 5c). The following slight decrease is related to limitations in the clustering algorithm. As the z number increases, the different solution groups may slightly overlap with each other, which cannot be distinguished by DBSCAN.
Concurrently with the increasing number of solution groups, the color difference ΔE can also be significantly reduced. Figure 5d plots the ΔE distribution of 5000 solutions when one z is sampled for each Lab. The majority of the ΔE is located between 0 and 5, with an average ΔE value of 2.98. While 1000 z are sampled for each Lab, the lowest ΔE for each color almost falls in 0–1 (can be considered no color difference, as shown in Figure 5e), with an average value of 0.44. To the best of our knowledge, this is the lowest color difference reported for structural color inverse design. The dependence of the z number on the average lowest ΔE is presented in Figure 5f. An average lowest ΔE can reach below 1 when the z number is over 15 (highlighted by a red dot in Figure 5f inset) and below 0.5 when the z number is over 400 (highlighted by the red triangle in Figure 5f). To elucidate the high design accuracy achieved by our network, we compared the absolute d 2 differences (Δd 2) between the solutions obtained from each color (using 1000 z) and its ground truth (global optimum). We discovered that 93.9% of the Δd 2 are within 5 nm, implying the majority of the ground truths have been successfully identified. Detailed distribution information of Δd 2 is provided in Figure S6.
All of these results suggest that both the quantity and quality of solutions can be improved using our network by increasing the number of z. It is worth mentioning that although multiple z sampling is required to achieve a high solution group number and low ΔE, the time required for such prediction is straightforward and largely effortless. For example, applying inverse design and solution optimization for 5000 colors with 1000 z sampling for each color will only consume 16.18 s in our computation environment, which is equivalent to 3.24 ms per color.
In order to verify the fabrication applications of multiple solution groups, we applied the generator to inversely design two colors generated by the F–P cavity. Table 1 lists the best solution for each obtained solution group. Both target colors have achieved more than two different solution groups. The ground truths (target designs) have been successfully identified as one of the solutions in both cases. In particular, five solution groups have been identified for color #2 with the d 2 value spanning from 39 to 985 nm, proving the powerful searching ability of our network. These multiple solution groups allow the selection of suitable designs for the manufacturing requirement. Films that are too thin can be challenging to fabricate with high quality. For example, a design with a very thin Ag layer (e.g., 3 nm in design 3 for color #2) could increase the fabrication relative error and might need to be discarded despite its low ΔE. Instead, a different design (e.g. design 4) could be selected due to its relatively thicker films but still competitive ΔE. The sensitivity of the solution to fabrication errors is also an important factor to consider. We calculated the sensitivity here as the ΔE when d 2 was added or subtracted by 1 nm (denoted as δ 1 and δ −1, respectively). Higher δ ±1 means that the solution is more sensitive to fabrication error and requires a more precise fabrication process. Although the ground truth of color #1 in Table 1 has been successfully identified in design 1, the sizeable δ ±1 inflicts extreme requirements on the fabrication accuracy. On the other hand, design 3 with a slightly larger ΔE but the smallest δ ±1 would be more suitable for the actual application. The capability of our network in identifying multiple solution groups for each color is clearly advantageous for providing viable designs for fabrication.
|Target D (d 1–d 2–d 3) nm||Target color (sRGB)||Designed D (d 1–d 2–d 3) nm||Designed color (sRGB)||Comparison (target, design)||ΔE||(δ 1, δ −1)|
We have proved from the discussion above that the proposed cGAN architecture can completely solve the “one-to-many” issue for this nanophotonics question. We argue that the vital component in this architecture is the evaluator, which enables the generator to learn the distribution of the dataset. Figure S8 presents the performance of the same networks with the evaluator disabled. The distributions of the three thicknesses designed for the testing set show no uniformity (Figure S8a–c). The measured JSDs between these designed thicknesses and the testing set are 0.21 for d 1, 0.64 for d 2 and 0.14 for d 3, respectively. More importantly, with 1000 z sampled for each color, the network failed to identify multiple solution groups. Instead, the distribution of d 2 is centralized between the range of 800–1000 nm, which leads to a relatively sizeable ΔE distribution (Figure S8d) with an average ΔE of 7.88 and a limited color coverage (Figure S8e). This proves that the evaluator plays a vital role in the solution quality and diversity of the generator. Finally, it is also worth pointing out that the balance between the generator and evaluator is also essential for the network performance (Figures S9–S11). A stronger discriminator will lead to a divergent generator, while a weaker discriminator will slow the training. Details are discussed in Section 4 of Supplementary Materials.
2.4 Experimental validation of designed sRGB color filters
To further evaluate the network generalization and search performance of our network, we performed the inverse design of three color filters with sRGB values of red (0.5, 0, 0), green (0, 0.5, 0), and blue (0, 0, 0.5) using our trained generator with 1000 z sampling for each color. Note that these three colors are not presented in our dataset and hence have no ground truth. These examples could test the capability of our cCAN in searching multiply solutions for colors outside the dataset. To better visualize the searchability, we fixed the thicknesses of the top and bottom Ag layers to be 30 nm (this Ag restriction is removed in the cGAN inverse design) while sweeping the thickness of the SiO2 layer (d 2) from 0 to 1000 nm to generate the loss (MSE) curves for each color (shown in Figure 6a–c). The loss between the target and swept colors varies as d 2 changes. A small loss represents a better solution, and a dip towards 0 in the loss curve indicates one locally optimal solution group for the target. Therefore, the solution group number of each target color can be estimated from the number of curve dips towards 0. In this case, we could identify two solution groups for both the red and green colors, and three solution groups for the blue color. In addition, all the solution groups correspond to the different resonant orders for each color. The two solution groups at ca. 16 and 350 nm correspond to the 1st and 2nd order resonances for red (Figure 6a); the ca. 130 and 320 nm designs are the 1st and 2nd order resonances for green (Figure 6b); and the ca. 100, 250, and 410 nm designs are the first three resonant orders for blue (Figure 6c). The inverse design results of these three colors are demonstrated in Figure 6d–f. Here, we plot the distributions of the 1000 predicted d 2 values for the three colors, respectively, in which d 2 is uniformly divided into 100 groups in the range of 0–1000 nm and a bar indicates the solution number of each thickness group. Our network has successfully produced multiple solution groups for each color, matching the number and value of the solution groups identified from the loss curves. This suggests that our network can identify all possible solutions for different target colors. The relationships between the value of z and solution groups are discussed in Supplementary Materials (Figure S7).
We will now experimentally validate our multiple solution groups by fabricating the red, green, and blue color filters previously designed by our cGAN. The parameters of the best solution in each solution group for each color are listed in Table 2. All the designed color filters were fabricated on quartz substrates. Details of the fabrication process are available in the method section. Figure 7a–c displays the cross-sectional SEM of all fabricated samples. A three-layered structure containing Ag (blue shading) top and bottom layers and SiO2 dielectric is clearly visible for all samples. It is also evident that the dimensions (especially the thickness of the SiO2 layer) are significantly different for different designs of the same target color. The measured transmission spectra for all samples are compared with their theoretical (calculated from designed parameters) counterparts and shown in Figure 7d–f. All measured spectra match well with the theoretical ones in terms of both transmission peak wavelength and intensity. The discrepancies in peak intensity at shorter wavelengths may be attributed to the surface oxidization of the Ag films [60, 61]. Figure 7g–i displays the resultant colors for each design. For each target color (TGT.), we compare the theoretical colors from different designs (DSG.) with the colors converted from corresponding experimental spectra (EXPT. SPT.) and the photographs of the samples taken by the camera under a sunlight-like source (EXPT. PHT.). The discrepancy in the R2 photograph may result from the light source intensity or the camera parameters (e.g., white balance, contrast, exposure time, etc.).
|Sample name||Target color (sRGB)||Designed D (d 1–d 2–d 3) nm||Designed lab||Designed sRGB||ΔE||(δ 1, δ −1)|
In conclusion, a variation of cGAN has been successfully implemented to resolve the typical one-to-many problem in nanophotonics using the inverse design of the F–P-cavity-based color filter as a demonstration. We suggest that the key to avoiding solution group degeneracy is generating solutions with the same parameter distribution of a dataset covering most solution spaces. Equipped with an evaluator that enables learning the dataset distribution, the cGAN developed in this work is able to identify an average of 3.58 solution groups for each color. Simultaneous with the improved solution diversity, the quality of the solutions is also significantly improved. An average ΔE of 0.44 has been achieved when sampling 1000 z for each color. The high design accuracy could be attributed to identifying 93.9% ground truths of the testing set. Finding multiple solution groups can also benefit design manufacturing to allow more viable designs for fabrication. The capability of our cGAN has also been verified experimentally by inverse designing the sRGB color filters. The measured spectra and photographs of fabricated two red, two green, and three blue color filters present excellent agreement with the designs. It should be acknowledged that the presented work is limited by its dataset range as well as its simple three-layered structure. However, it is worth highlighting that the three-layer F–P cavity structure only serves as an example to demonstrate the capability of cGAN and the novelties of this work remain valid. We envisage this cGAN-based design methodology to be applied to other nanophotonic structures or physical science domains where the identification of multi-solution across a vast parameter space is required.
Before fabricating the device, a series of experiments were conducted to identify the deposition rate for both Ag and SiO2 using both ellipsometry (Woollam M-2000DI) and atomic force microscopy (Park System X7) for calibration. The same recipes were then used to fabricate our designed structural colors to ensure the best experimental reproduction of the designed structures. During the fabrication of structural colors, the bottom Ag thin film was firstly deposited by the electron-beam evaporation system (Leybold LAB700) with a growth rate of 0.5 Å/s. Without breaking the vacuum, a 1 nm thin Ti film was formed by electron-beam evaporation with a growth rate of 0.1 Å/s on top of the Ag film to improve the adhesion between Ag and the following SiO2. The SiO2 layer was subsequently formed on the Ti layer in a reactive sputtering system (Leybold Helios Pro XL) using Si target and O2 plasma. The growth rate of SiO2 is 0.52 nm/s. After completing the SiO2 deposition, the top Ag film was formed with the same process condition as the bottom Ag film and then a 1 nm Ti film on top.
The refractive index of SiO2 used in the calculation is measured by an ellipsometer (Woollam M-2000DI). The transmission spectra under normal incidence were measured by a spectrometer (Bentham PVE300) with Air as reference. The photographs were taken using a digital camera (Canon EOS 500D) with a shutter speed of 1/20 s, an aperture of f/5, and the default white balance. In order to produce the source with the correct color temperature, an Apple iPad Pro (10.5 inch, 2017 year version) was used as a backlight source, with both true tone and night shift switched off and brightness set as maximum. The original photograph is shown in Figure S13. The cross-sectional morphologies of fabricated multiple thin-film structures were characterized using a field-emission scanning electron microscope (JEOL 7500 FEGSEM) at an acceleration voltage of 2 kV and a working distance of 4 mm.
All the calculations and neural network training were performed on a Windows PC with the hardware: CPU: Intel Core i9-9900K; RAM: 48 GB; GPU: NVIDIA RTX 2070. The involved software packages include Python 3.8, PyTorch 1.9.0, NumPy, Scikit-Learn and Colour-Science. The refractive index of Ag used in the calculation is Ag (Palik 0–2 μm) .
Funding source: Royal Society (UK)
Award Identifier / Grant number: IEC/NSFC/170193
Funding source: National Natural Science Foundation of China
Award Identifier / Grant number: IEC/NSFC/170193
Funding source: Engineering and Physical Sciences Research Council
Award Identifier / Grant number: EP/N035437/1
Author contribution: P.D. and R.H. conceived the project and constructed the research frame. P.D. developed the computer code to run the machine learning algorithms. K.S. and X.Y. performed the experimental fabrications and characterizations. O.L.M. and C.H.D.G helped to interpret the results and improve the research frame. Y.H., H.G. and X.Z. provided constructive suggestions to improve the significance of this project. R.H. supervised and directed the project. P.D. and R.H. wrote the manuscript with inputs from all the authors. All authors attended the result discussion and gave comments and advice on the manuscripts.
Research funding: The authors acknowledge the support of the International Exchange Scheme (IEC/NSFC/170193) between Royal Society (UK) and the National Natural Science Foundation of China (China). This work is a part of the ADEPT project funded by a program grant from the EPSRC (EP/N035437/1). All data supporting this study are openly available from the University of Southampton repository at https://doi.org/10.5258/SOTON/D2182.
Conflict of interest statement: The authors declare no conflicts of interest.
 W. Ma, Z. Liu, Z. A. Kudyshev, A. Boltasseva, W. Cai, and Y. Liu, “Deep learning for the design of photonic structures,” Nat. Photonics, vol. 15, p. 77, 2021, https://doi.org/10.1038/s41566-020-0685-y.Search in Google Scholar
 R. Yan, T. Wang, X. Jiang, et al.., “Design of high-performance plasmonic nanosensors by particle swarm optimization algorithm combined with machine learning,” Nanotechnology, vol. 31, p. 375202, 2020, https://doi.org/10.1088/1361-6528/ab95b8.Search in Google Scholar PubMed
 J. C. C. Mak, C. Sideris, J. Jeong, A. Hajimiri, and J. K. S. Poon, “Binary particle swarm optimized 2 × 2 power splitters in a standard foundry silicon photonic platform,” Opt. Lett., vol. 41, p. 3868, 2016, https://doi.org/10.1364/ol.41.003868.Search in Google Scholar PubMed
 B. J. Offrein, G.-L. Bona, R. Germann, I. Massarek, and D. Erni, “A very short planar silica spot-size converter using a nonperiodic segmented waveguide,” J. Lightwave Technol., vol. 16, p. 1680, 1998.10.1109/50.712252Search in Google Scholar
 J. A. Bossard, L. Lin, S. Yun, L. Liu, D. H. Werner, and T. S. Mayer, “Near-ideal optical metamaterial absorbers with super-octave bandwidth,” ACS Nano, vol. 8, p. 1517, 2014, https://doi.org/10.1021/nn4057148.Search in Google Scholar PubMed
 S. So, Y. Yang, S. Son, et al., “Highly suppressed solar absorption in a daytime radiative cooler designed by genetic algorithm,” Nanophotonics, vol. 11, no. 9, pp. 2107–2115, 2021. https://doi.org/10.1515/nanoph-2021-0436.Search in Google Scholar
 A. Michaels, M. C. Wu, and E. Yablonovitch, “Hierarchical design and optimization of silicon photonics,” IEEE J. Sel. Top. Quant. Electron., vol. 26, p. 1, 2020, https://doi.org/10.1109/jstqe.2019.2935299.Search in Google Scholar
 D. Liu, Y. Tan, E. Khoram, and Z. Yu, “Training deep neural networks for the inverse design of nanophotonic structures,” ACS Photonics, vol. 5, p. 1365, 2018, https://doi.org/10.1021/acsphotonics.7b01377.Search in Google Scholar
 M. M. Elsawy, S. Lanteri, R. Duvigneau, J. A. Fan, and P. Genevet, “Numerical optimization methods for metasurfaces,” Laser Photon. Rev., vol. 14, p. 1900445, 2020, https://doi.org/10.1002/lpor.201900445.Search in Google Scholar
 S. Molesky, Z. Lin, A. Y. Piggott, W. Jin, J. Vucković, and A. W. Rodriguez, “Inverse design in nanophotonics,” Nat. Photonics, vol. 12, p. 659, 2018, https://doi.org/10.1038/s41566-018-0246-9.Search in Google Scholar
 S. D. Campbell, D. Sell, R. P. Jenkins, E. B. Whiting, J. A. Fan, and D. H. Werner, “Review of numerical optimization techniques for meta-device design [invited],” Opt. Mater. Express, vol. 9, p. 1842, 2019, https://doi.org/10.1364/ome.9.001842.Search in Google Scholar
 S. So, T. Badloe, J. Noh, J. Bravo-Abad, and J. Rho, “Deep learning enabled inverse design in nanophotonics,” Nanophotonics, vol. 9, p. 1041, 2020, https://doi.org/10.1515/nanoph-2019-0474.Search in Google Scholar
 J. Jiang, M. Chen, and J. A. Fan, “Deep neural networks for the evaluation and design of photonic devices,” Nat. Rev. Mater., vol. 6, pp. 679–700, 2020.10.1038/s41578-020-00260-1Search in Google Scholar
 P. R. Wiecha, A. Arbouet, C. Girard, and O. L. Muskens, “Deep learning in nano-photonics: inverse design and beyond,” Photon. Res., vol. 9, p. B182, 2021, https://doi.org/10.1364/prj.415960.Search in Google Scholar
 I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, Cambridge, Massachusetts, MIT Press, 2016.Search in Google Scholar
 W. Ma, F. Cheng, and Y. Liu, “Deep-learning-enabled on-demand design of chiral metamaterials,” ACS Nano, vol. 12, p. 6326, 2018, https://doi.org/10.1021/acsnano.8b03569.Search in Google Scholar PubMed
 Y. Li, Y. Xu, M. Jiang, et al.., “A multi-frequency piezoelectric vibration energy harvester with liquid filled container as the proof mass,” Phys. Rev. Lett., vol. 123, p. 213902, 2019, https://doi.org/10.1103/physrevlett.123.213902.Search in Google Scholar PubMed
 M. Chen, J. Jiang, and J. A. Fan, “Design space reparameterization enforces hard geometric constraints in inverse-designed nanophotonic devices,” ACS Photonics, vol. 7, p. 3141, 2020, https://doi.org/10.1021/acsphotonics.0c01202.Search in Google Scholar
 J. Jiang and J. A. Fan, “Simulator-based training of generative neural networks for the inverse design of metasurfaces,” Nanophotonics, vol. 9, p. 1059, 2020.10.1515/nanoph-2019-0330Search in Google Scholar
 J. Jiang, D. Sell, S. Hoyer, J. Hickey, J. Yang, and J. A. Fan, “Free-form diffractive metagrating design based on generative adversarial networks,” ACS Nano, vol. 13, p. 8872, 2019, https://doi.org/10.1021/acsnano.9b02371.Search in Google Scholar PubMed
 Z. Huang, X. Liu, and J. Zang, “The inverse design of structural color using machine learning,” Nanoscale, vol. 11, p. 21748, 2019, https://doi.org/10.1039/c9nr06127d.Search in Google Scholar PubMed
 O. Hemmatyar, S. Abdollahramezani, Y. Kiarashinejad, M. Zandehshahvar, and A. Adibi, “Full color generation with Fano-type resonant HfO2 nanopillars designed by a deep-learning approach,” Nanoscale, vol. 11, p. 21266, 2019, https://doi.org/10.1039/c9nr07408b.Search in Google Scholar PubMed
 L. Gao, X. Li, D. Liu, L. Wang, and Z. Yu, “A bidirectional deep neural network for accurate silicon color design,” Adv. Mater., vol. 31, p. 1905467, 2019, https://doi.org/10.1002/adma.201905467.Search in Google Scholar PubMed
 J. Baxter, A. C. Lesina, J.-M. Guay, A. Weck, P. Berini, and L. Ramunno, “Plasmonic colours predicted by deep learning,” Sci. Rep., vol. 9, p. 1, 2019. https://doi.org/10.1038/s41598-019-44522-7.Search in Google Scholar PubMed PubMed Central
 N. B. Roberts and M. Keshavarz Hedayati, “A deep learning approach to the forward prediction and inverse design of plasmonic metasurface structural color,” Appl. Phys. Lett., vol. 119, p. 061101, 2021, https://doi.org/10.1063/5.0055733.Search in Google Scholar
 X. Xu, C. Sun, Y. Li, J. Zhao, J. Han, and W. Huang, “An improved tandem neural network for the inverse design of nanophotonics devices,” Opt Commun., vol. 481, p. 126513, 2021, https://doi.org/10.1016/j.optcom.2020.126513.Search in Google Scholar
 R. Unni, K. Yao, X. Han, M. Zhou, and Y. Zheng, “A mixture-density-based tandem optimization network for on-demand inverse design of thin-film high reflectors,” Nanophotonics, vol. 10, no. 16, pp. 4057–4065, 2021.10.1515/nanoph-2021-0392Search in Google Scholar
 Z. Zhen, C. Qian, Y. Jia, et al.., “Realizing transmitted metasurface cloak by a tandem neural network,” Photon. Res., vol. 9, p. B229, 2021, https://doi.org/10.1364/prj.418445.Search in Google Scholar
 Z. Hou, P. Zhang, M. Ge, et al.., “Metamaterial reverse multiple prediction method based on deep learning,” Nanomaterials, vol. 11, p. 2672, 2021, https://doi.org/10.3390/nano11102672.Search in Google Scholar PubMed PubMed Central
 H. Kabir, Y. Wang, M. Yu, and Q.-J. Zhang, “Neural network inverse modeling and applications to microwave filter design,” IEEE Trans. Microw. Theor. Tech., vol. 56, p. 867, 2008, https://doi.org/10.1109/tmtt.2008.919078.Search in Google Scholar
 P. Dai, Y. Wang, Y. Hu, et al.., “Accurate inverse design of Fabry-Perot-cavity-based color filters far beyond sRGB via a bidirectional artificial neural network,” Photon. Res., vol. 9, p. B236, 2021, https://doi.org/10.1364/prj.415141.Search in Google Scholar
 W. Ma, Y. Xu, B. Xiong, et al., “Pushing the limits of functionality-multiplexing capability in metasurface design based on statistical machine learning,” Adv. Mater., vol. 34, no. 16, p. 2110022, 2022.10.1002/adma.202110022Search in Google Scholar PubMed
 D. Zhu, Z. Liu, L. Raju, A. S. Kim, and W. Cai, “Building multifunctional metasystems via algorithmic construction,” ACS Nano, vol. 15, p. 2318, 2021, https://doi.org/10.1021/acsnano.0c09424.Search in Google Scholar PubMed
 Z. A. Kudyshev, A. V. Kildishev, V. M. Shalaev, and A. Boltasseva, “Machine learning–assisted global optimization of photonic devices,” Nanophotonics, vol. 10, p. 371, 2021.10.1515/9783110710687-028Search in Google Scholar
 W. Ma and Y. Liu, “A data-efficient self-supervised deep learning model for design and characterization of nanophotonic structures,” Sci. China Phys. Mech. Astron., vol. 63, p. 284212, 2020, https://doi.org/10.1007/s11433-020-1575-2.Search in Google Scholar
 W. Ma, F. Cheng, Y. Xu, Q. Wen, and Y. Liu, “Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi-supervised learning strategy,” Adv. Mater., vol. 31, p. 1901111, 2019, https://doi.org/10.1002/adma.201901111.Search in Google Scholar PubMed
 L. Gonog and Y. Zhou, in 2019 14th IEEE Conf. Ind. Electron. Appl. ICIEA, 2019, pp. 505–510.Search in Google Scholar
 J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye, ArXiv Prepr. ArXiv200106937, 2020.Search in Google Scholar
 Z. Liu, D. Zhu, S. P. Rodrigues, K.-T. Lee, and W. Cai, “Generative model for the inverse design of metasurfaces,” Nano Lett., vol. 18, p. 6570, 2018, https://doi.org/10.1021/acs.nanolett.8b03171.Search in Google Scholar PubMed
 S. An, B. Zheng, H. Tang, et al.., “Multifunctional metasurface design with a generative adversarial network,” Adv. Opt. Mater., vol. 9, p. 2001433, 2021, https://doi.org/10.1002/adom.202001433.Search in Google Scholar
 S. So and J. Rho, “Designing nanophotonic structures using conditional deep convolutional generative adversarial networks,” Nanophotonics, vol. 8, p. 1255, 2019.10.1515/nanoph-2019-0117Search in Google Scholar
 F. Wen, J. Jiang, and J. A. Fan, “Robust freeform metasurface design based on progressively growing generative networks,” ACS Photonics, vol. 7, p. 2098, 2020, https://doi.org/10.1021/acsphotonics.0c00539.Search in Google Scholar
 Z. Xuan, J. Li, Q. Liu, F. Yi, S. Wang, and W. Lu, “Artificial structural colors and applications,” Innovation, vol. 2, p. 100081, 2021, https://doi.org/10.1016/j.xinn.2021.100081.Search in Google Scholar PubMed PubMed Central
 Y. Zhao, Y. Zhao, S. Hu, et al., “Artificial structural color pixels: a review,” Materials, vol. 10, no. 8, p. 944, 2017. https://doi.org/10.3390/ma10080944.Search in Google Scholar PubMed PubMed Central
 F. Chen, S.-W. Wang, X. Liu, et al.., “Colorful solar selective absorber integrated with different colored units,” Opt Express, vol. 24, p. A92, 2016, https://doi.org/10.1364/oe.24.000a92.Search in Google Scholar
 Z. Yang, Y. Zhou, Y. Chen, et al., “Reflective color filters and monolithic color printing based on asymmetric Fabry–Perot cavities using nickel as a broadband absorber,” Adv. Opt. Mater., vol. 4, p. 1196, 2016. https://doi.org/10.1002/adom.201600110.Search in Google Scholar
 Y. Wang, M. Zheng, Q. Ruan, et al., “Stepwise-nanocavity-assisted transmissive color filter array microprints,” Research, vol. 2018, Art. No.: 8109054, 2018. https://doi.org/10.1155/2018/8109054.Search in Google Scholar PubMed PubMed Central
 C. C. Katsidis and D. I. Siapkas, “General transfer-matrix method for optical multilayer systems with coherent, partially coherent, and incoherent interference,” Appl. Opt., vol. 41, p. 3978, 2002, https://doi.org/10.1364/ao.41.003978.Search in Google Scholar PubMed
 G. Sharma, W. Wu, and E. N. Dalal, “The CIEDE2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations,” Color Res. Appl., vol. 30, p. 21, 2005, https://doi.org/10.1002/col.20070.Search in Google Scholar
 M. Mirza, S. Osindero, ArXiv Prepr. ArXiv14111784, 2014.Search in Google Scholar
 W. Mokrzycki and M. Tatol, “Colour difference ∆E - a survey,” Mach. Graph. Vis., vol. 20, p. 383, 2011.Search in Google Scholar
 H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena, in Proc. 36th Int. Conf. Mach. Learn, PMLR, K. Chaudhuri and R. Salakhutdinov, Eds., 2019, pp. 7354–7363.Search in Google Scholar
 J. H. Lim and J. C. Ye, ArXiv Prepr. ArXiv170502894, 2017.Search in Google Scholar
 D. Tran, R. Ranganath, and D. M. Blei, ArXiv Prepr. ArXiv170208896, 2017.Search in Google Scholar
 T. Miyato and M. Koyama, ArXiv Prepr. ArXiv180205637, 2018.Search in Google Scholar
 J.-H. Qiu, P. Zhou, X.-Y. Gao, et al.., “Ellipsometric study of the optical properties of silver oxide prepared by reactive magnetron sputtering,” J. Kor. Phys. Soc., vol. 46, p. 269, 2005.Search in Google Scholar
 A. C. Nwanya, P. E. Ugwuoke, B. A. Ezekoye, R. U. Osuji, and F. I. Ezema, “Structural and optical properties of chemical bath deposited silver oxide thin films: role of deposition time,” Adv. Mater. Sci. Eng., vol. 2013, p. 450820, 2013, https://doi.org/10.1155/2013/450820.Search in Google Scholar
 E. D. Palik, Handbook of Optical Constants of Solids, San Diego, Academic Press, 1998.Search in Google Scholar
The online version of this article offers supplementary material (https://doi.org/10.1515/nanoph-2022-0095).
© 2022 Peng Dai et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.