Figure 2 shows the basic structure of the presently considered L3 nanocavity, where the lattice constant is a, the radius of each air hole is 0.25 a, the thickness of the slab is 0.5366 a, and the refractive index of the slab material (Si) is n=3.46. These values were chosen by considering the standard dimensions of fabricated nanocavities (a=410 nm, t=220 nm) operating at optical communication wavelengths [9], [27] and the refractive index of Si at these wavelengths. The radii of the air holes are the same as those used in [16], and the slab thickness is similar to that in [16] (0.55 a). The color plot in Figure 1 shows the electric field distribution of the fundamental mode in the y direction (E_y). The distribution was calculated for the base cavity structure by a first principle method [three-dimensional finite-difference time-domain (3D-FDTD) method], and the resulting Q factor of the base structure is 7160. The modal volume V_cav of the mode is 0.61 cubic wavelengths in the material (λ/n)³. Further details of the calculation conditions are provided in [19].

Figure 2:

A three-missing-air-holes (L3) cavity is used as the base structure for structural optimization.

The lattice constant a is 410 nm. The circles indicate the air holes (hole radius: 102.5 nm=0.25 a) formed in Si slab with a refractive index n=3.46 and a thickness of 220 nm (0.5366 a). The distribution of the y component of the electric field (E_y) of the fundamental resonant mode is plotted in color. The theoretical Q factor and modal volume V_cav of the base structure determined by FDTD are 7160 and 0.61 (λ/η)³, respectively. The displacements of the 50 air holes inside the red square are the structural parameters that are used to optimize the cavity design with respect to Q.

[Step (I)]: The positions of the 50 air holes within the area of 11 (a)×5 (rows) (indicated by the red square in Figure 2) are the structural parameters that are used to optimize the cavity design with respect to the Q factor, because most of the electric field intensity of the mode concentrates in this area [19]. We used the air hole positions as the variable parameters in the optimization process, because a specified but small offset in the air hole radius or shape that is different for each hole is more difficult to control in the etching step during fabrication. In contrast, the air hole positions, which are precisely defined in the electron beam writing process, can be reproduced with higher accuracy during etching. Each sample cavity structure (labelled by index i) is defined by the base structure and a set of 2D displacement vectors $\hspace{0.17em}{\{{\overrightarrow{d}}_1,{\overrightarrow{d}}_2,\dots \}}_i,$ where ${\overrightarrow{d}}_h=\mathrm{(}{d}_{hx},d_{hy}\mathrm{)}$ defines the displacement of the hth air hole in the x-y plane and h enumerates all air holes that are selected for structural optimization (from 1 to 50 in the present case). The parameter space vector of structure i, ${\overrightarrow{x}}_i$ as defined in Figure 1, is a single column vector with the structure (d_1x, d_1y, d_2x, …, d_50y)^T and contains displacements corresponding to the single set ${\{{\overrightarrow{d}}_1,{\overrightarrow{d}}_2,\dots ,{\overrightarrow{d}}_{50}\}}_i.$ Although we have 100 degrees of freedom in the 2D displacements of 50 air holes, the actual degrees of freedom in the present analysis are 25 because we have to impose mirror symmetries with respect to the central x and y axes to obtain high Q factors [12].

[Step (II)]: Random displacements are applied to all air holes in the x and y directions in such a way that the mirror symmetries of the structure are maintained and that a uniform distribution between −0.1a and 0.1a is obtained. The appropriate magnitude of the fluctuation has been determined in previous manual optimizations of L3 cavities [2], [17]. In this demonstration, we initially prepare n=1000 random nanocavity structures (the whole set is denoted by ${\overrightarrow{x}}_{1:n}$) using the above outlined displacement restrictions and calculate their Q factors using the 3D-FDTD method. The FDTD cell dimensions used in this work are about a/10 in x and y directions. For the discretization of the distribution of the dielectric constant in the FDTD calculation, we employed a sub-cell size of about a/4000, and the dielectric constants of each cell was determined by averaging over its sub-cells. Therefore, a change on the order of a/4000 in the dielectric constant distribution (which reflects the air hole displacements) can be resolved in the FDTD calculation of the Q factor. The obtained set of Q values, $Q\mathrm{(}{\overrightarrow{x}}_{1:n}\mathrm{)},$ exhibits a distribution between 10³ and 10⁵, and the average is 6700 (see Figure 7A). Because the first principles Q values of the initial set are spread over two orders of magnitudes, and this difference should increase in the subsequent optimization cycles, we employ ${\text{log}}_{10}\mathrm{(}Q\mathrm{(}{\overrightarrow{x}}_i\mathrm{)}\mathrm{)}$ as the target of machine learning. As a result, the initial training dataset consists of the structural parameters ${\overrightarrow{x}}_i$ and ${\text{log}}_{10}Q\mathrm{(}{\overrightarrow{x}}_i\mathrm{)}$ of the 1000 structures, i.e. $\{{\overrightarrow{x}}_{1:n},{\text{\hspace{0.17em}log}}_{10}Q\mathrm{(}{\overrightarrow{x}}_{1:n}\mathrm{)}\}$ instead of $\{{\overrightarrow{x}}_{1:n},Q\mathrm{(}{\overrightarrow{x}}_{1:n}\mathrm{)}\}.$

[Step (III)]: Ten four-layer-NNs with the same configuration as in [19] are prepared (Figure 3). The input nodes are two-channel 2D tables, where each channel corresponds to the x and y components of $\{{\overrightarrow{d}}_1,{\overrightarrow{d}}_2,\dots ,{\overrightarrow{d}}_{50}\}.$ The first layer is a convolutional layer [28] with 50 filters with a size of 3 (holes)×5 (rows)×2 (channels) that is connected to the second layer with 450 units. The last part of each NN comprises the third layer (200 units), the fourth layer (50 units), and the output layer (one unit). These layers are fully connected through rectified linear units (ReLU [29] and affine transformations. Stochastic information selection units (DROPOUT [30]) are additionally inserted between the third and fourth layers. The single output unit is intended to predict ${\text{log}}_{10}\mathrm{(}Q\mathrm{(}\overrightarrow{x}\mathrm{)}\mathrm{)}.$

Figure 3:

Configuration of the neural network prepared to learn the relationship between displacements of air holes and Q factors (ReLU: rectified linear unit. Affine: affine transformation. Dropout: random dropping of units including connections.)

[Step (IV)]: For this phase, we employ a conventional loss function L consisting of two terms: the squared difference between the output of the NN and the teacher data (i.e. ${\text{log}}_{10}\mathrm{(}Q\mathrm{(}{\overrightarrow{x}}_i\mathrm{)}\mathrm{)}$), and the summation of the squared connection weights w_m in the network (weight decay method [31]), where the latter is used to avoid the overfitting,

$$L={\left|\text{Output(}i\text{)}-{\log }_{10}Q\mathrm{(}{\overrightarrow{x}}_i\mathrm{)}\right|}^2+\frac{1}{2}\lambda {\displaystyle \sum }_m{w}_m^2\text{.}$$(1)

For the hyperparameter λ we use 0.00333 determined from the (10-fold) cross-validation method. In the training process, we randomly select one set $\{{\overrightarrow{x}}_i,{\text{\hspace{0.17em}log}}_{10}Q\mathrm{(}{\overrightarrow{x}}_i\mathrm{)}\}$ from the training dataset $\{{\overrightarrow{x}}_{1:j},{\text{\hspace{0.17em}log}}_{10}Q\mathrm{(}{\overrightarrow{x}}_{1:j}\mathrm{)}\},$ where j is the number of samples in the present dataset as defined in Figure 1, and change the internal parameters of the NN to reduce L using the back-propagation method [32]. Here, the actual output of the NN is referred to as log₁₀Q_NN, where Q_NN is an approximation of the Q factor. We apply the momentum optimization method to speed up convergence [33], where the learning rate and the momentum decay rate are set to 1.0×10⁻⁴ and 0.9, respectively. The random selection of one structure and following reduction of L by using the back-propagation method is repeated 5×10⁴ times. Ten separate NNs are trained by the same method, but with different orders of data feeding. Therefore, after the training, each NN has acquired different internal parameters, which widens the divergence of the candidate structures that are generated in the following step (V).

[Step (V)]: Several candidate structures (here, we use m=70) with potentially higher Q factors are generated using the gradient method. For this we define the loss function L′,

$$L^{\prime }=|{\log }_{10}{Q}_{\text{target}}-{\log }_{10}{Q}_{\text{NN}}{|}^2+\mathrm{(}\text{Artifitial\hspace{0.17em}loss}\mathrm{)},$$(2)

and calculate the gradient of L′ with respect to $\overrightarrow{x}$ (i.e. ${\nabla }_{\overrightarrow{x}}{L}^{\prime }$) using the back-propagation method [32], where Q_target is set to a very high value (here, we use 1.0×10⁸). Starting from a randomly generated initial structure defined in the parameter space by ${\overrightarrow{x}}_k^{\text{ini}}$ (k>j), we incrementally change the structure to reduce the loss L′ (i.e. ${\overrightarrow{x}}_k\leftarrow {\overrightarrow{x}}_k+\Delta \overrightarrow{x},$ where $\Delta \overrightarrow{x}$ is a set of incremental hole displacements calculated from ${\nabla }_{\overrightarrow{x}}{L}^{\prime }{|}_{{\overrightarrow{x}}_k}$ based on the momentum method [33]), which is repeated 2×10⁴ times. The artificial loss or regularization term in Eq. (2) is used to constrain the structural parameter space that is explored during the optimization, and different conditions are used to obtain different candidate structures. We designed the following three types of artificial losses, where λ′ is a control parameter.

(A) Squared distance from the base structure or the best structure in the previous round:

$$\frac{1}{2}{\lambda }^{\prime }|\overrightarrow{x}-{\overrightarrow{x}}_b{|}^2,$$(3)

where ${\overrightarrow{x}}_b$ refers to the sample structure with the highest Q in the previous rounds (i.e. the highest Q among $Q\mathrm{(}{\overrightarrow{x}}_{1:j}\mathrm{)}$). (In the case of the first round, ${\overrightarrow{x}}^b$ is set to 0, because the base structure has no displacements). This artificial loss is designed to explore the parameter space in the vicinity of the best structure in the previous rounds.

(B) Squared distance from a randomly generated initial structure ${\overrightarrow{x}}_k^{\text{ini}}:$

$$\frac{1}{2}{\lambda }^{\prime }|\overrightarrow{x}-{\overrightarrow{x}}_k^{\text{ini}}{|}^2.$$(4)

This artificial loss forces exploration of unknown parameter space stochastically. It is expected that a structure with a higher Q that is not predictable from the training dataset can be accidentally found by using this artificial loss.

(C) Sum of the inverse of the distances from all the structures in the training data set:

$${\lambda }^{\prime }{\displaystyle \sum }_{i\le j}|\overrightarrow{x}-{\overrightarrow{x}}_i{|}^{-1}.$$(5)

This artificial loss increases as the parameter space vector of the structure that is being optimized approaches the locations of the known structures, ${\overrightarrow{x}}_i$ with i≤j. This restriction forces exploration of unknown parameter space more strictly than (B).

For the present demonstration, we designed and investigated the following three strategies of candidate generation:

• Strategy (A): Each NN generates seven different candidates using the artificial loss (A) with seven different λ′ (3.0, 1.0, 0.1, 0.01, 0.001, 0.0001, 0.00001).

• Strategy (A+B): Each NN generates three candidates using the artificial loss (A) with three different λ′ (1.0, 0.1, 0.01), one candidate without using artificial losses (λ′=0), and three different candidates using the artificial loss (B) with three different λ′ (1.0, 0.1, 0.01).

• Strategy (A+C): Each NN generates three candidates using the artificial loss (A) with three different λ′ (1.0, 0.1, 0.01), one candidate without using artificial losses (λ′=0), and three different candidates using the artificial loss (C) with three different λ′ (1.0, 0.1, 0.01).

[Step (VI)]: The Q factors of the 70 candidate structures obtained in step (V) for each strategy are determined by 3D-FDTD calculations. The calculation conditions are the same as in [19].

[Step (VII)]: The new data consisting of 70 candidate structures (defined by ${\overrightarrow{x}}_{j+1:j+70}$) and their Q factors $\mathrm{(}Q\mathrm{(}{\overrightarrow{x}}_{j+1:j+70}\mathrm{)}\mathrm{)}$ calculated in (VI) for each strategy are added to each strategy’s training dataset.

[Step (VIII)]: Steps (IV)–(VII) are repeated 101 times. During this iterative optimization of the regression function $Q_{\text{NN}}\mathrm{(}\overrightarrow{x}\mathrm{)}$ and, consequently, also that of the cavity design, different series of training datasets are accumulated for each strategy, and 8070 sample cavities are accumulated in each dataset after 101 rounds of optimization.

Figure 4 shows the highest Q factors of the additional 70 sample structures generated in each iteration step cycle as a function of the size of the dataset (training set+70 structures generated in that cycle). The corresponding Q_NN are not discussed in the following, because the regression function is only employed to identify structures with potentially higher Q factors (via the gradient method). The results for the different strategies (A), (A+B), and (A+C) are shown with the blue, orange, and green curves, respectively. We find that the highest Q achieved in each round overall increases with further iteration, although some fluctuations exist. The highest Q factors of the structures that have been detected by 101 iterations of cavity design optimization are 5.75×10⁶, 9.12×10⁶, and 1.10×10⁷ for strategies (A), (A+B), (A+C), respectively. These values are larger than the Q factor of the original structure (Figure 2) by factors of about 800–1500. Figure 5 plots the inter-structure distances between the best structure in the present round and the best structure in the previous rounds in terms of the parameter space vector $\overrightarrow{x},$ indicating how large the modifications in each round of optimization are. It can be confirmed that the inter-structure distances tend to decrease as the optimization proceeds. The inter-structure distances for strategy (A+C) is basically always larger than those for the other structures, and that for strategy (A+B) is larger than that for (A) only at early stages (<4000 samples). The air hole displacements of the structures with the highest Q factors found during 101 optimization cycles for the three strategies are shown in Figure 6. The distribution of E_y is shown as well. It is interesting to note that the displacements of the best cavities for the three strategies are significantly different. The modal volumes of the cavity modes are also provided in Figure 6: the V_cav of the optimized cavities are 0.73, 0.68, and 0.74 (λ/n)³ for strategies (A), (A+B), (A+C), respectively, which are slightly larger than that of the base structure [0.61 (λ/n)³ as shown in Figure 2]. We thus confirmed that the increase in V_cav is less than 22%, which is much smaller than the presently achieved increase in Q (increase by a factor of about 10³). Therefore, the optimization of Q in the present case is almost equivalent to the optimization of Q/V_cav.

Figure 4:

The highest Q factor of the additional 70 sample cavities generated in one round as a function of the size of the dataset.

The results for the three different strategies (A), (A+B), and (A+C) are shown with blue, orange, and green curves, respectively. The highest Q factors of the candidates identified in 101 rounds of optimization of the regression function are 5.75×10⁶, 9.12×10⁶, and 1.10×10⁷ for strategies (A), (A+B), and (A+C), respectively.

Figure 5:

Inter-structure distances between the structure with the highest Q factor in the present round (at ${{\overrightarrow{x}}^{\prime }}_b$ in the parameter space) and that in the previous rounds (at ${\overrightarrow{x}}_b$) as a function of the size of the dataset.

The inter-structure distance $|{\overrightarrow{x}}_b^{\text{'}}-{\overrightarrow{x}}_b|$ indicates how large the modifications in each round of optimization are. The results for the three different strategies (A), (A+B), and (A+C) are shown with blue, orange, and green curves, respectively.

Figure 6:

Air hole displacements of the optimized cavities.

The results for three different strategies (A), (A+B), and (A+C) are shown in (A), (B), and (C), respectively. The circles represent the determined optimum positions of the air holes, and the arrows represent the displacement vectors with the scale shown by the red arrow. The electric field distributions of E_y are also plotted in color. The highest Q factors of the structures generated by our proposed method during 101 iteration cycles are 5.75×10⁶, 9.12×10⁶, and 1.10×10⁷ for strategies (A), (A+B), (A+C), respectively.

At first, we compare the results of the iterative optimization proposed in this report with the optimization results of the previously reported one-round NN-based optimization method [19]. For accurate comparison, the same dataset sizes need to be employed. The size of the training dataset that is utilized in different iteration steps of the new optimization method exceeds 1000 after the first round. On the other hand, the number of random cavities in the training dataset that can be utilized for the verification of the previous optimization method is only 1000. Therefore, we generated 1050 random sample cavities in addition to the initially prepared 1000 random cavities (2050 cavities in total), calculated their Q factors, and performed the previously proposed optimization method [19] using these data. The following conditions were used for this optimization: only one NN is employed, and this NN generates five different candidates using the artificial loss (A) with five different λ′ (1.0, 0.5, 0.1, 0.05, 0.01), and the structure with the highest Q among the 2050 cavities (No. 158, Q=8.12×10⁴) serves as the initial structure for the gradient method, in order to reproduce the condition used in Ref. [19]. The Q factor of the best structure obtained with this method is 1.41×10⁵. This result has to be compared with the result of the 16th round of the present iterative optimization method, because here the size of the training dataset equals 2050. As shown in Figure 4, the Q factors of the best structures after the 16th round of the iterative optimization method are 1.91×10⁶, 3.99×10⁶, and 4.32×10⁶ for the strategies (A), (A+B), and (A+C), respectively. The values obtained by the new method are 13–30 times larger than the one obtained by the previous method. This evidences that the proposed iterative method is very effective compared to the previous method.

The reason for this significantly different behavior is explained with the distributions of Q factors in the training datasets provided in Figure 7. In the case of random structure generation, the distribution of the Q factors in the dataset does not change largely even if we increase the number of the random structures from 1000 (Figure 7A) to 2050 (Figure 7B). Because the Q factors of cavities optimized by extrapolation cannot largely exceed those in the dataset, it is difficult to increase the Q factor by simply increasing the number of randomly generated sample structures. In the case of the new method we employed the distribution in Figure 7A as initial training set, but in the course of the 15 iteration cycles (resulting in 1050 new structures generated by the NNs), the distribution of the Q factors at the learning phase of the 16th round significantly expands to higher values as shown in Figure 7C. This result evidences that iterative expansion of the dataset is more effective than to simply increase the number or randomly generated structures.

Figure 7:

Histrograms of the Q factors in the training datasets generated by different methods.

(A) The initial 1000 random patterns. (B) The initial 1000 pandom patterns+1050 newly generated random patterns. (C) The initial 1000 random patterns+1050 patterns generated by the regression function provided by the NN according to strategy (A+C).

The effect of iterative optimization is more apparent when we compare the learning result for the initial dataset and that for the 101st round of the optimization. Figure 8A shows an example of the correlation between Q_NN (predicted by the NN) and the actual Q (calculated by FDTD) after learning the initial 1000 random structures. A good correlation between Q_NN and Q can be confirmed. Therefore, the regression functions obtained by training NNs are useful for extrapolation of new candidate structures [19]. However, only structures with Q factors less than 8×10⁴ are included in the initial dataset. The color scale indicates the frequency of samples structures within a certain range of Q factors: the yellow region in Figure 8A indicates that most sample structures have a Q slightly less than 10⁴. Therefore, it is impossible to extrapolate structures with Q factors on the order of 10⁷ from the initial dataset only. To solve this issue, we proposed the use of iterative expansion of the dataset. Figure 8B shows the correlation between Q_NN and Q after learning the 8000 structures that include 7000 new structures generated according to strategy (A+C). Also here, the correlation is good, and additionally a considerable amount of sample structures near Q=10⁷ can be confirmed. This demonstrates that the new iterative method successfully generated additional sample structures that are necessary for large improvements of Q factors.

Figure 8:

Correlation between Q factors calculated by the 3D-FDTD method (Q) and predicted by neural networks (Q_NN ) at different stages.

(A) The correlation after the learning phase of the first round where the initial 1000 random structures have been used as training dataset. (B) The correlation after the learning phase of the 101st iteration step of strategy (A+C) where 7000 automatically generated structures and the initial 1000 random structures have been used in the training. The correlation coefficients and the decision coefficients are provided on the upper side of the graphs.

In the following we compare the three strategies with respect to the best Q and the computational costs. The best Q factors found with strategies (A+B) and (A+C) are about 1.6 and 1.9 times larger than that of (A). The differences in the improvement ratios originate from the differences in the methods of generating candidate cavity structures. In strategy (A), candidates for high Q structures are generated by exploring the parameter space following the gradient of Q_NN predicted by the trained NN while keeping the distance from the best structure in the previous rounds small. This constraint is controlled by parameter λ′, which was changed from 3 to 1×10⁻⁵ to generate seven different candidates. Candidates with higher Q factors are frequently generated, but the candidate structures are generated according to the past experience (although some randomness is introduced by the initial structures). Therefore, the possibility to get stuck in a local maximum during the repetition of the optimizations is relatively large. The rapid decrease of the inter-structure distance for this strategy shown in Figure 5 supports this interpretation.

In strategy (A+B), half of the candidates are generated according to the past experience, and half of the candidates are generated by exploring the parameter space near randomly generated initial structures. The latter approach is expected to add diversity to the generated candidates and the training dataset. It is important to note that the latter half is not just a random generation; here, candidates are explored based on experience (the gradient of Q_NN), while the space to be explored is intentionally limited. This can prevent getting stuck in an already known local maximum. This explanation is supported by the larger inter-structure distances for this strategy compared to those for strategy (A) in the early stages of optimization (Figure 5; <4000 samples). It seems that the advantage of this strategy decreases as the number of iteration cycles increases as shown in Figures 4 and 5. We explain this with the higher probability of an overlap between the randomly generated initial structure and some sample in the training dataset after many iterations, reducing the diversity of the generated candidates. However, this strategy detected a structure with a Q factor that is 1.6 times larger than the best structure found with strategy (A). It is noted that the computational cost for this strategy is the same as that for (A), because the computational cost for the artificial loss (B) [Eq. (4)] equals that of the artificial loss (A) [Eq. (3)].

In strategy (A+C), half of the candidates are generated according to the past experience, and half of the candidates are generated by exploring the parameter space according to the gradient of Q_NN while avoiding the space near the already known structures in the training data set. Therefore, unknown parameter space is explored more explicitly compared to the case of using artificial loss (B). The maximum Q factor found with this strategy is 20% larger than that detected with strategy (A+B) as shown in Figures 4 and 6. Moreover, the tendency to detect significantly higher Q factors in the next iteration step is still not saturated even at 101 optimization cycles (Figure 4; the inclination of the green curve is relatively steep). This is in contrast to the case of (A+B) and means that strategy (A+C) can avoid local maxima more effectively. The much larger inter-structure distances of this strategy shown in Figure 4 support this interpretation. The drawback is the increase in the computational cost: as can be seen from Eq. (5), the evaluation cost of the artificial loss (C) scales with the number of samples in the training dataset (N), while the other terms in the loss function do not scale with N as can be seen in Eqs. (2)–(4). However, this evaluation has a marginal effect on the total cost for optimization because the computational cost of the first principle calculation is much larger than that for the training and structure generation [34]. Therefore, the total computational cost for optimization can be considered proportional to the number of sample structures, which directly represents the number of first principle calculations necessary. It is also interesting to note that the evaluation cost of the artificial loss (C) scales much slower than that of the Bayesian optimization discussed later.

Here, we compare the L3 cavity optimization performances of our proposed method and other state-of-the-art optimization methods as a benchmark. The genetic algorithm based method was used in [16] to optimize five parameters in the Si-based L3 cavity and enabled detection of a structure with a Q of 4.2 million by using ~8000 sample cavities. Reference [17] optimized nine parameters in L3 cavity using a leaky component visualization method and found a structure with a Q of 5.3 million by using 200 sample cavities. In comparison, our proposed method was used to optimize 25 parameters of the L3 cavity, and we detected a structure with a Q of 11.0 million by using 8070 sample cavities generated with strategy (A+C). The maximum Q detected by the proposed method is more than 2.6 times larger than that found in [16], while the number of sample cavities that have been used is almost the same.

Compared to the leaky component visualization method, the Q obtained in the present method is more than two times larger. Although the number of sample cavities used in [17] is only about 200, these sample structures had to be explored manually, which usually consumes similar time and more effort compared to the proposed automated method. Therefore, our proposed method has provided a structure that is more optimized than those of the two previous methods, while the computational costs are similar. We consider that the higher optimization efficiency of the proposed method has two origins: a training database that contains all experiences accumulated during the whole calculation and also the aggressive search of unknown parameter space, which results in generation of candidate structures that are useful for the optimization.

From a generic point of view, other approaches based on a combination of direct and inverse NNs can be considered as well [35], [36], [37], [38], [39], [40]. These methods are able to derive a target structure with given (desired) properties in case that the target response of the physical system (the input structure) can be well interpolated by a direct NN using the training dataset. However, we believe that these approaches are not very efficient in our case, because our purpose is to obtain structures with Q factors much larger than those in the initial training dataset. Therefore, it is necessary to implement a method that iteratively generates physically more significant candidate structures that are necessary for a large improvement of Q. Theoretically, it should be possible to employ NNs for the generation of such candidate structures, but prior to that we have to study suited NN learning strategies. This is considered to be an important topic of future research.

We also note that the idea of dimensionality reduction may help in some cases. For example, it has been shown that the problem of optimizing a grating structure with five tuning parameters can be reduced to search of the optimum on a 2D hyperplane [41]. This has been achieved by using an initial simple optimization via search from random starting points and sub-space identification based on the principal component analysis. The dimensionality reduction enables full search over the reduced parameter space, which requires less computational resources than the exhaustive search in the original design space. However, it is likely that this strategy is difficult to apply when the dimension of the tuning parameter is much larger (e.g. 25 dimensions as in the present case), because the characteristics of such a problem usually cannot be represented by a much smaller number of parameters.

Finally, we compare the proposed method and the well-known Bayesian optimization in the context of generic optimization methods. The Bayesian optimization is a powerful tool to optimize a black-box function that is expensive to calculate [42], [43]. In this method, an approximate function (usually a so-called Gaussian process [43]) that predicts not only the mean but also variance (uncertainty) of the values of the black-box function is generated by using the present dataset (the training dataset in our case). Then, an acquisition function that evaluates the probability of obtaining better values is prepared based on the predicted mean and uncertainty. As a new observation point (= candidate), the point with the highest value of the acquisition function is searched in its parameter space, and the value of the black-box function at this observation point is calculated and added to the dataset. This procedure is iterated many times. We note that the approach of our proposed method constitutes almost the same procedure. As explained in Section 2, our framework uses a NN to construct an approximation function of the black-box function $Q\mathrm{(}\overrightarrow{x}\mathrm{)}.$ The artificial loss (C) roughly evaluates the inverse of the uncertainty, and the use of several NNs trained with different data feeding orders also corresponds to the evaluation of the uncertainty of the predicted values.

However, there are important differences between the Bayesian optimization and our proposed optimization method. One is the computational cost for the search of candidate structures in the high-dimensional parameter space (this applies to the so-called normal Bayesian optimization only), and the other is the computational cost of the learning of a large-scale training dataset (this also applies to other types of Bayesian optimization). Concerning the former difference, the normal Bayesian optimization usually uses direct search methods without relying on derivatives [43], and therefore sufficient search in a high-dimensional parameter space is impossible from the viewpoint of computational cost (a practical parameter space is usually limited to less than 10 dimensions) [43], [44], [45]. The gradient method starting from random initial points would be useful for the search in high-dimensional space, but it is difficult to implement because the gradient of the acquisition function in the normal Bayesian optimization tends to be 0 over a wider region in the parameter space as its dimension increases, because of the characteristics of the kernel function [44]. The Bayesian optimization with the elastic Gaussian process [44] can overcome this issue, but computation costs for the learning of large-scale datasets remain high as discussed later. The random embedding method [45] can also treat high-dimensional parameter spaces but only under the rather restrictive assumption that the numbers of important dimensions are very small. In contrast, our method is able to effectively utilize the gradient method in high-dimensional space because the gradient of the loss function does not disappear owing to the ReLU nonlinear layers in the NNs and the properly designed artificial loss terms. Therefore, a parallel trial of gradient-based searches starting from many randomly generated initial points works well in our case.

Regarding the relationship between the scale of the training dataset (N) and the computational cost for the training, the cost in the Bayesian optimization scales with N³ because the inverse of a N×N matrix has to be calculated for the training [43]. (Ref. [46] utilized a deep neural network with a Bayesian linear regressor in the last hidden layer to resolve this issue. However, the maximum feasible dimension of the parameter space is still limited because direct (parallel) search is utilized [46]). Fortunately, the training cost of a NN only scales with N¹, and thus a large-scale dataset can be employed to increase the precision of the candidate search, which is especially important for the optimization in a high-dimensional parameter space. In addition, our method employs the gradient method starting from random initial points, which enables efficient exploration of a high-dimensional space. In total, it is considered that the proposed approach benefits from the characteristics of a NN-based regression, which enables training of a large-scale dataset and search in a high-dimensional parameter space while introducing the policy of the Bayesian optimization (i.e. the mean and variation of the prediction are taken into account).

The deviations of the actually fabricated geometry from the intended design can be classified into the average deviations of air hole radius r and slab thickness t from the corresponding design values and the random fluctuations of radii and positions of the air holes.

Figure 9 visualizes the influences of variations in the average r and t on the Q factors of the optimized structures obtained after 101 iteration steps by the three different strategies. It can be confirmed that the Q factor depends weakly on the slab thickness, while a change of the air hole radii on the order of 2 nm is detrimental (the Q factor decreases by about 50%). Fortunately, r (and t) can be adjusted in steps on the order of about 0.5 nm (1 nm) using a post process consisting of surface oxidization and oxide removal [47].

Figure 9:

The influences of changes in the air hole radii r and the slab thickness t on the Q factors.

The results for the cavity designs optimized by the strategies A, A+B, and A+C, are shown in (A), (B) and (C), respectively. The values employed in Section 3 are r=102.5 nm and t=220 nm.

Figure 10 shows the influence of random fluctuations of hole radii and positions on the Q factors. To obtain a reliable statistical statement, we considered 200 samples with the same design and randomly applied small changes to all air hole positions (in x and y direction) and their radii without keeping the symmetry of the structure to reflect random fluctuations due to finite fabrication accuracy. The applied random deviations follow a Gaussian distribution with a standard deviation of a/1000, and the same 200 random deviation patterns are applied to the cavities optimized by the three different strategies. This magnitude of the randomness corresponds to actual fabrication errors determined from experiments on state-of-the-art photonic crystal nanocavities [9]. Figure 10C clarifies that the random structural fluctuations lead to an average Q factor of 4.47×10⁶ for the case of the cavity with a predicted Q factor of 1.10×10⁷ in the ideal case (optimization result of Strategy A+C). For the case of the cavity with a Q factor of 5.75×10⁶ in the ideal case (Strategy A), the average Q factor decreases to 3.14×10⁶ due to the same structural fluctuations. Although the Q factors obviously degrade due to the random fluctuations, the average and the highest Q factors including effects of fabrication errors are still larger for cavity designs with a larger Q factor in the ideal case. Therefore, the designs with higher Q factors determined by our method have a significant impact even when fabrication errors are taken into account.

Figure 10:

The influences of random fluctuations of air hole radii and positions on the Q factors.

The results for the cavity designs optimized by the strategies A, A+B, and A+C are shown in (A), (B), and (C), respectively. The random deviations follow a Gaussian distribution with a standard deviation of a/1000, and the same 200 random deviation patterns are applied to the cavities optimized by the three different strategies.