Today, the rapid development of computer technology changes with each passing day. In the computer field, computer animation has rapidly grown from a new thing to a leading industry, and animation has entered the era of three-dimensional animation and computer graphics. This article aims to study the application of artificial intelligence-based style transfer algorithm in animation special effects design. It proposes methods such as adaptive loss function, style transfer process, animation special effect design, etc., and conducts related experiments on the application of style transfer algorithm in animation special effect design in the article. The experimental results show that the style transfer algorithm based on AI can effectively improve the effect of animation special effects. In this survey, more than 80% of the people are satisfied with the animation special effects design based on the style transfer algorithm.
Animation is one of the four major types of movies (feature films, pioneer films (or experimental films), cartoons, and documentaries). Animation mainly uses exaggeration, similarity, distortion, fantasy, and other expression methods to express people’s ideals and desires in social life. Various algorithms based on style transfer can realize the diversification of animation special effect images to improve the display of animation; style transfer based on texture similarity can better integrate textures and obtain better performance results to achieve the special effects and purposes of 3D animation.
The development of animation technology is inseparable from the continuous progress of human science and technology. The advancement of human technology not only embodies human wisdom, but also embodies the endless pursuit and exploration of unknown things. With the rapid development of animation special effects processing technology in recent years, style transfer, as an important research topic in the field of image processing, still has very important application value in changing image segmentation and texture styles. For the texture style of the image, the texture conversion and fusion can be performed more naturally through style transfer. With the development of virtual reality technology, style transfer also has important application value in this field. Since users need to obtain a high degree of natural feedback in a simulated environment, in 3D real-time modeling, the desired scene can be quickly changed through style transfer. This makes the built scenes more natural, and the virtual reality and color scenes are also strengthened.
With the continuous development and progress of social science, people’s research on artificial intelligence (AI) is also deepening. In his research on AI, Hassabis et al. investigated the historical interaction between AI and neuroscience, and emphasized the current progress of AI inspired by neural computing research in humans and other animals . As the most advanced mobile communication network in the contemporary 5G mobile communication network, the blessing of AI technology is of course indispensable. Rongpeng’s research highlights the opportunities and challenges of using AI to implement smart 5G networks, and demonstrates the effectiveness of AI in managing and orchestrating cellular network resources . Its machinery fault diagnosis is of great significance to the reliability and safety of modern industrial systems. As an emerging field of industrial applications and an effective solution for fault identification, AI technology has attracted more and more attention from academia and industry. Lee et al. briefly introduced the application of AI in stroke imaging, paying special attention to its technical principles, clinical applications, and future prospects . As a kind of AI algorithm, style transfer algorithm has also received a lot of attention from researchers in recent years. In his research, Zhang et al. developed a semantic correspondence-guided photographic style transfer algorithm. When he transfers the color of the style image, he ensures that the semantic structure of the content image does not change, while retaining the semantic correspondence between the style of the landscape and portrait image and the content image . In the research by these researchers, most of them have carried out corresponding investigation on the application of AI. In their study, most people also ignore the corresponding research on the development of AI itself. The innovation of this article is to study the style transfer algorithm based on AI, and to conduct a very detailed research on the pre-training model, style transfer process, and style transfer loss function of the style transfer algorithm. It also conducts corresponding research on the adaptive instance normalized image migration algorithm.
2 Style transfer algorithm and animation special effects design
2.1 Style transfer algorithm
Style migration algorithm refers to the use of given content samples and style samples, the computer generates new samples that not only have content characteristics, but also have artistic style characteristics. The generated sample not only retains the outline and shape of the target content sample, but also retains the color and texture of the style sample. Style transfer applications are very popular on social networking sites, and users can share and exchange their dreamy pictures. At the same time, style transfer can also act as an auxiliary tool for painters, helping painters to create works of art with specific styles more conveniently. Figure 1 shows some pictures generated based on image style transfer.
2.1.1 Selection of pre-training model for style transfer
The training process of deep learning algorithms related to images requires the use of appropriate pre-trained models as feature extractors. An excellent pre-trained model has been perfectly trained on large datasets and large GPU clusters. It can provide a set of neural network weight parameter values with high adaptability, which is beneficial to other neural network training tasks for feature extraction . VGG16 has 13 convolutional layers in total, denoted by conv3-XXX, respectively. The 3 fully connected layers are represented by FC-XXXX, and the 5 pooling layers are represented by MaxPool. Convolutional layers and fully connected layers have weight coefficients, so they are also called weight layers. The total number is 13 + 3 = 16, which is the source of 16 in VGG16. The VGG network is still a pre-training model for the mainstream deep learning of style transfer, as shown in Figure 2.
The given input image will be abstracted into a set of filtered feature vectors in the middle process of feature extraction, that is, abstract feature expression . The abstract expression process of image features is reconstructed by using the abstract feature expression of the designated middle layer to finally express the expression of image information in different layers . Due to the black box characteristics of CNN, the style transfer process is uncontrollable, and it is difficult to achieve finer control in the image transfer process.
2.1.2 Design of loss function for style transfer
The style transfer process requires two pictures, a content input picture and a stylized picture. Style transfer is to extract the style from the style picture to describe the content and input the picture .
where Losstotal is the total loss function, which is also the target of the iterative calculation process; is the loss function of the content image, is the loss function of the style image, and O represents the Oth layer model of the depth model.
The style transfer algorithm needs to solve the following two basic problems: the first is the problem of structure preservation, the second is the problem of semantic accuracy, the semantic scene of the original image must be respected .
2.1.3 The calculation process of style transfer
The style transfer process includes three main steps: calculating partial derivative functions, synthesizing gram matrix, and calculating style loss. The details are shown in Figure 3.
The process of style transfer extracts content and style features from content and style images, respectively, and recombines these two features into the target image. After that, the target image is reconstructed online iteratively, based on the difference between the generated image and the content and style image.
2.1.4 Adaptive instance normalized image migration algorithm
184.108.40.206 Adaptive instance standardized network structure and principle
The adaptive instance normalized image migration algorithm is given a content image input and a style image input. Adaptive instance normalization (AdaIN) adjusts the mean and variance of the content input to match the input style image to realize image style transfer. Among them, IN is more suitable for style transfer than batch standardization (BN). BN is mainly for data processing of batch samples, which will be affected by other samples during calculation. In the face of style transfer, the overall information obtained by BN is equivalent to introducing noise for a single data, but it weakens the independence between instances. This method is suitable for classification problems where the results are related to the overall distribution . IN is mainly used for data processing on a single image, and all confidence comes from its own image. It can be understood as a normalization operation of data, an integration of global information, and overall distribution of data. It is suitable for image processing operations such as style transfer. The feature mapping of IN and BN is shown in Figure 4.
In Figure 4, Y represents the number of samples included, T represents the number of channels for each sample, and G and K represent the height and width of the feature map, respectively. The sample normalization operation formula is as follows:
where θ represents the affine parameter learned from the data. The formulas for mean value v and standard deviation δ are
For the data mean value ν and standard deviation δ of the G and K dimensions of the sample, the T and Y dimensions are retained, as shown in the following formula:
where χ is a small constant, and the difference from BN is that ν(n) and δ(n) of IN are calculated independently for each channel and each sample. These two methods are usually used to normalize the data to improve the convergence speed of the model and improve the accuracy of the model.
Adaptive instance normalization proposes to no longer learn affine parameters based on the above theoretical basis, and adaptively calculates affine parameters through the input style image . Assume that the input content image I and the input style image S are input. After the content image is normalized, the mean value is 0 and the standard deviation is 1, and multiply by the standard deviation of the style image and add its average value. It is shown in formula (10).
where ν(n) and δ(n), respectively, represent the mean value and standard deviation of the content image. Similarly, ν(S) and δ(S), respectively, represent the mean value and standard deviation of the style image. Align the mean value and standard deviation of the content image with the mean value and standard deviation of the style image through the above formula. The style transfer is carried out in the feature space by transferring the feature statistical information, and the structure of the adaptive instance normalized style transfer network model is shown in Figure 5.
220.127.116.11 Adaptive loss function
According to the upper right part shown in the network structure diagram, r(f(x)) can be understood as the result obtained by re-inputting the obtained result into the VGG, and then calculating the content loss function S v with the target feature m. Then, S v can be expressed as a formula
The style loss function S f is defined as the regularization of the difference between the mean value and the standard deviation of the feature map between the style image I x and the generated image I y. Then, S f can be expressed as:
where n represents the feature layer used to calculate style loss in VGG16.
The loss function S still consists of two parts, including content loss and style loss. The loss function S can be expressed as:
where λ represents the weight, which is a hyperparameter that balances the content and style.
2.1.5 The style transfer process
It selects two pictures from the training set, namely, style A and style B, and an image T with the same size as the content map and input them into the network. T will eventually become a new composite figure P with A style and B content.
When performing image style transfer, first extract the content image features. It first inputs image B into the VGG16 network, and selects the required sample feature map. Assuming that the layer number of the selected sample feature map is the ith layer, x i is the number of feature maps in this layer, and y i is the number of pixels in this layer of feature maps. Then, the feature map of this layer can be expressed as
Similarly, image A is also input into the network, and the ith layer feature map of image A is set to T i . When the values of T i and H i are very close, it can be considered that A and B are very close in the image content. It can be concluded that the content loss function of A and B is
It defines the weight of the content loss function as 0.025, from which we can calculate the gradient value of the content loss function and , so as to realize the gradient update from back to forward
Style image feature extraction: The extraction of texture features from the style map is different from the feature extraction of content images. The style map does not need to consider the specific location of the image information, so Gram matrix is needed when extracting
Q i is the feature map formed by disrupting the feature map of the style image in the i layer and the Gram matrix. R i is the mean square error of Q i and C i , and R i can be expressed as
The loss function is:
Finally, the value of formula (20) can be used to update the content of the target image A
2.2 Animation special effects design
Computer animation technology is a new multimedia technology that combines computer technology and animation art. Its technology is mainly reflected in the computer animation production software and hardware. In terms of art, computer technology has become a new tool for modern artists to create art. Animators have removed the heavy and boring work of animation in the past, and instead spent more energy on creative work. On the other hand, the development of the animation industry puts forward higher requirements for computer engineers and also promotes the development of computer animation software. At present, common animation special effects include wind, rain, thunder, electricity, flame, explosion, etc.
Since the special effects art of three-dimensional (3d) animation was hardly understood a few years ago, people’s awareness of its acceptance is still relatively weak. However, with the advent of the digital age and the rapid development of computer technology, the special effects art of 3D animation has become a leader in the fields of film, television, advertising, and medical treatment. Especially, in movies and advertisements with science and magic as the theme promoted by modern technology, the special effects of 3D animation have become an indispensable means of artistic expression.
2.3 AI technology
The rapid development of AI in recent years and its rich research results are exciting. In its essence, it is a simulation of human thinking. Thinking is a subjective reflection of objective reality, so thinking is both subjective and objective .
AI technology is a highly comprehensive subject. The development of artificial intelligence technology to the present has been widely used in the fields of industry, service industry, autonomous driving, medical industry, and so on. Figure 6 shows the areas where AI technology is more widely used.
In medicine, artificial intelligence is an effective tool for electronic medical records and sharing medical information. Integrating machine learning-based modeling specifically designed for managing datasets can help detect potential complications. It is the result of improving the utilization of medical resources and the level of personalization. Using natural language processing, an artificial intelligence tool, it can read and process electronic medical records in context. This gives users a good way to accurately compile and link various electronic medical records data accumulated over decades.
3 Animation special effects design experiment based on style transfer algorithm
3.1 Comparison experiment of the time required for different style transfer algorithms to generate pictures
In this experiment, the time required for different style transfer algorithms to generate pictures will be counted. During the experiment, pictures of different sizes were used. Table 1 shows the statistical table of processing time required by different algorithms.
|Size of the picture||Gatys et al.||Johnson et al.|
|10 Iterations (s)||50 Iterations (s)||100 Iterations (s)|
|256 * 256||2.3||3.8||5.4||0.05|
|512 * 512||5.8||8.7||13.6||0.31|
|1,024 * 1,024||9.6||15.8||30.9||0.93|
It can be seen from Table 1 that the classic neural style transfer algorithm proposed by Gatys takes a lot of time to iteratively optimize the generated pictures. The table shows the time it takes to process images of different resolutions in 10, 50, and 100 iterations. Johnson algorithms do not need to be iterated after training, so their execution speed is much faster than the classic neural style transfer algorithm. The fast style transfer greatly reduces the running time required by the algorithm while ensuring the quality of the generated image, making real-time style transfer possible.
In this experiment, the flexibility results of Gatys algorithm and Johnson algorithm are also compared as shown in Table 2.
|Gatys et al.||Johnson et al.|
|(1)||Whether to support any style||Yes||No|
|(3)||Picture processing speed||Slow||Fast|
First of all, this experiment discussed whether the algorithm can render any style. The algorithm proposed by Gatys supports arbitrary style conversion. When rendering different styles, it only needs to switch the corresponding style pictures. The method proposed by Johnson is based on model optimization, and a model can only correspond to a single style. The second is whether various algorithms need to have a model training process. The algorithm proposed by Gatys is based on the iterative optimization of pictures. The algorithms proposed by Johnson are based on model optimization, and the model needs to be trained before it has the ability to transform styles. Finally, this experiment discussed the speed of each algorithm for image rendering. Since the algorithm proposed by Gatys requires multiple iterations to optimize the picture, its processing speed is slow. In the algorithm proposed by Johnson, a feedforward propagation method is used to process pictures, so the speed is very fast and can be used for real-time style rendering.
3.2 Style transfer network model training experiment
Animation special effects refer to special effects that are “made frame by frame.” Therefore, it is very necessary to carry out experimental statistics on the loss of the image. The indicator to determine the purpose of style transfer is to observe the changes in their loss function and weight parameters. This experiment obtained the total style loss function, total content loss function, and total loss function of the entire model through image input. Table 3 shows the data of part of the total style loss function, total content loss function, and total loss function.
|Total style loss function||Total content loss function||Total loss function|
3.3 Animation special effects design experiment based on style transfer algorithm
In this experiment, a mass satisfaction survey was conducted on the effects of animation special effects based on the style transfer algorithm. A total of 500 public satisfaction questionnaires were distributed in this survey, and 480 were recovered, with a recovery rate of 96%. In this survey activity, people of all ages and genders actively participated. Table 4 shows the distribution of the participants in this survey.
This experimental investigation is to mainly investigate whether the animation effect produced based on the style transfer algorithm meets the needs of the public and whether it has a certain feasibility. Table 5 shows the statistical table of the results of this survey activity.
|Fluid special effects||9||7.6||8.2||8.8||7|
|Particle special effects||8||8.8||6.9||8.6||8|
|Cluster special effects||7||9||7.1||8.2||9|
4 Result analysis of animation special effects design experiment based on style transfer algorithm
4.1 Comparative experimental results of the time required for different style transfer algorithms to generate pictures
In the experiment, the time required for the pictures to be generated by two different style transfer algorithms was recorded. According to the experimental data, a comparison chart of the time required for two different style transfer algorithms to generate pictures can be drawn. The details are shown in Figure 7.
According to Figure 7, it can be seen that the classic neural style transfer algorithm proposed by Gatys takes longer to generate pictures. The time to generate pictures by the classic neural style transfer algorithm proposed by Gatys is generally higher than 2 s.
4.2 Experimental analysis of style transfer network model training
In the training experiment of the style transfer network model, the data of the total style loss function, the total content loss function, and the total loss function were recorded. According to the data in Table 3, the training result of the style transfer network model can be obtained, as shown in Figure 8.
Figure 8(a) shows the total style loss function. It can be seen that the longitudinal indicator of the function gradually tends to 0 with less fluctuation. The more the artistic image textures are extracted, the richer the picture texture; Figure 8(b) shows the content image loss function graph, that is, the picture that needs to be processed. It can be seen from the image that the function value has been fluctuating from l × 104 to 3 × 104 after falling from 5 × 104. The content image still has the more detailed content characteristics of the image after the style transfer process. In other words, we can still see the content in the image after the conversion, but the style image cannot be distinguished. Therefore, the fluctuation range of the content image loss function is not large, and it is also a manifestation of tending to be stable. Figure 8(c) shows the weighted total loss function. It can be seen that the total loss function is affected by the style loss function and the content loss function. After the function drops from 2.5 × 105 to 5 × 104, it fluctuates in a small range. It tends to a stable state as a whole and has successfully converged.
4.3 The results of the investigation experiment of animation special effects design based on the style transfer algorithm
In this research activity, all age groups actively participated. According to Table 4, a basic distribution map of the population participating in this survey can be obtained, as shown in Figure 9.
It can be seen from the figure that the number of teenagers and youth groups participating in this survey accounted for more than 50%. This also reflects from another aspect that these two age groups pay more attention to animation than other age groups. This also requires the majority of animation producers to always pay attention to the needs of these two groups when making animations, and produce some animation works that meet the pursuit of the public according to the needs of the public. According to the data in Table 5, a result graph of people’s satisfaction with the animation special effects based on the style transfer algorithm can be obtained, as shown in Figure 10.
According to Figure 10, it can be seen that people are quite satisfied with the animation special effects design based on the style transfer algorithm. The overall score exceeds 7.5 points (the full score is 10 points, the scoring range is 1–10 points, the higher the score, the higher the satisfaction), which is a satisfactory result. Animation special effects design based on style transfer algorithm can integrate different styles of creative methods into animation special effects. This enriches the content of special effects, makes animation special effects more enjoyable, and makes people more satisfied.
Through the research of this article, the following conclusions can be drawn: AI-based style transfer algorithm can quickly and effectively generate a variety of image styles. Can greatly improve the work efficiency of animation special effects designers, and reduce personnel input and waste of material resources. Due to the continuous development of science and technology, AI has been widely used in animation production. It can not only provide certain reference materials for animators but also continuously stimulate the imagination and creativity of the animators.
There are many shortcomings in this research. In the future, in-depth research on animation special effects design will be carried out, and will strive to create more and better research results.
Funding information: This work was supported by the Project supported by Southwest music research center of Sichuan social science key research base (expansion) (xnyy2017004).
Conflict of interest: The author declares no conflicts of interest to report regarding the present study.
Data availability statement: The data that support the findings of this study are available from the corresponding author upon reasonable request.
 D. Hassabis, D. Kumaran, C. Summerfield, and M. Botvinick, “Neuroscience-inspired AI,” Neuron, vol. 95, no. 2, pp. 245–258, 2017.10.1016/j.neuron.2017.06.011Search in Google Scholar PubMed
 R. Li, Z. Zhao, X. Zhou, G. Ding, and Y. Chen, “Intelligent 5G: when cellular networks meet AI,” IEEE Wirel. Commun., vol. 24, no. 5, pp. 175–183, 2017.10.1109/MWC.2017.1600304WCSearch in Google Scholar
 E. J. Lee, Y. H. Kim, N. Kim, and D. W. Kang, “Deep into the Brain: AI in stroke imaging ,” J. Stroke, vol. 19, no. 3, pp. 277–285, 2017.10.5853/jos.2017.02054Search in Google Scholar PubMed PubMed Central
 X. Zhang, X. Zhang, and Z. Xiao, “Deep photographic style transfer guided by semantic correspondence,” Multimed. Tools Appl., vol. 78, no. 24, pp. 34649–34672, 2019.10.1007/s11042-019-08099-7Search in Google Scholar
 C. Cath, S. Wachter, B. Mittelstadt, M. Taddeo, and L. Floridi, “AI and the ‘Good Society’: the US, EU, and UK approach,” Sci. Eng. Ethics, vol. 24, no. 7625, pp. 1–24, 2017.Search in Google Scholar
 M. Hutson, “AI faces reproducibility crisis,” Science, vol. 359, no. 6377, pp. 725–726, 2018.Search in Google Scholar
 J. H. Park, S. Park, and H. Shim, “Semantic-aware neural style transfer,” Image Vis. Comput., vol. 87, no. JUL, pp. 13–23, 2019.10.1016/j.imavis.2019.04.001Search in Google Scholar
 Y. Chen, Y. Hua, and H. Guanghui, “3D product display based on inventor animation design,” Instrumentation, vol. 7, no. 1, pp. 35–43, 2020.Search in Google Scholar
 N. Jones, “Book review: Plastic reality: special effects, technology, and the emergence of 1970s blockbuster aesthetics turnock Julie A, Plastic reality: special effects, technology, and the emergence of 1970s blockbuster aesthetics, New York: Columbia University Press,” Animation, vol. 13, no. 1, pp. 93–95, 2018.10.1177/1746847718754769Search in Google Scholar
 Z. Shen, “Agricultural climate change and rural animation film application based on remote sensing images,” Arab. J. Geosci., vol. 14, no. 7, pp. 1–18, 2021.10.1007/s12517-021-06858-1Search in Google Scholar
 X. You, F. Tian, and W. Tang, “Highly efficient facial blendshape animation with analytical dynamic deformations,” Multimed. Tools Appl., vol. 78, no. 18, pp. 25569–25590, 2019.10.1007/s11042-019-7671-8Search in Google Scholar
 J. Yang, G. Sun, M. Xing, X. Xia, Y. Liang, and Z. Bao, “Squinted TOPS SAR imaging based on modified range migration algorithm and spectral analysis,” IEEE Geosci. Remote. Sens. Lett., vol. 11, no. 10, pp. 1707–1711, 2017.10.1109/LGRS.2014.2306433Search in Google Scholar
© 2023 the author(s), published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.