As known, differential equations occur in many science phenomena. Whenever, there is a meaningful relation between different values, states or times in which the rate of variable changes at different times or states are known, it can be modeled via differential equations. Hence, various applications of differential equations have led mathematicians and engineer scientist to focus on appropriate methods in solving these kinds of mathematical problems. More complicated differential equations arising from the modeling of complex phenomena can not be solved simply by existing conventional methods. Therefore, finding alternative numerical techniques seems to be necessary. During recent decades, various numerical techniques have been used for approximating different types of differential equations in linear and nonlinear cases. Among these, we can mention the homotopy perturbation method [2, 4, 5], variational iteration method [21, 22, 26], Adomian decomposition method [1, 3, 10], etc. An effective method in solving a wide variety of complicated mathematical problems is artificial neural networks (ANNs) approach. Since, these networks have high efficiency in approximating solutions of mathematical problems. Recently, some structures of neural networks have been applied for solving variety of mathematical problems [11, 12, 13].
In this paper, a suitable structure of neural networks will be applied for solving initial-boundary one-dimensional wave-like and heat-like equations. The heat-like equation can be a mathematical modeling for temperature changes of the composite materials. The solution of the mentioned partial differential equation is proportional to kinetic energy of particles in the material. Meanwhile, the wave-like equation can describe moving the particles under Hook’s law in the disordered systems. In this kind of equation, the standard initial and boundary conditions can be considered. In recent years, some researchers have solved numerically these problems [14, 18, 19, 20, 24, 25]. To do this, a four-layer feed-forward neural network corresponding to the mentioned bivariate power series solution is designed satisfying both initial and boundary conditions. The designed neural architecture can easily approximate solution functions on the space of unknown constant coefficients using a suitable learning algorithm. In other words, after discretizing the differential domain using a standard rule and doing some simplifications, the origin problem is transformed to solve a minimizing optimization problem. The yield nonlinear problem is solved iteratively using a back propagation learning rule, which is based on the gradient descent method. Approximating the constant series coefficients lead to finding the mentioned series solution on the given differential domain. In Section 2, we will first have an overview of artificial neural nets and their computational process. Then, having the introduction of the proposed neural architecture, the numerical solution of the mentioned differential equations will be considered using the defined combination iterative method. Two numerical examples with computer simulations are presented in Section 3. Also, to better express the effectiveness of the presented technique, the obtained numerical results will compare via the ones achieved from another classical method. Finally, conclusions and recommendation for future research are presented in Section 4.
2 Description of the method
The general purpose of this section is to introduce a combination iterative method for approximating solution of two well known partial differential equations. In order to better clarify all the fundamental mathematical features of the method presented in this research, we first deal with more general theories of neural networks issue. The proposed iterative technique which is based on the combination of power series method and a modification of ANNs approach, is used to find approximate solutions of boundary-initial value wave-like and heat-like equations on a given closed domain.
2.1 Basic idea of ANNs
Artificial neural networks theory revolves around the idea that certain key processing properties of human being brain can be modeled and applied to approximate computational methods using biological processes. In other words, artificial neural networks attempt to get knowledge of the relation between a set of data through training and finally store the knowledge gained for similar purpose. The main ideas of these networks are partly inspired by a way biological nervous system functions, to process data and information for learning and createing knowledge. In a neural network, simple processing elements are known as “neurons”, which can display complex global behavior determined by the connection between processing elements and element parameters. A neural net can not be adaptive itself. The practical application of this network is needed to enable employing algorithms which are designed to alter and adjust unknown parameters. For this purpose, using knowledge of computer programming, we can design a structure that acts as a nervous system. So, a learning algorithm is defined for networks by creating a network of interconnected artificial neurons. Mentioned networks have exhibited high performance on estimation and approximation. There are many references on neural nets field, see for example Ref. [6, 7, 8].
Now, let’s consider the neural architecture shown in Figure 1. This network is a four-layer feed-forward neural architecture with two input signals and one output neuron. Each input signal is multiplied by its respective weight values, and these weighted signals are then perfectly summed to produce the next layer’s inputs. In other words, each hidden or output layer receives its inputs from the previous weighted neurons and then presents it to a suitable activation function. The network output can be calculated. The remarkable notation in this area is that the bias term in output layer is added with the weighted signals of second hidden layer and then makes up the overall input of the final layer. According to the above, each neuron’s input-output relation can be written as follows:
First hidden units:
Second hidden units:
The designed neural structure is a prototype model, which should be created having some minor changes caused by conditions of problem. In other words, by presenting boundary or initial conditions of a given partial differential equation problem, the neural architecture will be ready to learn. This will cause the network output desirability to the solution of the mentioned math problem.
2.2 Implementation of the method
As mentioned before, the main idea of this study is to apply the designed multi-layer feed-forward neural architecture to the function approximation of two famous types of partial differential equations. Consider the one-dimensional wave-like Equation (5) and heat-like Equation (6):
subject to the initial conditions:
and the boundary conditions:
In the above relations, k and h are given continuous positive and real-valued functions, respectively. The defined wave-like and heat-like equations are more applicable in modeling different physical phenomena. The main problem occurs when we encounter more complicated modeled problems of these types, in which the existing analytical or numerical methods can not solve them. Therefore, finding an effective alternative method seems to be necessary. In the last decade, some modifications of the power series method have been employed for solving several types of partial differential equations with initial or boundary conditions [16, 17, 23]. Extensive studies on these polynomials, show that this series solution technique is really a powerful tool for solving complex mathematical problems. As previously mentioned, a combination of these polynomials and ANNs approach will be considered as an iterative method for solving the above partial differential equations. The basic idea in this issue is that the solution function u(x, t) on domain Ω = [0, 1]×[0, T], can be completely represented in a polynomial series of degree (n, m) as:
for the constant coefficients ai,j(for i = 0, …, n; j = 0, …, m). It should be noted that the solution function can be represented as the power series (7) if and only if it is complex differentiable in the open set (0, 1) × (0, T). It is reasonable to consider that the designed neural architecture are fully associated with the polynomial series (7). The interesting point in this approach is that the growing degree of basis polynomials, increases the accuracy of the recommended combination method. It must be considered that, this work will lead to more complex relations.
2.2.1 Discretization of the problems
Power series method is based on the fact that any modification of these series before being used, must be satisfied in the initial or boundary conditions. For the introduced partial differential equations, the trial solution is written as follows:
in which the function A(x, t) has been chosen in a manner that satisfies in the both initial and boundary conditions, simultaneously. The introduced trial function involves the given feed-forward architecture that satisfies in both initial and boundary conditions. Supposedly u͂(x, t) symbolizes the trial solution with adjustable parameter ai,j, the problem is transformed from the original construction to an unconstrained one by direct substitution (8) in the primary equations. So, the Eqs. (5) and (6) are shape changes into the form (9) and (10), respectively:
Now, we intend to define a set of acceptable mesh points for the discretization of Equations. (9) and (10). For positive integers n′ and m′, let Ωp, q be a partition of square Ω with the mesh points (xp, tq) = (for p = 0, …, n′; q = 0, …, m′). Substituting collocation point (xp, tq) into the resulted relations, reduces the problems into the following systems of equations:
Continuing, we intend to construct an iterative procedure with the help of artificial neural networks approach to solve the resulting systems.
2.2.2 Proposed error function
As it is known, in iterative methods a valid criterion is required to measure the produced error of each iteration. Here, the differentiable least mean square (LMS) function is employed to measure the accuracy of solutions. This rule is stated for wave-like and heat-like equations, respectively as follows:
Minimizing the defined error functions over the space of possible weight parameters can be an interested issue. To do this, a set of training rules is build to minimize EW and EH by adaptively adjusting the network parameters. Hence, one suitable error correction technique must be essentially employed to achieve this goal. More details concerning minimizing techniques can be found in Ref. .
2.2.3 Proposed learning algorithm
What makes this particular use of neural networks so attractive in many applications is that they express the ability to learn, though this remarkable property might be challenged by some researchers. In terms of ANNs, “learning” simply means changing the weights and biases of the network in response to some input data. Once a particular learning algorithm succeeds, programming the network to have a particular unequivocal performance is not vitally important. In other words, we need no prior knowledge to adjust the weights and biases. The designed neural architecture adjusts its parameters for a learning algorithm. That is, the error alters as the weights and bias term are changed. In this sense, the neural network learns from experience. Therefore, the back-propagation algorithm is the most widely-used method for feed-forward networks. The learning rules are mathematical formalizations that are believed to be more effective in the ANN’s performance. To fine-tune the neural network, the network parameters are first quantified with arbitrary initial guesses; then, the neural network calculates the output for each input signal. Next, the defined error rule is employed by substituting the proposed network model instead of the solution function in the origin problem. To train the present network, we have employed an optimization technique that in turn required the computation of the gradient of the error with respect to the net parameters.
Now, a suitable error correction rule must be initially used for single units training. This rule essentially drives the output error of the network to zero. We start with the classical generalized delta learning rule and give a brief description for its performance. Throughout this section, an attempt is made to point out the criterion function that is minimized by using this rule. Learning in neural nets is appropriate selecting the connection weights, which yields to minimize the error function on a set of mesh points. During the training, the initial parameters ai,j are put into the network and flow through the network generating a real value on the output unit. As seen in the last part, the calculated output is compared with the desired one, and an error is computed. The differentiable cost functions EW and EH are always decreasing in the opposite direction of its derivative. It means that if we want to find one of the local minima of this function starting from a initial guess. We employ the supervised back-propagation learning algorithm to reach this goal. The mentioned self learning mechanism starts with randomly quantifying the initial parameters ai,j (for i = 0, …, n; j = 0, …, m). The mentioned algorithm is well presented for wave-like equation as follows:
where η and γ are the learning rate and momentum term, respectively. In the above, the index r in ai,j(r) ascribes to the repetition number and the subscript i, j in ai,j is the label of the training connection weight. Moreover, ai,j(r + 1) and ai,j(r) depict the adjusted and current weight parameter, respectively. To complete the derivation of learning procedure for the output layer weights, the above partial derivative can be expressed as follows:
The above computational process can similarly be employed for heat-like equation. To prevent taking much of the time on this part, the behavior of adjusting weight parameters for this equation is not provided. It should be mentioned clearly that Matlab v7.10 is a high quality and easy to use mathematical computing software, which researchers and students can employ to omit wasting time and enhance the accuracy of calculations.
3 Numerical examples
In this section two test problems with computer simulations are provided to illustrate ability and accuracy of the iterative technique proposed in this research. Furthermore, the obtained numerical results of this technique will be compared with those obtained by Taylor series method . Here, the mean absolute error Emid i.e.:
will be implemented to compare the effectiveness of both methods.
The following heat-like equation is considered:
with initial conditions:
and the boundary conditions:
Note that the exact solution of the problem is given as u(x, t) = x3 et. Now, we intend to approximate the solution function by using the defined combination method on the domain Ω = [0, 1]2. Here, we use the regular discretization technique on the given differential domain in x and t directions. We intend to show that the proposed four-layer feed-forward neural architecture is sufficient to solve the defined math problem. Hence, the incremental learning process begins to work by quantifying the connection weights ai,j (for i = 0, …, n; j = 0, …, m) with small real-valued random constants. The convergence speed of back-propagation is directly related to the learning rate and momentum constant parameters. The optimal tuning parameters for fast convergence of back-propagation gradient descent search is the inverse of the largest eigenvalue of the Hessian matrix of the defined error function, evaluated at the local point. Thus, the norm of the converged weight vector gives a good estimate of learning rate in back-propagation. In this study, for better comparison of the obtained numerical and simulation results, we had to use same quantities for rate and momentum parameters. This work makes easy to compare the obtained results from using different initial parameters and iterations numbers. In other words, using same valued learning rate and momentum constant makes better comparison of the obtained results. Then, the training patterns were used to successively adjust the connection weights until a suitable solution was found. Typically, more than one step using the training set is needed to derive an appropriate solution. To demonstrate the accuracy of technique presented in the previous section, the indicated mean absolute errors for different network parameters are shown in Table 1, for n = m and n′ = m″. The absolute errors between the approximate and exact solutions on the mesh points are numerically compared for n = m = 5 and n′ = m′ = 10 in Table 2. As can be seen, with increasing number of iterations the ANNs approach offers more accurate approximations rather than the Taylor series method of order 5. The indicated error function is plotted in Figure 1, for m = n = 5 and m′ = n′ = 5. It can be easily concluded that by increasing the number of iterations, the network error is rapidly reduced until it go to zero. The exact and approximate solutions are plotted in Figure 3. The performance of proposed neural structure for different control elements is using the mean absolute error function in Figure 4.
Consider the following one-dimensional wave-like equation:
with initial conditions:
and exact solution:
Similarly, the proposed neural architecture has been employed for approximating solution of this initial-boundary problem on square Ω = [0, 1] × [0, 0.5] for m = n = 5 and m′ = n′ = 10. The obtained numerical results are presented in Table 2. We are now allowed to claim that the combination method proposed in this paper can be applied to accurately approximate the unknown functions to any desired degree of preciseness. In particular, thanks to this unique characteristic of artificial neural networks, the algorithmic power series method can be converted into an iterative non-algorithmic one. Here, if we consider an arbitrarily large number of iterations, the proposed boosting method will be able to approximate the unknown function with high precision.
Certain properties of artificial neural networks are typically configured to distinguish the networks based on iterative approach from other classical numerical methods. In this research, a combination of ANNs approach and power series methods, was used for numerical solution of two special types of partial differential equations. Wave equations and heat equations are considered as two main boundary-initial value partial differential equations which have played pivotal part in modeling physics phenomena. The proposed approach could convert solving a differential problem into related optimization minimizing problem. This work combined initial and boundary conditions of the problem, which could be easily modeled with suitable network architecture. Discretizing the differential domain and then using the back-propagation learning algorithm lead to solveing the optimization problem for the unknown series of coefficients. The validity of our method was based on the supposition that the convergence rate quickly rises by increasing the number of node points and learning steps. However, the initial values for the network parameters had a considerable impact. The learning rate and momentum constant were sensitive tools that were considered as convergence speed control parameters. Inappropriate choices for these parameters led to a lack of convergence or an excessive increase in the number of repeating steps. To make a better description of the offered technique, one numerical example was presented with computer simulations. Also, comparison of numerical results with exact solutions and those of another classical method has helped us to precisely understand this exercise. The achieved results support our claim that the designed neural architecture gives better convergent approximation without any restrictive assumptions. However, most of the equations belong to mathematical applications in real-world problems; therefore they require complex solutions. According to the numerical results obtained from two examples, it was natural to claim that the proposed procedures were valid and possessed unique properties along with high efficiency. The proposed method was more efficient than other methods. With a little care in the performance of this method, it can be easily concluded that our combination technique can be classified in row of non-algorithmic methods. Despite having control levers such as learning rate, momentum constant or variety of learning algorithms and cost functions increase the accuracy in determining the mathematical mystery. It is obvious that most classical methods are not available to solve a variety of complex mathematical problems. In many cases, these non-algorithmic methods can solve difficult mathematical problems. Research in this area can provide great benefits to the related fields, while extension of the fractional partial differential equations can be a milestone for the near future research.
Al-Mazmumy M., Al-Malki H., Some modifications of Adimian decomposition method for ninlinear partial differential equations, IJRRAS, 2015, 23(2), 164-173. Google Scholar
Boukehila A., Benmostefa F.Z., Solution process of a class of differential equation using Homotopy analysis wiener hermite expansion and perturbation technique, Int. Journal of Math. Analysis, 2014, 8(4), 167-186. CrossrefGoogle Scholar
Cakir M., Arslan D., The Adomian decomposition method and the differential transform method for numerical solution of multi-pantograph delay differential equations, Applied Maths., 2015, 6(3), 1332-1343. CrossrefGoogle Scholar
Desail Kh.R., Pradhan V.H., Solution by Homotopy perturbation method of linear and nonlinear diffusion equation, Inte. Journal of Emerging Tech., 2013, 3(4), 169-175. Google Scholar
El-Tawil M.A., Fareed A., Solution of stochastic cubic and quintic nonlinear diffusion equation using WHEP, pickard and HPM Methods, Open Journal of Disc. Math., 2011, 1(1), 6-21. CrossrefGoogle Scholar
Fuller R., Neural fuzzy systems, Abo Akademi University press, 2005. Google Scholar
Graupe D., Principles of artificial neural networks (2nd Edition), World Scientific Publishing, 2007. Google Scholar
Hanss M., Applied Fuzzy Arithmetic: An introduction with engineering applications, Springer-Verlag, Berlin, 2005. Google Scholar
Hassoun M.H., Fundamentals of artificial neural networks, MIT Press, New york, 1995. Google Scholar
Hendi F.A., Bakodah H.O., Al-Mazmumy M., Alzumi H., A simple program for solving nonlinear initial value problem using Adomian decomposition method, Int. J. of Rese. and Review in Appl. Scien., 2012, 12(3), 397-406. Google Scholar
Jafarian A., Alizadeh R., A new combinative method to the two-dimensional Bratu problem, Adva. in Difference Equ., In press. Google Scholar
Jafarian A., Kulaii L., A new combinative method to the onedimenional fractional Bratu problem, Int. J. of Dynamical Sys. and Differe. Equ., In press. Google Scholar
Jafarian A., Mokhtarpour M., Baleanu D., Artificial neural network approach for a class of fractional ordinary differential equation, Neu. Com. and Appli., 2016, 3(2), 271-9. Google Scholar
Jianu M., Solution of heat and wave-like equations by Adomian decomposition sumudu transform method, Rom. J. of Building, 2015, 2(1), 1-8. Google Scholar
Jianu M., Popescu I., Heat and wave-like equations with variable coefficients solved by Taylor series method, Rom. J. of Building, 2015, 2(1), 1-8. Google Scholar
Jui-Sheng Ch., Chuen-Fa N., Ching-Pin L., Analytical power series solutions to the two dimensional advection-dispersion equation with distance-dependent dispersivities, Hyd. Pro., 2008, 22(4), 4670-4678. CrossrefGoogle Scholar
Kurulay M., Bayram M., A novel power series method for solving second order partial differential equations, Eur. J. of Pure and Appl. Math., 2009, 2(2), 268-277. Google Scholar
Marin M., On existence and uniqueness in thermoelasticity of micropolar bodies, C. R. Math. Acad. Sci., 1995, 321(12), 475-480. Google Scholar
Matinfar M., Saeidy M., Raeisi Z. , Modified variational iteration method for heat equation using He’s polynomials, Bull. Math. Anal. Appl.,2011, 3(2), 238-245. Google Scholar
Mohyud-Din S.T., Noor M.A., Noor KI., Variational iteration method for Burgers and coupled Burgers equations using He’s polynomials, Zeitschrift Fur Naturforschunge Section A-A J. of Phy. Sci., 2010, 65(4), 263-267. Google Scholar
Nuseir A., Ameina S., Al-Hasson A., Power series solution for nonlinear system of partial differential equations, Appl. Math. Sci., 2012, 6(104), 5147-5159. Google Scholar
Secer A., Approximate analytic solution of fractional heat-like and wave-like equations with variable coefficients using the differential transforms method, Secer Adva. in Diff. Equa., 2012, 4(3), 187-198. Google Scholar
Tabatabaei Kh., Eluik E.C., Tabatabaei R., The differential transform method for solving heat-like and wave-like equations with variable coefficients, Turk J. Phys., 2012, 36(9), 87-98. Google Scholar
Wazwaz A.M., The variational iteration method for solving linear and nonlinear ODEs and scientific models with variable coefficients, Cent. Eur. J. Eng. 2014, 4(1), 64-71. Google Scholar
About the article
Published Online: 2017-12-29
Citation Information: Open Physics, Volume 15, Issue 1, Pages 1086–1094, ISSN (Online) 2391-5471, DOI: https://doi.org/10.1515/phys-2017-0135.
© 2017 A. Jafarian and D. Baleanu. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. BY-NC-ND 4.0