Multi-Agent System Based on the Extreme Learning Machine and Fuzzy Control for Intelligent Energy Management in Microgrid

: Renewable energies constitute an alternative to fossil energies for several reasons. The microgrid can be assumed as the ideal way to integrate a renewable energy source in the production of electricity and give the consumer the opportunity to participate in the electricity market not just like a consumer but also like a producer. In this paper, we present a multi-agent system based on wind and photovoltaic power prediction using the extreme learning machine algorithm. This algorithm was tested on real weather data taken from the region of Tetouan City in Morocco. The process aimed to implement a microgrid located in Tetouan City and composed of different generation units (solar and wind energies were combined together to increase the efficiency of the system) and storage units (batteries were used to ensure the availability of power on demand as much as possible). In the proposed architecture, the microgrid can exchange electricity with the main grid; therefore, it can buy or sell electricity. Thus, the goal of our multi-agent system is to control the amount of power delivered or taken from the main grid in order to reduce the cost and maximize the benefit. To address uncertainties in the system, we use fuzzy logic control to manage the flow of energy, to ensure the availability of power on demand, and to make a reasonable decision about storing or selling electricity.


Introduction
Since the 1970s, energy production risks were demonstrated to rest on the fossil resource exploitation following successive crises on the production of oil [8,45], whose reserves are badly distributed and exhaustible. Atmospheric pollution, climate warming, nuclear power risks, and resource limitations push awareness of the need for economic development that is respectful of the environment in which we live.
Renewable energies constitute an alternative to fossil energies for several reasons: they generally disturb the environment to a lesser extent, do not emit gas with greenhouse effect, do not produce waste, are inexhaustible, and enable a decentralized production adapted at the same time to resources and the local needs [14]. For these reasons, many countries tend to adapt the new concept of microgrids (MGs). The term MG refers to the concept of single electrical power subsystems associated with a small number of distributed energy resources, both renewable and/or conventional sources, including photovoltaic (PV) power, wind power, internal combustion engine, gas turbine, and micro-turbine together with a cluster of loads [26,27]. MG combines multiple local and diffuse production facilities, consumer facilities, storage facilities, and monitoring and demand management tools. It can be connected directly to the distribution network (DN) or operate in an isolated mode. This type of system is affected by changes in site climatic data, that is why the different sources must be controlled by an intelligent energy management system (EMS). One of its main objectives is to achieve a high level of flexibility, not only during operation but also during outages and throughout its life cycle; that is, the system must be able to adapt to most changes. Classical control methods are not efficient for the management and control of the different operations for such systems. For this reason, in the past few years, the paradigm of multi-agent system (MAS) has been invented and used in power systems in order to ensure more powerful results, thanks to the advantages that characterize this approach.
There are many proposed works based on the concept of MAS for the control of MG. Previous studies [12,15,34] presented the innovation of different strategies for buying and selling electricity depending on the price. Le and Ilic [29] proposed the predictive dispatch model to minimize the total production cost. Logenthiran and others [31,32,47] were interested in the efficient method for energy resource scheduling of integrated MG with various distributed generators and storage devices. Arai et al. [1] used MAS for power generation control or to restore the power system after a fault. Elamine and Boumhidi [9] focused on finding the optimal coalitions between the different MGs by proposing a MAS based on the law of gravity and fuzzy logic for a multi-MG environment. In Ref. [4], MAS was used in EMS in order to fulfill the load demand of the network and adjust power along with its surplus and shortage information. Some papers also addressed the reliability of MG by distributing power control for distributed energy resources, based on MAS, with communication delay technologies having effective skills for distributed demand management [24,28]. Elamine et al. [10] proposed a MAS based on wind power prediction using artificial neural network (ANN) trained by the hybrid of particle swarm optimization algorithm and back-propagation (BP) algorithm. Khatibzadeh et al. [25] developed a hierarchical MAS (HMAS) and a distributed algorithm for HMAS to increase the speed of decision making in a hybrid MG.
In the MG, the main energy sources are the renewable energy sources like wind and solar energy. The integration of the forecast of weather data and output power is necessary, and there are many proposed works for predicting wind and solar power. In general, there are two approaches to predict power. The first approach is based on the direct prediction of energy, and this was chosen for this work; the second approach relies on the estimation of energy using predicted weather data and some power curves. Some works used time series to model the characteristics and behavior of wind speed as the autoregressive moving average [20,44]. Vico et al. [46] applied deep neural networks into two problems: wind energy and daily solar radiation prediction. Gensler et al. [13] used combinations of different deep learning and ANN algorithms, such as deep belief networks, AutoEncoder, and long short-term memory, to show their forecast strength compared to a standard multilayer perceptron and a physical forecasting model.
Most related works that used the ANN algorithm for prediction have some problems, such as the slow convergence time, the weakness in finding global optimistic results, and the difficulty of calculating the quadratic convex program modeling the real problem. Besides, these techniques are unable to cope with the strong fluctuation of observations. Taking into account these problems mentioned above, we opted to use the extreme learning machine (ELM) algorithm for the prediction of power produced by renewable sources, because it overcomes the shortcomings of several other machine learning techniques. This has been verified in the latest works [18,30,43,48,49]. It has shown good generalization ability on benchmark tasks, both in classification and in function regression [19]. Moreover, it requires less human interference and can run faster than the conventional methods, due to the random choice of hidden nodes and the analytical determination of the output weights.
ELM was used in several applications. In Ref. [21], an approach of ensemble ELM for data classification based on the dropout technique, to address the problem of architecture selection and improve the prediction stability, was proposed. In Ref. [22], the authors focused on the framework of two-class imbalanced problems and proposed a novel algorithm that consists of four stages based on MapReduce and an ensemble of ELM classifiers. In Ref. [23], based on a voting mechanism, the authors proposed a large data set instance selection algorithm with MapReduce and random weight networks. In Ref. [2], an enhanced optimization-based ELM was proposed in its primal form, whose solution is obtained by solving an absolute value equation using a simple functional iterative method. In Ref. [40], the authors proposed two adaptive neural control schemes for a class of affine multi-input multi-output non-linear systems with the ELM algorithm, which is utilized to determine the parameters of hidden nodes in single hidden layer feedforward neural networks (SLFNs) by assigning random values to them. In Ref. [36], the learning ability of the batch version of online sequential-fuzzy-ELM, called fuzzy-ELM, was further evaluated to train a class of fuzzy inference systems that cannot be represented by the radial basis function networks. In Refs. [35,37], the authors were interested in direct and indirect adaptive neural controller for a class of non-linear systems when the neural controller was constructed based on SLFN while its hidden node parameters were modified using ELM.
In this paper, we propose a MAS scheduling energy system based on the ELM algorithm for the prediction of power produced by renewable sources for the next hour and the fuzzy logic control (FLC) while respecting MG constraints. In our study, the system attempts to supply its costumers locally. In case the total power generated by the MG is not enough to cover the supplying load, the energy is bought from the DN.
The remainder of this paper is organized as follows. In Section 2, we provide a brief glance at the used methodologies. In Section 3, we present the adopted MG and its components. Then, a detailed multi-agent architecture that involves ELM and fuzzy logic unit control is proposed in Section 4. In Section 5, we present the simulation and discuss the results. We conclude the paper in Section 6.

Used Methodologies
As described before, a consistent combination of MAS based on prediction and reliable decision will allow an improved energy scheduling through the MG. In the following section, we give a brief presentation of the MAS technology, ELM algorithm, and FLC.

MAS
MAS is a distributed system consisting of a set of agents. Unlike artificial intelligence systems that simulate human reasoning capabilities, MAS is ideally designed and implemented as a set of interacting agents, most often according to modes of cooperation, competition, or coexistence [6].
The main advantages of MAS are robustness, velocity, and scalability [5]. Robustness refers to the ability of the system to tolerate failures of one or more agents, if control and responsibilities are sufficiently shared among agents within a MAS. Velocity is mainly due to the parallelism because many agents can work together to solve a problem. The scalability of MAS originates from its modularity, which makes programming easier. It should be easier to add new agents to MAS than to add new capabilities to a monolithic system.

ELM
Huang et al. [17] introduced a new approach, so called ELM, which overcomes the shortcomings of several other machine learning techniques. Besides its flexibility, the approach has also shown good generalization ability on benchmark tasks, both in classification and in function regression [19]. It requires less human interference and can run thousands of times faster than the conventional methods.
Usually, the parameters of the applied machine learning algorithms are selected depending on the input and target example data. Determining the parameters of the algorithms is often performed iteratively, by using genetic algorithms or by hyperparameter optimization [3]. Contrary to the mentioned approaches, which require either a manual or a computationally very expensive parameter setting, all of the parameters of the ELM hidden nodes are not dependent on the target function or the training dataset [19]. Hidden nodes are chosen randomly and the output weights are determined analytically by determining the optimal combination of the output signals of the hidden layer [19]. Therefore, there is no need to set the parameters manually and to fine tune them iteratively. Therefore, the parameters in the hidden layer can be independent of the training samples [16].

Technical Background of ELM
The theoretical concepts and the computational algorithms were derived from Ref. [5]. Suppose that there is a training set Figure 1: Illustration of the ELM Structure [42].
hidden neurons. In SLFN, the input layer and the hidden layer are interconnected by an l × n input weight matrix ϖ, where the ith row vector is ϖ j = [ϖ j1 ; ϖ j2 ; . . . ; ϖ jn ] T , b is an l × 1 hidden bias vector for each hidden neuron, and the output layer and the hidden layer are interconnected by an l × m output weight matrix β, where the ith row vector is The structure of ELM consists of an input layer, an output layer, and a single hidden layer. It is shown in Figure 1.
More specifically, the ELM network structure can be defined as follows: where i ∈ 1; 2; . . .; N, j ∈ 1; 2; . . .; l, ϖ j · x i indicates the inner product of ϖ j and x i , and g(x) is an activation function. According to the universal approximation capability theorems of ELM [19], various activation functions can be used (step function, sigmoid function, etc.). Equation (1) can be written as the following compacted form: where H is called the hidden layer output matrix of the network, T is the output matrix, and T = [t 1 ; t 2 ; . . .; The output weights are calculated analytically as where H + is the MoorePenrose generalized inverse of matrix H [38].

Fuzzy Logic
Fuzzy logic arose from a desire to incorporate logical reasoning and the intuitive decision making of an expert operator into an automated system [39]. The tools provided by fuzzy logic allow a modeling of the phenomena Rules Fuzzifier Defuzzifier Inference system that can, in a certain sense, approach human reasoning. It provides solutions to complicated or difficultto-model problems, and it does not have general methods of synthesis; thus, it is necessary to vary at the entrance and do tests until the set point. The aim is to make a decision based on a number of learned or predefined rules, rather than numerical calculations. However, before the rule base can be used, the input data should be represented in such a way as to retain meaning while still allowing for manipulation. The fuzzy rule representation is based on linguistics [39]. In FLC, the term "linguistic variable" refers to whatever state variables the system designer is interested in [39]. The fuzzy variable is perhaps better described as a fuzzy linguistic qualifier. Once the linguistic and fuzzy variables have been specified, the complete inference system can be defined as developing a fuzzy inference system, and applying it to a control problem involves several steps (see Figure 2). A. Fuzzification: Fuzzification is the action to process the input variables with membership functions (MFs) and determine the degree to which the input variables belong to each of the appropriate fuzzy sets via MFs. MFs are usually decided based upon the user's requirement, although there is no restriction on the number of MFs. However, the setting of the interval and number of MFs are very important, and many evolving fuzzy logic systems have been proposed in the discussion of parameters [41,42]. B. Inference system: In the fuzzy inference block, the fuzzified input variables are processed with fuzzy operators and the IF-THEN rule implementation. The output fuzzy sets for each rule are then aggregated into a single output fuzzy set.
C. Defuzzification: Defuzzification is a process that converts a fuzzy set or fuzzy number into a crisp value. There are different methods of defuzzification, the most common are centroid and mean of maxima.

Adopted MG
MG consists, in general, of distributed generators (e.g. diesel generators, fuel cells), renewable generators (e.g. wind turbines and PV panels), energy storage, and local demands. Figure 3 presents the adopted MG located in Tetouan, a city in northern Morocco. It is composed of eight PV panels with 36 cells each. Taking into account the unpredictable weather conditions and to assume a continuous availability of energy, the PV panels are supplied with one wind turbine. The system also includes a number of batteries to either store the excess of energy generated by the sources or provide the energy demanded by the load when there is low renewable energy produced. We note that the MG is on connected mode.
In this paper, we classify the sources of the MG according to the following priority order: -First: renewable energies as primary source; -Second: batteries as auxiliary source; -Third: DN as the last source.

Wind Turbine
The wind turbine produces electricity from wind. It is considered a primary source in our MG. The output power generated by the wind turbine can be considered as a function of the wind velocity [7] and can be calculated as follows: where P wr , V ci , and V co are the rated power, cut-in wind speed, and cut-out wind speed, respectively. Furthermore, V r and V ac are the rated and actual wind speed, respectively. Constants a, b, and c depend on the type of the wind turbine. We assume AIR403 wind turbine model is used in this paper (AIR 403, Southwest Windpower, Flagstaff, AZ, USA). According to the data from the manufacturer, their characteristics are as follows [11]:

PV Module
The PV modules ensure the production of electrical energy during daylight hours. It is considered as the main source of energy. The output power of the module can be calculated using the following equation [11]: where P PV is the output power of the module at irradiance G ING , P STC is the module maximum power at standard test condition (STC), G INC is the incident irradiance, G STC is the irradiance at STC (1000 W/m 2 ), k is the temperature coefficient of power, T c is the cell temperature, and T r is the reference temperature. We assume that Solarex, MSX-83 is used in this paper (Solarex, Frederick, MD, USA). Their output characteristics are as follows: peak power = 83 W, voltage at peak power = 17.1 V, current at peak power = 4.84 A, short circuit current = 5.27 A, and open circuit voltage = 21.2 V at STC.

Energy Storage
Batteries are used to store excess energy, after filling the energy requirement, for subsequent use in the event that the energy produced by the PV panels and wind field is insufficient. The state of charge (SOC) of a battery is defined as its available capacity expressed as a percentage. It can be calculated using the following equation: where C is the capacity of battery and C Ref is the reference capacity of battery.

Proposed EMS
The energy management in flexible structures such as MG is a complex problem. This complexity comes from the uncertainty of the environment and its variations. Energy management should consider these constraints and attempt several objectives at the same time. In this paper, we present an energy management supervision for the MG. Its goal is to control the amount of power delivered or taken from the DN in order to reduce the cost and maximize the benefit. Therefore, the main goals of this paper are to -Maximize the benefits by selling electricity to the DN, as soon as possible, after filling the local demand of MG and charging the batteries; -Minimize the costs by purchasing electricity from the DN only if the load demand is greater than the produced energy and batteries are discharged.
To achieve these goals, we divide the energy management strategy into two main phases: prediction phase and decision making phase (see Figure 4).

A. Prediction phase:
Given the advantages of learning speed and selection parameters of ELM quoted above, we choose the ELM algorithm to predict the amount of PV and wind energy produced for the next hour.
B. Decision making phase: The decision making with FLC is adopted to determine which auxiliary energy source (batteries or DN) will fill the needed energy, and to make a reasonable decision about storing or selling electricity.

MAS Architecture of the Adopted MG
The renewable energy system is affected by the site climatic data change, that is why the different sources must be controlled by an intelligent EMS. One of the main objectives of the EMS is to achieve a high level of flexibility. Thus, we are obliged to call upon one artificial intelligence approach, which is MAS, in order to ensure more powerful results, thanks to the advantages that characterize this approach. This concept allows the possibility of the different generation units of smart MG to collaborate in order to achieve the optimal strategy to deal with the problem of economical exchange with the DN. In this work, six agents are established in the entire system, as mentioned in Figure 5: -AgentPV: This agent estimates the PV power produced by solar panels at time t. It uses the ELM algorithm to predict the PV power for the next hour. -AgentWindTurbine: It predicts the wind power produced at time t by the wind turbine. It uses the ELM algorithm to predict the wind power for the next hour. -AgentBatteryBank: It is the agent who is responsible for the storage of energy. Among its roles, it gives information on the state of storage and provides power to the MG when a trigger request comes from the AgentController.   Table 1.

Adopted Prediction Method
A neural network trained by ELM is proposed to predict the PV and the wind power for the next hour. We use two predictors. The first takes as input the irradiation and the temperature and gives as result the amount of PV energy produced (P PV ). The second predictor takes as input the wind speed and gives as result the amount of wind energy produced (P W ). Then, we sum P PV and P W to get the total energy produced (P T ) by renewable

Communication number Description of communication
1 "AgentLoad" informs "AgentController" on charge 2 "AgentController" demand energy information produced by PV and wind power generators and energy stored in batteries (P PV , P W and SOC) 3 The three agents, "AgentPV", "AgentWindTurbine" and "AgentBatteryBank" respond to "AgentController" 4 "AgentController" makes decision using FLC and send messages to the "AgentLoad" and "AgentBatteryBank" to update its SOC R Representing the agent's reasoning energy sources in the MG at each period of time t.
The system power delivered by the renewable sources of energy P T was determined. We can now calculate ∆P, the difference between P T and P L (load demand).
where ∆P can be positive (when the renewable energy produced is bigger than the load) or negative (when the load is bigger than the renewable energy produced). If ∆P is negative, we must complete the need from batteries or DN when batteries are discharged. To make this last decision, we use FLC. To sum up, the energy management strategy is presented by the flowchart in Figure 7, which describes the different stages of treating the problem.

Start
Predict the amount of photovoltaic and wind energy produced P PV , P W Deduce the total energy produced P

Fuzzy Logic Controller Structure
An energy management controller is proposed in this paper based on FLC to make a decision about selling/ purchasing electricity from DN and about charging/discharging batteries. It is necessary to properly manage the flow of energy, in order to satisfy the demanded load while trying to keep the battery SOC level between certain target margins (20% and 80%), and to control the amount of power delivered or taken from DN in order to reduce the electricity bill and make profit by selling the surplus in the energy market.
The adopted fuzzy controller takes into account the battery SOC and the difference between the power generated by renewable resources and the load demanded: ∆P. Thus, the inputs are ∆P and the battery SOC. The output is the decision that we should make among four choices: take the needed power from DN, discharge batteries, charge batteries, and inject excess power in DN.
MFs in Figure 8 are chosen as below and were designed taking into account the previous knowledge of the MG and load power demand.
1. Four functions to represent ∆P: "LargeNegative"; "SmallNegative"; "SmallPositive"; "LargePositive." 2. Five functions to represent the variable SOC: -"Low" from 0% to 30%; -"SmallMedium" from 25% to 35%; -"Medium" from 30% to 70%; -"LargeMedium" from 65% to 75%; -"High" from 75% to 100%. 3. Four functions to represent the output decision: take from main grid, "TMG"; discharge batteries, "DB"; charge batteries, "ChB"; and inject in main grid, "IMG." Linguistic control rules derived from the flowchart of Figure 7, respecting the MG constraint and the ranking according to the priority of resources, were written based on the inputs ∆P and SOC, and summarized in Table 2. The number of rules is 20 because we have four functions to represent ∆P and five to represent the SOC. Decisions for each combination has been chosen to allow good use of batteries by ensuring that they will not be discharged to <20% or charged to >80%.

Interpretation of Rules
If ∆P is LargeNegative, we need an important amount of energy to fill the energy needed. Thus, we discharge batteries when SOC is Medium, LargeMedium, or High. However, in case where SOC is Low, we fill the need by purchasing electricity. Even when SOC is SmallMedium (i.e. SOC is between 25% and 35%), we must choose the purchase to avoid deep discharge and damage of batteries. If ∆P is SmallNegative, we need a small amount of energy to fill the energy requirement. Thus, we discharge batteries whatever the value of SOC, except when it is Low, because in this case we must purchase the quantity needed to avoid battery discharge to <20%.
If ∆P is SmallPositive, we have a small amount of excess power. Consequently, we must charge the batteries without any constraint that they can be charged to >80% in the case where SOC is Low, SmallMedium, Medium, or LargeMedium. Nevertheless, we have to sell the excess power when SOC is High to avoid an entire charging of batteries and obtain important benefits.
If ∆P is LargePositive, we have an important amount of excess power. Therefore, we must charge the batteries when SOC is Low, SmallMedium, or Medium. However, when SOC is LargeMedium (e.g SOC is between 65% and 75%), we prefer to sell excess power because if we charge batteries, we can go beyond the limit (>80%). Finally, if SOC is High, the decision is to inject excess power in DN.

Simulation and Results
In this section, we present the results and discussion of the numerical analyses. First, we present the numerical results, and later we discuss the performance of the EMS proposed.
MAS can be implemented based on a number of open-source agent platforms that can aid developers in building a complex agent system in a simplified fashion [33]. Our MAS was developed in Java Agent Development Framework, an open-source FIPA (Foundation for Intelligent Physical Agents of the Institute of Electrical and Electronics Engineers)-compliant platform. This MAS is employed to perform some simulations.
The ELM applied in this case study is a feedforward network with one neuron in the input layer for the wind power (two neurons for the solar power) and one neuron in the output layer. It was tested by using different numbers of hidden layer neurons. Then, we chose the number that gives results closest to those calculated. Finally, we opted for five neurons for the PV network and four neurons for the wind network. The algorithm was trained and simulated using Java.
To learn and test the ELM algorithm, we used real observations of irradiation, temperature, and wind speed of Tetouan City. To test the robustness of the algorithm, the following profiles are chosen: the power of load (Figure 9), solar irradiance ( Figure 10), temperature (Figure 11), and wind speed ( Figure 12) of two different days (1 day of summer and 1 day of winter). Figures 13 and 14 present curves, in both cases PV power and wind power, comparing the results predicted using the ELM algorithm and the results obtained using mathematical formulas [Eqs. (5) and (6) cited in Section 3].
To show the potential of the ELM prediction model, a performance comparison in terms of the estimation capability and training time was made between ELM and the conventional feedforward ANN model run   with the BP algorithm. The estimation capability was statistically evaluated using the root mean square error (RMSE) as a numerical tool for comparing the measurement and predicted values by ANN. In this case, RMSE could be represented as where y i is a predicted value, x i is a measured value, and N is equal to the number of observations. From Table 3, it may be inferred that the performance of ELM is substantially promising in terms of response time as compared to the feedforward BP neural network. The training time, which is still an issue in a feedforward BP network, is simply bypassed in ELM. The comparison results also showed that the ELM model gave better estimation than the ANN model for the overall test. ELM was expeditiously efficient and yielded lower prediction errors than the ANN model. In fact, the best RMSE value was obtained with ELM.
On the other hand, using FLC for decision making gives more capability to control the different operations as storage or selling, in order to have the maximum benefit and the minimum cost. A simulation over a period of 100 days was performed to compare the curve of benefit and the curve of cost. When we had a benefit, this appears in red; otherwise, the cost appears in blue ( Figure 15). In the case, where the two curves are at zero, this means that we had discharged or charged the batteries and we did not need the DN. It is clear that we had managed to achieve more benefits than costs according to the test period.  In order to ensure a long lifetime for the battery, its SOC should be, as much as possible, between 20% and 80%. The FLC adopted allowed obtaining this level of guard. Figure 16 presents the SOC of batteries during the test period. As we can notice, the variation of SOC is always between 20% and 80%. From these results, it can be said that the objective of the FLC-based energy management method of the proposed MG is achieved.

Conclusion
The MG presented in this paper depended on two important and very useful renewable energies combined together to increase the efficiency of the system. Using batteries as a storage unit is suggested for this system. A MAS was proposed as EMS. Each component of MG is represented by an autonomous agent able to interact with other agents according to its current state and specific goals.
The power generated from renewable sources was predicted using the ELM algorithm to overcome problems related to complex mathematical modeling. After consumption of renewable energy, the FLC technique was used to make a decision among four choices in order to satisfy the economic goals and reduce the balance between cost and benefit.
After the test and simulation, the obtained results were very satisfactory. First, they demonstrated that the proposed MAS is able to control different energy flows in MG. Second, the results of the case study confirmed that ELM can be applied in a very flexible way and shows several advantages. It is a powerful, very fast, and efficient learning algorithm, especially compared to approaches with gradient-based learning algorithms. Third, using FLC for decision making gave more capability to control the different operations in order to have the maximum benefit and minimum cost. The maximal discharge rate and maximal SOC of the batteries were taken into account to ensure their proper use.