Rainfall prediction system for Bangladesh using long short-term memory

1 Introduction

Bangladesh is primarily an agricultural nation, and the sector’s contribution to rapid economic growth is crucial. In order to guarantee long-term food security for humans, it is crucial to create a lucrative, sustainable, and environmentally friendly agricultural system [1]. As a result, accurate rainfall forecasting is an essential tool for the economy’s improved development. Rainfall predictions are used in weather forecasting, date fixing of crop plantation, flood prediction, and many other sectors [2]. So undoubtedly, rainfall prediction holds a huge significance and importance.

In the past, monthly rainfall predictions were done using linear regression models [3,4,5]. Later application of random forest regression models made predictions more accurate [6,7]. Different popular approaches including Naive Bayes (NB) [8], decision tree [9], random forest and some other classification techniques [10,11,12] have also been used for the same purpose. Other attempts and research works have utilized various ways ranging from pure probabilistic mathematical approach to advanced techniques of machine learning. However, using raw probabilistic formulas cannot yield high enough accuracy due to the complex nature of the datasets [13]. Therefore, dealing with optimized and standardized data along with implementing advanced neural networks will obviously yield better accuracy in predictions.

Predicting rainfall is a time-series problem which can be solved with great efficiency using long short-term memory (LSTM) networks [14]. LSTM is a special branch of recurrent neural networks (RNNs). Previous predictions in this field have not been involved heavily with LSTM network [15]. As LSTM network implementation is relatively advanced technique, it is an optimal choice for quality prediction. Machine learning predictions require huge amount of data for high-end predictions, while it must also be ensured that a significant number of features are involved in the prediction algorithm. If the size of the dataset is not enough, then the quality of prediction does not reach maximum threshold [16].

One of the major challenges in processing real-life data is its high number of dimensions. In order to cut down dimensions of data and thereby to avoid overfitting [17], we have used principal component analysis (PCA), in which unnecessary dimensions (or features) are trimmed off and projected onto other vital dimensions. Thus, we have used optimized data as parameters of prediction in our LSTM network [18]. After feature selection we have applied cross validation technique. In the process of training and testing, the optimized data are then feed forwarded through the input layers to fully connected layers of deep LSTM–LSTM network. Finally, after applying the sigmoid function, we derived the predicted quantity of rainfall followed by measuring the accuracy of the test dataset predictions.

The main contributions of this work are as follows:

1. To the best of our knowledge, this is one of the pioneer works on Bangladeshi rainfall prediction system.

2. LSTM–LSTM network was built for fast and highly accurate classification.

3. The result has been compared with different machine learning approaches.

4. Instead of taking all the features, the most useful ten features had been chosen for enhancing execution.

2 Literature review

Various data mining techniques have been used for rainfall prediction. Numerous research papers are found in the literature.

In ref. [19], support vector machine (SVM), artificial neural networks (ANN), and Adaptive Neuro Fuzzy Inference System (ANFIS) have been used. ANFIS avoided noise in data by using varying lags of input and SVM dealt with out-of-pattern instances in the dataset. In another research [20], the team in Malaysia worked with random forest, SVM, NB, ANN, and decision tree. They obtained data from various weather stations of Malaysia (Selangor). The noise and missing values in that dataset were fixed using ANFIS and other pre-processing techniques. Random forest algorithm was used to utilize small training dataset against large amount of test data. The study of Chatterjee et al. [21] is based on a survey which examined the Neural Network architectures used for rainfall prediction over the last 25 years. They implemented SVM, multilayer perceptron (MLP), BPN, RBFN, and SOM for forecasting. In the Lahore rainfall dataset [22], a combination of SVM, NB, k-Nearest Neighbor (kNN), decision tree (J48) and MLP has been used to calculate F 1-scores. F -1 values represent correlation and balance between precision and recall. Higher F -1 values indicate that precision and recall are less complementary. Research team in ref. [23] implemented Back Propagation, Radial Basis Function, and ANN and obtained quality prediction on monthly rainfall. The dataset was from Coonoor region in Nilgiris district (Tamil Nadu). Mean square error was taken as the performance measure. They obtained smaller mean square error using Radial Basis Function Neural Network. In ref. [24], usage of linear regression method was notable. This article also involved a new feature named “rainfall pace” and correlated it with various seasonal crops such as rabi, Kharif, zaid and then predicted future rainfall. The dataset was sorted and grouped according to crop name, season, and year.

In ref. [25], a combination of classical ANN and Genetic Algorithm has produced a hybrid system, which is intelligent enough to make plausible predictions. The purpose of using the Genetic Algorithm was to better organize the input layer, the connection and communication among input nodes, the output layers, and to train the network more efficiently. In ref. [26], using only ANNs, 1-month and 2-month forecasting models have been built that predicted rainfall. Multiple stations in North India have contributions to the dataset and the data records were on past 141 years. Along with both Feed Forward and Back Propagation Neural Network, Levenberg–Marquardt training function has been used. Performance evaluation has been done by mean square error, regression analysis, and magnitude of relative error. Surprisingly, the results proved that the forecasting model for 1 month can predict more accurately the rainfall than that of 2 month.

In ref. [27], authors worked with a time series dataset containing data on Monsoon Rainfall of Summer in India (based on monthly and seasonal time periods of 1871 till 2014). Due to the unpredictable and dynamic nature of monsoon rainfall, entropy theory, fuzzy set, and ANN were used. The fuzzy set theory handled uncertainties and made transition from one inference to another smooth using degrees. In the modified entropy computational concept, the input was treated as a degree of membership into the entropy function (also called Fuzzy Information-Gain [FIG]). The ANN then defuzzified every fuzzified rule. Finally, the output of FIG of every fuzzy-set was fed into the ANN. Evaluation of performance was done through root mean square error (RMSE), standard deviations (SDs), correlation coefficient (CC), and performance parameter (PP). This integrated combination worked better than many other models.

Table 1 demonstrates a comparative analysis showing the merits and demerits as long as methods, datasets, and materials that the literature have deployed for rainfall or weather forecasting.

From Table 1, it is seen that most of the works are for different specific regions having some major drawbacks such as very small data set, small feature set, and less accuracy. However, in this work, two-layer LSTM method has been developed [28,29,30] for predicting rainfall in Bangladesh. It solves the backflow problem [31,32] found in other works. A larger dataset containing 20 features (most prominent 10 features) is used. Proposed work can predict rainfall in advance for any season and for any region of Bangladesh.

3 Proposed structure

In this work (Algorithm 1), we have made rainfall prediction by the LSTM model. Rainfall data has been collected and preprocessed. Also, we have normalized the dataset for making it perfect for applying in LSTM. After normalizing the dataset, a feature selection technique, PCA is applied. Initially, the dataset contained totally 20 features. As all the features may not be suitable for every purposes, feature selection method was applied on the dataset. However, ten most effective features are selected. LSTM is used as a deeper learning-based network. For training the LSTM model, 10-fold cross validation method was applied for splitting the training and testing dataset properly. Two levels of LSTM were applied before the fully connected layer. As output, the model predicts amount of rainfall in millimeters. Flow diagram of the proposed work is as depicted in Figure 1.

Figure 1

Flow diagram of the proposed system.

3.3 Working process of LSTM

Instead of randomly splitting the dataset into training and testing, 10-fold cross-validation technique has been used. Dividing the whole dataset into 10-folds, the model is fitted using the first fold for testing and others for training. After that, second fold is used for testing and remaining for training the model. This process will repeat until 10th iteration. In every iteration, it gives an error estimation. Then mean is applied to all error estimation.

LSTM has been used for a deeper learning-based classifier (Figure 2). Two LSTM layers are used for enhancing efficiency. Both LSTM layers have 50 hidden neurons units. In the first LSTM, dropout layer is applied. Dropout technique was used to prevent a model from overfitting. After that, this method approximates training a huge sum of networks with different architectures in parallel. In the training phase, some outputs are randomly “dropped out.” This has the impact of creating the layer look-like and be treated-like a layer with a special range of nodes and property to the previous layer. In this study, the dropout technique has been used for both LSTM layers. In the first LSTM, 50% dropout was used, while for the second LSTM layer it was 20%.

Figure 2

LSTM network.

In every LSTM layer, batch normalization has been used (Figure 3). Batch normalization is a technique for improving the speed, performance, and stability of deep learning-based architecture. Otherwise, there is probability of some numerical data point in the dataset such that one is very high while others might be very low. The data need to be on the same scale for reasonable comparison and analysis. The larger data point in this non-normalized dataset can cause instability in neural networks, which may cause an imbalanced gradient problem. To avoid this, batch normalization technique was used. Subsequently, a two-layer LSTM model had been developed. Hidden neuron units are set to be 50 for both LSTM layers and the “epoch” is set to be 30. Table 4 contains all the mathematical formula and measurements deployed in this work.

Figure 3

Structure of double LSTM layers.

Table 4

Deployed mathematical formula and measurements

Formula used in PCA	If we use PCA for dimensionality reduction, we construct a d ∗ k -dimensional transformation matrix W that allows us to map a sample vector x onto a new k -dimensional feature subspace that has fewer dimensions than the original d-dimensional feature space:
	x = [ x 1 , x 2 , … , x d ] , x ϵ R d , W ϵ R d ∗ k	(1)
	z = [ z 1 , z 2 , … , z k ] , z ϵ R k	(2)
Formula used in LSTM	LSTM network maps an input sequence x = ( x 1 , x 2 , … , x T ) to and output sequence y = ( y 1 , y 2 , y 3 , … , x T ) calculating network unit activations using the following equations iteratively from t = 1 to T as follows:
	i t = σ ( W i + U i h t − 1 + b i )	(3)
	z t = tanh ( W z x t + U z h t − 1 + b z )	(4)
	f t = σ ( W f x t + U f h t − 1 + b f )	(5)
	C t = i t ∗ z t + f t ∗ C t − 1	(6)
	o t = σ ( W o x t + U o h t − 1 + V o C t + b o )	(7)
	h t = 0 t ∗ tanh ( C t )	(8)
	Here, W t , W z , W f , W o , U i , U z , U f , U o are model parameters which are estimated during model training; σ and tanh are activation functions; b is bias
Performance measurements	To assess the performance of proposed methods against other classifiers the following measures have been adopted:
	Accuracy = TP + TN TP + FN + FP + TN ,	(9)
	where TP is the True Positive rate, TN is the True Negative rate, FP is the False Positive rate, FN is the False Negative rate

4 Results and discussion

5 Conclusion and future work

In this work, a two-layer LSTM model has been applied for the Bangladeshi rainfall prediction system. Weights of features are calculated with the PCA method. Furthermore, the most weighted ten features are used. The proposed method has been compared with the performance of the traditional machine learning classifiers, such as KNN, Logistic Regression, SVM, random forest, NB and neural network. The accuracy of the Rainfall dataset had been compared with different classifiers. The best results were achieved by neural network 76.9% and KNN 76.8%. Proposed method obtained 97.14% accuracy. The result obtained by the proposed method is higher than every classifier applied for this dataset. In future works, the proposed system will be examined by larger data sets. Meanwhile, higher result can be obtained by using the Bidirectional LSTM layer. Additionally, a web application will also be formulated.