Social Unrest Prediction Through Sentiment Analysis on Twitter Using Support Vector Machine: Experimental Study on Nigeria’s #EndSARS

1 Introduction

The impact of social unrest on the peaceful existence and development of a society cannot be overemphasized. Social unrest in Nigeria is a serious social problem that affects the socioeconomic stability of the country. Social unrest also affects the progress and advancement of a nation. In Nigeria, whenever social unrest happens, it not only affects the socioeconomic stability but also the political stability of the country. Social unrest can stymie the economic growth and development of a country, lowering the international confidence level as well as increasing insecurity and the level of uncertainty of foreign investors. Osaghae (2015) highlighted the possibility of the country disintegrating in 15 years citing the report published by the Central Intelligence Agency in 2005 due to civil unrest, strikes, and violence. The researcher noted that social unrest is a menace whose impacts and costs cannot be truly quantified as it transcends just material and financial costs to emotional trauma and possibly the loss of lives. Recent advances in technology and the consequent emergence of social media have undoubtedly offered humanity several innovative ways to communicate and share useful information regardless of location and other conventional constraints. Social media have been reported by several researchers to be one of the main tools leveraged in recent times to fuel social unrest globally. Chiamogu, Obikeze, Chiamogu, and Odikpo (2021) stated that there has been a massive growth in the rate of internet penetration as well as the production and consumption of social media content. They noted that this increase has greatly influenced participation in politics as well as policy formulation and governance in most countries of the world, especially in countries with emergent liberal democracies where populism has been misapplied. These countries, according to them, have duly minimized the conventional checks and balances existent in every democratic government, leading to the subversion of justice and unfortunate strangulation of the citizenry, who consequently turn to social media as a veritable platform for activism, justice-seeking awareness creation on socio-political issues, and to mobilize protests. Over the years, platforms such as Facebook, WhatsApp, Instagram, Twitter, and other social media platforms in Nigeria have radically altered how information is shared and disseminated. This development enables the seamless transmission of information about notable events, trends, and opinions, as well as promotes calls for accountability on government’s actions and misdeeds. Chiamogu, Obikeze, Chiamogu, and Odikpo (2021) stated that social movements such as “#BringBackOurGirls,” “#RevolutionNow,” and “#EndSARS” gained traction by uniting people across the country and in the diaspora around a common cause through social media. The emergence and subsequent adoption of social media over the years in Nigeria have resulted in enormous growth in the amount and speed at which information is created, transmitted, and consumed. While this trend has ushered in several opportunities for socioeconomic growth, social media has been a critical platform in recent times for disseminating hate speech, political activism, and the proliferation of fake, distorted, or exaggerated news, as well as inciting violence and civil unrest. Samantha and Olu (2010) reported that technology-savvy youths are using social media to raise funds in bitcoins, United States of America dollars, Nigerian nairas, and a variety of other currencies in support of the “#EndSARS” campaign, as well as mobilize support from celebrities, international media, and institutions both at home and abroad. Although the #EndSARS protest began peacefully, it was swiftly taken over by hoodlums and politically motivated groups, which led to quick escalation. This escalation further led to shooting of citizens, part of which is the infamous “Lekki massacre,” which the Nigerian government has denied its occurrence, despite international findings. While social media has become a veritable tool for demanding accountability in governance, exposing various ills and occurrences in society, and promoting products and services offered by businesses, it also has potential to cause social unrest and catastrophic breakdown of law and order as well. This has been a major source of concern in recent times. The masses can also easily be incited to act in civil disobedience over fake and unverified information disseminated by people with selfish interests. Audu (2018) noted that over the years, conflict along the ethnoreligious fault lines has been increasing at an alarming rate due to several factors, among which is the rapid rate at which unverified information is disseminated via social media. The author further noted that much more dangerous than the proliferation of fake news is the intentional exaggeration or distortion of true stories, as these types of issues are much more difficult to spot. To stem this rather ugly trend, which could lead to social unrest, several studies have been conducted on suitable means to detect and flag potentially chaotic social media posts and messages (Cadena et al., 2015; Compton et al., 2013; Ranganath et al., 2016). Recent advances in fields such as Artificial Intelligence, Data Science, Big Data, and Machine Learning have led to some solutions, allowing for more efficient social media data collation and analysis. Korolov et al. (2016b) noted that social media have grown to become a salient tool for protest. In recent times, one can hypothesize a relationship between the volumes of a particular type of social media communication. Korolov et al. (2016b) noted the presence of challenges in identifying relevant and genuine messages in a relatively large stream of messages. As such, one suitable method of solving this problem is the use of relevant data mining techniques to extract relevant data from Twitter and/or other social media platforms and employing Natural Language Processing (NLP) methods to analyze social media threads. In this work, we model public opinions on #EndSARS as a binary classification problem using data obtained from Twitter and annotated using standard procedure. The need to leverage recent advances in technology, especially in the fields of data science and machine learning, to develop a suitable model for analyzing social media data for the likelihood of social unrest is timely. We have developed a model for the analysis of tweets with the capability of classifying them based on the potential of causing civil unrest or otherwise. The implication of this is that it will enable interested people, organizations, and relevant authorities to develop early warning systems to inform them about potential social unrests. The rest of the paper is organized as follows: Section 2 discusses the literature. In Section 3, the methodology including data development and experiments, is presented. Section 4 reports on the evaluation of the developed model, while Section 5 concludes the article.

2 Literature Review

Social media platforms such as Facebook and Twitter currently attract millions of users around the globe, giving them an open platform to share their thoughts and opinions (Mouthami et al., 2013). Traditionally, sentiments are classified in binary form, i.e., either positive or negative (Mouthami, Devi, & Bhaskaran, 2012) but have evolved over the years with the addition of the neutral class (Ayetiran, 2022) in numerous tasks. Sentiment classification is an offshoot text classification (Ayetiran, 2020; Novotný, Ayetiran, Stefánik, & Sojka, 2020; Sebastiani, 2002), including the ones with applications in the legal domain (Ayetiran, 2013, 2017). There are several machine learning methods for sentiment categorization such as Maximum Entropy, a feature-based model that does not take independent assumptions (Cheong & Lee, 2011). Stochastic Gradient Descent is another machine learning-based algorithm that is capable of making the classifier learn even if it is based on a non-differentiable loss function (Bifet & Frank, 2010). Breiman (2001) introduced a Random Forest, which targets the enhancement and storage of classification trees. Wang, Can, Kazemzadeh, Bar, and Narayanan (2012) introduced a system for real-time Twitter analysis with an experiment on the 2012 U.S. presidential election cycle. Naïve Bayes is another easy-to-implement and efficient machine learning algorithm and was originally proposed by Thomas Bayes (Husain & Singh, 2014). Support vector machine (SVM) is one of the most famous supervised machine learning-based classification algorithms (Khairnar & Kinikar, 2013). Different studies and research featuring these different machine learning algorithms and tools are available on sentiment analysis, but as compared to several other methods, SVM performed better in terms of accuracy and efficiency as the results were comparatively higher (Kumar & Sebastian, 2012; Pak & Paroubek, 2010). There are some research studies on social unrest in the fields of social and political sciences, each employing varying approaches to the subject. Studies have been carried out at the macro level: looking at the influence of the political and economic situation in general on protest moods (Dalton, Sickle, & Weldon, 2010) and viewing protests as organized social movements (Porta & Diani, 2007). From the perspective of an individual, scholarly research has been concerned with recruitment and activism (Verhulst & Walgrave, 2009; Walgrave & Wouters, 2014), as well as factors and cognitive processes affecting individual participation in protests (Klandermans & Oegema, 1987; Stekelenburg & Klandermans, 2013; Stekelenburg, 2013). Such factors have also been used for modeling protest participation (Klandermans & Oegema, 1987; Kelloway, Francis, Catano, & Teed, 2007; Stekelenburg & Klandermans, 2013) and in a recent study combining social science with a computational simulation of the dynamics of participants (Lu, 2016). Since online social media gained worldwide popularity, a wide stream of research studies have emerged on the extraction and structuring of the available information. Among the various social media websites, Twitter is especially popular among scholars due to the availability of large numbers of data that can be collected while the event is unfolding – a naturally occurring experiment. Some approaches to information extraction from Twitter are described by Kumar, Morstatter, and Liu, (2014) and Imran, Elbassuoni, Castillo, Diaz, and Meier, (2013). Scholarly works taking advantage of these technologies have been concerned with the public response to extreme events (Tyshchuk, Li, Ji, & Wallace, 2013; Tyshchuk, Wallace, Li, Ji, & Kase, 2014; Vieweg, 2012), modelling human behavior (Tyshchuk & Wallace, 2015; Korolov et al., 2015, 2016c), forecasting actions (Korolov et al., 2015,2016a), attitudes (Magdy, Darwish, & Weber 2015), and even election results (Tumasjan, Sprenger, Sandner, & Welpe 2010). Research studies on the role of social media in protests emerged after the Arab Spring when its contribution to the emergence of revolutions in Arab countries became apparent. Studies were undertaken on the role of social media in revolutions (Oh, Eom, & Rao 2015; Tufekci & Wilson, 2012), information dissemination during protests (Starbird & Palen, 2012; Valenzuela, 2013), and activism and support of social movements (Freelon, McIlwain, & Clark, 2016; González-Bailón, Borge-Holthoefer, Rivero, & Moreno, 2011; Magdy et al., 2015). Despite the abundance of research studies on social media and protest, studies focused on the prediction modeling of a protest using data from social media are scarce. In one of the few articles on this (Filchenkov, Azarov, & Abramov, 2014), the authors compare the prediction of a protest with the prediction of election results. A study reported by Compton et al. (2013) presents a solution, relying on a constructed vocabulary from data. A recent work (Cadena et al., 2015) addresses this problem using influence cascades as a predictor of protest. This study, however, also relies on a constructed vocabulary. Ranganath et al. (2016) exploit the previous activities of a social media users to predict whether their next tweet will be related to a protest. The relationship between social media activities and protests during the Arab Spring was established in Steinert-Threlkeld, Mocanu, Vespignani, and Fowler (2015), using hashtags to filter relevant tweets. Our approach also uses a geographical and topical grouping of tweets. The notable difference in our approach is that we were able to develop a standard dataset on which a model was trained for classification of tweets into positive and negative classes with excellent performance in all evaluation metrics used. It was observed that previous works have either used baseline classification with two or three machine learning models. As mentioned earlier, our choice of SVM is due to its great classification accuracy based on evaluation studies available in the literature (Kumar & Sebastian, 2012; Pak & Paroubek, 2010).

3 Methodology

The classification/prediction system processing pipeline is presented in Figure 1.

Figure 1

Prediction system development pipeline.

3.1 Data Analysis and Development

Twitter was searched for relevant tweets with #EndSARS using the Twitter Application Programming Interface (API). #EndSARS is a popular subject of conversation on Twitter, which started toward the commencement of the protest in October 2020 and continues till today. For the purpose of our experiment, 400 raw tweets were retrieved, with each consisting of several fields, including the tweet’s (text), date, number of retweets (reposts), etc. Some of them also have null fields. These were discarded during the pre-processing phase.

3.1.1 Exploratory Data Analysis (EDA)

EDA was carried out on the selected dataset to help understand the dataset better and to help determine the pre-processing to be carried out on the data. The exploratory dataset carried out on the dataset includes data size and data description. EDA is also used to investigate and summarize the main characteristics of the dataset using visualization/plot methods. Diverse machine learning libraries were used through the Python programming language to carry out the exploratory analysis. Machine learning library is a library of functions and methods that enables easy scaling of scientific and numerical computations to streamline machine learning workflows. Under different emotional conditions, humans tend to behave differently. This includes the way they talk and express their feelings. Therefore, it is important to rely not only on the vocabularies used but also on the expressions and sentence structures used under the different conditions to quantify and model these feelings. Therefore, these were considered when manually selecting the 200 tweets. We used a Python library called Tweepy (Roesslein, 2020), developed to provide access to the official Twitter streaming Restful API with predefined methods that accept various parameters and return responses. The library is used to extract tweets containing one or more social unrest keywords for the purpose of this analysis.

3.1.2 Data Cleaning and Preprocessing

Since the dataset contains a lot of irrelevant and ambiguous information, preprocessing or data cleanup is crucial to prevent noise. As a result, before processing, extraneous values and other symbols that are not essential for classification purposes are deleted. The numerous preprocessing activities used in this investigation are as follows:

1. Removal of null data from the dataset.

2. Investigating all elements within each feature.

3. Cleaning of the dataset.

4. Removal of stop words using the natural language toolkit (Bird, 2006).

5. Conversion of tweets to lowercase.

6. Stemming of words.

7. Punctuation removal.

8. Count vectorization of the data.

3.1.3 Dataset Annotation

The preprocessed data are manually annotated by three expert annotators. Tweets on which they have a unanimous agreement were classified into two groups: positive and negative classes because almost all the collected tweets belong to these categories. A basic binary classification typically involves two classes, usually positive and negative for sentiment analysis. If the data are classified with positive and negative polarity, then the remaining data for which polarity cannot be decided due to disagreement among the annotators are earmarked for emoticon analysis. Furthermore, based on the emoticons used in the earmarked data, tweets are classified accordingly as either having a positive or negative polarity. Thereafter, the remaining disputed tweets without emoticons are discarded. There are popular libraries used to achieve this like TextBlob (Loria, 2018), a text processing package for Python 2 and 3. It offers a basic API for doing standard NLP activities including part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, and translation, among others. But the issue with these libraries is that they mainly detect emotions like sadness, happiness, etc. Furthermore, the results from these tools are not perfect and not good enough to entirely rely on in sensitive cases such as this. In our own case, we need to classify based on whether a tweet or text is in support of protest (positive) or not (negative). For reliability and accuracy, we develop the dataset manually based on unanimous agreement by three different annotators. Table 1 shows a sample of the annotated dataset with two categories: a class tagged as 0 (negative), which are the texts or tweets against social unrest, and a class tagged as 1 (positive), which consists of tweets in support of social unrest.

Table 1

Samples of the manually annotated dataset

Text	Label
The movement #EndSARS Now Is basically irrelevant pls… What we need now is to get them do their jobs not harassing people they are suppose to be protecting	0
We need them… SARS are special Anti-Robbery Squad… They’re fighting Criminals in major cities…	0
Instead of the policemen to save this mans life, they stole from him as he was bleeding to death and helped the driver escape. Never forget! This is why we keep fighting!	1
This rubbish needs to stop. Shameless government and their pawns #EndSARS #end oppression https://t.co/4iflRBvYlM	1

3.1.4 Emoticon Analysis

After the initial text classification phase, some of the data on which the annotators do not have a unanimous decision are discarded, but if they contain, emoticons are further analyzed. If the emoticon is classified as positive, the tweet sentiment is classified into a positive class appropriately and vice versa. The algorithm for classifying disputed tweets based on emoticon analysis works as follows:

1. The disputed tweets are searched for emoticon(s). If present, the emoticon(s) is/are analyzed for positive or negative emotion.

2. Investigate the symbolic icon features in the emoticons. Each emoticon has a corresponding symbolic icon. For example, the symbolic icons for smiley and angry faces are :-) and > :(, respectively.

3. Since emoticons can belong to either of two classes: positive, e.g., smiley, happy, and laughing, and negative, e.g., frown, angry, and sad. The emoticons are then classified accordingly by matching their symbolic representations to either of the two classes.

Samples of the manually annotated dataset are presented in Table 1 and count plot in Figure 2.

Figure 2

A count plot of the manually annotated dataset.

The statistics of the dataset are as follows:

1. 100 positive tweets from the selected overall 200 tweets supporting the social unrest campaign.

2. 100 negative tweets being the other half of the entire tweets that are against the protest.

4 Model Evaluation

4.2 Results and Discussion

Experimental results obtained from the SVM classifier are shown in Table 3. The confusion matrix in Figure 3 shows 17 instances as true positives, 3 false negatives, 1 false positive, and 19 true negatives. The model achieves a 90% accuracy, 94% precision, 85% recall, and 89% F1 score. The results show that the model is better at accurately predicting negative instances. Observation on the test results shows that the total number of unique words and training data have a big influence on the accuracy rate attained. SVM has demonstrated to be a good classifier to learn these words with great predictive power when applied to a new dataset, leading to high accuracy in identifying sentiments in tweets.

Table 3

Model prediction results

	Metrics
	Accuracy	Precision	Recall	F1-score
Result	0.9000	0.9444	0.8500	0.8947

Figure 3

Model’s confusion matrix with labels.

5 Conclusion

Social unrest increases risks in the financial, economic, and political spheres, with possible long-term effects depending on the duration and frequency. An early warning system for these catastrophes may be helpful for policy makers, according to the rising frequency of such incidents over the previous few decades. Causes of unrest are diverse and interact with a wide variety of socioeconomic variables in complicated and non-linear ways, despite the fact that just a few indicators have been recognized in the literature as being significant in causing disturbance. To predict the occurrence of social unrest, in this work, we have annotated a dataset based on Nigeria’s #EndSARS. Furthermore, we have utilized SVM to model public sentiments toward the protest. Experimental results show that the model generalizes well with good performance. This can be used in developing early warning systems for authorities about potential social unrest.