Factors Predicting Acceptance and Recommendation of Covid-19 Vaccines Among Previously Infected Academic Dental Hospital Personnel; An Artificial Intelligence-Based Study

: Objectives The study aims to construct artificial neural networks that are capable of predicting will-ingness of previously infected academic dental hospital personnel (ADHP) to accept or recommend vaccines to family or patients. Methods: The study utilized data collected during a cross-sectional survey conducted among COVID-19 infected ADHP. A total of ten variables were used as input variables for the network and analysis was repeated 10 times to calculate variation in accuracy and validity of input variables. Three variables were determined by the best network to be the least important and consequently they were excluded and a new network was constructed using the remaining seven variables. Analysis was repeated 10 times to investigate variation of accuracy of predictions. Results: The best network showed a prediction accuracy that exceeded 90% during testing stage. This network was used to predict attitudes towards vaccination for a number of hypothetical subjects. The following factors were identified as predictors for undesirable vaccination attitudes: dental students who had an insufficient vaccine awareness, a long symptomatic period of illness, and who did not practice quarantine. Conclu-sions: It is concluded that vaccine awareness is the most important factor in predicting favorable vaccine attitudes. Vaccine awareness campaigns that target ADHP should give more attention to students than their faculty.


Introduction
The emergence of coronavirus disease-19  in December 2019 has instigated the introduction of major changes to many healthcare systems around the world. The causative virus, severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), is characterized by the highly contagious global spread that has cancelled borders and transformed the world into one small village. The infection itself may take the form of severe disease or less frequently, asymptomatic illness in approximately 17.9-33.3% of cases [1,2], and in many cases the patient lies under the burden of long-term debilitating complications [3]. One of the methods used to mitigate spread of the virus is vaccination. As of November 4, 2022, 12.92 billion doses of COVID-19 vaccines have been administered globally, and 2.84 million are now being administered each day [4]. Several vaccines got emergency approval before or after phase III trials and as of September 2021, there were 12 marketed vaccines against the SARS-CoV-2 [5]. However, there is still resistance to these vaccines represented by anti-vaccine campaigns. This dictated activating several reforms and modifications of healthcare systems that involved establishing new or modified roles of healthcare professionals. All healthcare professionals whether those working in the frontline or elsewhere, are required to take a stand to curb the increasing numbers of infections. These professionals remain the most trusted advisors and influencers of vaccination decisions [6]. Healthcare personnel were among the first groups to receive COVID-19 vaccines, and they often showed favorable vaccine attitudes, particularly nurses and physicians working in the frontline [7,8].
Several studies acknowledged the potential role of academic institutions in the COVID-19 pandemic. On one hand, campuses have been described as "super spreaders" of the infection to adjacent communities [9], and on the other, a significant role in vaccine awareness and acceptance has been attributed to university personnel including faculty and students. Within the context of healthcare students, few studies were conducted to evaluate their vaccine attitudes, wherein medical, nursing, and pharmacy students reportedly had an acceptance rate of 76%. [10] Dental students and their faculty are a unique group of healthcare personnel. Their attitude toward COVID-19 vaccines is important for numerous reasons. Safety and durability of the academic process should be adequately sustained to guarantee provision of successful clinical dental training. Moreover, an essential characteristic of dental education is to influence attitudes and behaviors of patients by spreading public awareness on several communicable and non-communicable diseases [11], including the introduction of immunization training in dental education [12] that eventually proceeds to advocating and administering vaccines [13]. Dental students and their faculty have a golden opportunity to actively promote COVID-19 vaccines, particularly since a substantial proportion of patients get in contact with their dentists at least once a year [11]. Recently, some researchers evaluated attitudes of dental students regarding COVID-19 vaccines. Riad et al (2021) surveyed dental students from 22 countries and reported that there is still a percentage of 22.5% who were hesitant, while 13.9% rejected COVID-19 vaccines [14]. At this advanced stage of the pandemic more studies are required to evaluate attitudes of faculty and compare them to those of their students.
Artificial intelligence (AI) is concerned with computational understanding of intelligent behavior, and creation of artifacts that exhibit such behaviours. [15] Artificial neural networks (ANN) is an intelligent data analysis tool that is claimed to be more accurate than classic regression [16]. An ANN consists of an interconnected group of artificial neurons that process information using a connectionist approach to computation. ANNs can be used to improve delivery of care at a reduced cost, and they are increasingly used to inform health care management decisions [17]. Despite the utilization of ANN in many aspects of prediction in healthcare including diagnosis and prognosis of disease [18][19][20], AI methods including ANN are still underestimated, whether in predicting healthcare or health risk behaviors. Therefore, this study aimed to construct ANNs that can predict willingness of dental students and their faculty to accept or recommend COVID-19 vaccines to patients and family.

Participants
The study was a cross-sectional analytic study conducted among faculty and clinical students at Taibah University Dental Hospital located in Al Madinah, western Saudi Arabia during March-August 2021. Inclusion criteria were clinical dental students, interns, and faculty with past COVID-19 infection who were working at the hospital during the study period. Staff staying abroad during the study period, and pre-clinical students were excluded from the study.
Sample size was determined using the epidemiological software Epi Info™ (CDC, Centers for Disease Control, Atlanta, USA). It was based upon the total number of infected clinical students and faculty at the time of the study which was estimated to be a total of 66. The expected frequency of accepting or recommending vaccination (outcome probability) was assumed to be 70%, based on a range of 63% -76.9% COVID-19 vaccine acceptance rate among comparable populations in previous studies [7,8,10]. A sample size of 62 participants was determined to provide 97% confidence level at 3% margin of error.
The list of all infected students and faculty names and their contact details were obtained from the college administrative office. All eligible (infected) faculty and students at the hospital were contacted and invited to participate in this study during working hours.

Study tool and ethical approval
Study tool was an anonymous, online questionnaire created using Google Forms. Consenting participants completed the questionnaire in presence of co-investigators without interfering or influencing their responses. The questionnaire was composed of 19 closed-ended questions divided into three sections of demographics, COVID-19 clinical factors and outcomes and finally a section on attitudes towards vaccination. A pilot test was performed to ensure clarity of questions and reproducibility of responses. A group of five students and five faculty were invited to complete the questionnaire on two occasions separated by one week to compare responses. Face validity was carried out within the authors group who did not participate in the questionnaire design. Unclear or vague questions were modified. The calculated Cronbach alpha and Kappa values were considered acceptable (0.72 and 0.77 respectively). Ethical approval was obtained from Taibah University College of Dentistry Ethics Committee, reference # TUCDREC/17012021/NDar-Odeh.

Build-up of artificial neural networks and prediction of attitude towards vaccination
Specific variables were used as input variables for ANN analysis. These input data were gender, age, professional role (student or faculty), nationality (Saudi or non-Saudi), quarantine practice during infection (yes, no), vaccine awareness (yes, no), number of acute symptoms, duration of illness in days, number of long-term post-COVID complications, and number of socioeconomic complications encountered by the participant due to infection. Output (dependent or predicted) variables for constructed networks were willingness to take the vaccine, willingness to recommend vaccine to patients, and willingness to recommend vaccine to family. Each network was constructed with two hidden layers of neurons. The two layers of neurons were used in the buildup of the network with 10 neurons in the first layer and three in the second. Artificial neural networks were constructed using IBM-SPSS (The Statistical Package for Social Sciences, version 21), and were trained with data of 19 subjects (30%) of the sample. The same network was then used to predict attitude for the remainder 43 subjects (70%). The analysis was run 10 times, each time with different random input data (these were always input and output data for 30 % of the subjects). Each neural network of these 10 networks produced output during training for the exposed random 30% of the sample (training data) and produced output for the hidden 70% of the sample (testing data). Accuracy of the network was displayed at the end of each run.

Results
Total number of clinical students and staff at time of the study was 316. With total number of 66 previously infected personnel in the hospital, the infection rate was estimated to be 19.6%. However, only 62 individuals consented to participate in the study (response rate = 93.9%). Table 1 shows the results of the 10 networks with full number of input variables each (10 input variables). These 10 networks were constructed with random allocation of 30% of data for training and 70% of data for testing. The output was always the prediction of the three attitudes towards vaccination.
Overall, in average, these networks were 87.1% accurate (range: 74.5%-100%) during training and 81% accurate (range: 76.4-86.7%) during testing (Table 1). Networks also indicated important variables for the calculation (Figure 1), i.e. the variables that have the highest impact on neural synaptic values. Reported important variables varied between the different networks. However, Figure 1 shows important variables as reported by the best network with full number of independent variables (10 variables). Hence, variables of gender, professional role group, nationality, quarantine practice and number of long term symptoms were removed to investigate impact on accuracy of the resulting networks with reduced number of predicting variables (only 5 independent variables).
Analysis was carried out again by constructing 10 different networks with random input and output values for the training phase of the analysis. Table 2 shows average accuracy of these 10 networks with reduced number of predicting variables.   Table 3 shows average accuracies (and their ranges) for the 10 networks with only five input variables.
Overall, in average, the networks were 89.1% accurate (range: 81.9%-98.3%) during training and 81.9% accurate (range: 74.2-91.3%) during testing. Removing redundant variables resulted in slight improvement in average prediction capability of the 10 networks with 5 variables only during testing (i.e. from 81%-81.9%). The best network with prediction ability during testing was a 5-variable network. This network had a testing accuracy of 91.3%. Table 4 shows importance and normalized importance of independent variables (similar to B weights in regression analysis) as reported by the network of best performance (with 5 input variables only).
The following ROC (receiver operating characteristics) graphs in Figure 2 show the accuracy of the model during the prediction of three attitudes. It was noticed that the straight line at 45° at the middle of the three ROC graphs shows the accuracy of a random model with no predictive capacity indicating that the more the curves are away from this random line the better is the model. This shows the performance of the best network with 5 input variables.   Figure 3 below shows three graphs of cumulative gains charts for the prediction of three attitudes towards vaccination (best network with 5 input variables). Figure 4 is the Lift chart of three attitudes to vaccination of the best network with 5 variables only.
The data file was modified to include further fictitious input data of hypothetical patients with assumptions for the most important variables as displayed in Table 6.
The network predicted that ADHP will always recommend vaccines to patients. It predicted 3 situations when the young 21-year-old student, regardless of gender and duration of illness, will not recommend vaccine to family. These situations were lack of quarantine during illness and lack of vaccine awareness. The network also pre-dicted five situations where students would not demonstrate willingness to accept vaccines. These situations are not gender sensitive nor sensitive to the experience of long COVID symptoms. These situations will occur mainly when the students did not practice quarantine (four out of six times), had a long duration of illness (in four out of the six cases) and did not have sufficient vaccine awareness (in four out of the six cases).

Discussion
Since the introduction of COVID-19 vaccines in December 2020, the scientific community has closely monitored the efficacy and safety of these vaccines [21]. Attitudes of pivotal sectors of the population, including healthcare workers, towards these vaccines are important. Further, the evaluation of these attitudes is an important step towards establishing and designing the most effective vaccination campaigns. Healthcare authorities require an accurate and reliable method that can be utilized in Figure 2: ROC curve (receiver operating characteristics) of prediction of willingness to accept vaccine (left graph), willingness to recommend vaccine to patients (middle graph) and willingness to recommend vaccine to family (right graph) by the best 5 input variable network. Importance can also be shown in terms of areas under these as reported in Table 5 below. predicting attitudes and hence identifying the appropriate targets for vaccine education and awareness among healthcare professionals. In recognition of their important role in spreading awareness and promoting population acceptance of vaccines, an increasing number of studies has been conducted to assess attitudes of healthcare professionals towards these vaccines. In recognition of their role in public awareness and healthcare behaviors, a number of studies were conducted recently among dental students in different parts of the world to assess their attitudes in vaccine acceptance and hesitancy [14,[22][23][24][25]. All these studies utilized conventional statistical methods such as regression and other inferential statistics to obtain their results. To the best of our knowledge, this is the first study to utilize ANN in predicting vaccine attitudes among a unique group of ADHP. In this study, ANN was designed and tested to produce the most accurate networks for prediction. Most of the healthcare workers evaluated in the studies done so far were in the main inclined to accept COVID-19 vaccines [8,26], however, their attitudes towards recommending vaccines to others are still under-investigated. This study utilized AI in predicting vaccine attitudes among a group of dental faculty and their students. The most accurate network in this study predicted that faculty members will always accept and recommend vaccines to others. A higher acceptance rate of COVID-19 vaccines was reported previously in older adults, as it was estimated to increase from 71.2% to 91.2% in the 31-40-year-old, and 51-60-year-old groups respectively [26]. Among healthcare workers, vaccine acceptance has also reportedly increased in subjects with older age and higher education levels [27,28]. The network also predicted that three hypothetical 21-year-old students out of eight refused to recommend vaccines to family. Those cases were males or females (gender was not a determinant factor) with variable periods of symptomatic infection. In these three cases, students did not practice quarantine and they indicated lack of vaccine awareness. This confirms that lack of vaccine awareness and practicing health risk behaviors (failure to quarantine) will both predict refusal to recommend vaccine to family. Previous studies on dental students concluded that lack of vaccine awareness is a major predictor of unfavorable vaccine attitudes [22]. HCWs are more likely to make healthcare decisions established on evidence-based published literature of safety and efficacy [28], which is an important aspect that has not yet been completely established. This emphasizes the importance of timely publication of research data pertinent to vaccine efficacy and side effects for the healthcare community and the public as well [29]. The network also predicted that six out of eight 21-yearold students are not willing to accept the vaccine. Those six students were males and females and they had variable experience with long COVID, indicating that gender is not a determinant factor in vaccine attitudes. This is contrary to other studies that reported more favorable vaccine attitudes among the male gender [27]. This can probably be explained by the more significant role of young age that cancels the role of gender. It also seems that long COVID in this study is not a determinant factor in vaccine attitudes. However, increasing attention is currently being paid to long-term health complications of COVID-19, which on many occasions cause debilitating effects on various organs of the body [3]. Although a number of participants in this study suffered long COVID, it had no influence in predicting vaccine attitudes. On the other hand, these students admitted that they did not quarantine, and they did not have sufficient vaccine awareness, which highlights once again that lack of vaccine awareness is an adverse factor in vaccine attitudes. As for failure to practice quarantine, it should be noted that although quarantine is a recommended practice for COVID-19 patients, the period of quarantine is considered a debatable issue, due to the appearance of asymptomatic cases and variable incubation period [30]. Subjects who show health risk behaviors may do so because of lack of sufficient evidence to negate these behaviors. Studies conducted among comparable populations of dental practitioners showed that COVID-19 vaccine acceptance was correlated with vaccine knowledge, and it was adversely correlated with specific health risk behaviors like visiting the physician only when the need arises [31].
The study predicted that students will always recommend vaccines to patients. Important predictors were in the following order: symptom duration, age, number of socioeconomic sequelae, vaccine awareness, and number of symptoms. This indicates that willingness of students to recommend vaccines to patients was mainly predicted by clinical attributes of illness including number/ duration of symptoms, and socioeconomic sequelae.
The study has limitations. It was a single-center, cross-sectional study that employed self-perceived data. However, it should be noted that the participating hospital is a major teaching hospital that represents the only academic institution providing dental services to a high flow of patients in Al Madinah region. Further, the anonymous nature of the questionnaire, and the inclusion of only clinical students and their faculty contribute to the reliability of study data.

Conclusions
Dental faculty are willing to accept and recommend COVID-19 vaccines to patients and family. On the other hand, dental students are willing to recommend vaccine to patients, but some of them are still not willing to either accept the vaccine or recommend it to family. Vaccine awareness was the most important factor predicting all these attitudes. More efforts are warranted from healthcare authorities to publicize all current and updated data on COVID-19 vaccine safety and efficacy to promote confidence and assure all categories of healthcare personnel.

Funding information:
The authors state no funding involved.

Conflicts of Interest:
The authors state no conflict of interest.
Informed consent: Informed consent has been obtained from all individuals included in this study.

Ethical approval:
The research related to human use has been complied with all the relevant national regulations, institutional policies and in accordance with the tenets of the Helsinki Declaration, and has been approved by the Taibah University College of Dentistry Ethics Committee, reference # TUCDREC/17012021/NDar-Odeh.

Data availability statement:
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request. Table 6: Net predictions of attitudes towards vaccine for a number of hypothetical conditions. Nationality was assumed to be Saudi, number of symptoms was assumed to be 3 and number of hardships encountered during and after infection were assumed to be 1.