Mobile app for COVID - 19 patient education – Development process using the analysis, design, development, implementation, and evaluation models

: There are many factors that can lead to the transmission of coronavirus disease 2019 ( COVID - 19 ) , one of which is the lack of knowledge on the virus and its prevention, notably in Indonesia. This study was focused to design and build an interactive learning app for COVID - 19 education. The design of this study was research and development, and in terms of the app devel - opment, it utilized the analysis, design, development, implementation, and evaluation model. The project was carried out from July to December 2021, and it involved 25 study participants. The ﬁ ndings of this study con ﬁ rmed that the educational app consisted of education, a symptom checker, a list of vaccine information links, the latest news, and COVID - 19 statistics. The validity assessment showed that the educational app in this study was very appropriate to be utilized as a digital medium for patient education. In addition, it was also con ﬁ rmed that all the functions of the app worked well, and participants strongly agreed that the educational materials and features of the app were interesting and helped them to learn COVID - 19 prevention easily. It could be concluded that the app could be used as a learning medium for patient education. Further studies, however, were needed to prove its e ﬀ ectiveness in the real clinical world.


Introduction
Coronavirus disease 2019 (COVID-19) is a new infectious disease caused by a novel strain of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). The virus belongs to a large family of respiratory viruses that can cause illness in animals or humans. Since the first outbreak in November 2019 in Wuhan, China, the infectious disease has been moving rapidly into a worldwide pandemic, resulting in serious implications for society [1,2]. The latest report by the World Health Organization (WHO), as of March 20, 2022, showed that over 468 million confirmed cases and just over 6 million deaths have been reported globally. According to the report, compared to the previous week, the number of new cases has continued a decreasing trend [3].
The signs and symptoms of COVID-19 disease range from mild to severe, but include fever (higher than 37.5°C), dry cough, and tiredness, as well as SARS and Middle East respiratory syndrome (MERS) [4,5]. Other symptoms include runny nose, sore throat, stuffy nose, diarrhea, and loss of taste or smell [6]. In a severe condition, pneumonia and shortness of breath may occur with or without chest discomfort. In general, the symptoms will go away without any special treatment [7,8]. However, older people and individuals with certain medical conditions, namely, chronic respiratory disease, diabetes, hypertension, and cardiovascular conditions, need special medical attention since they are more likely to develop serious illness [9,10]. If left untreated, the disease can potentially lead to death due to decreased oxygen levels and other conditions such as heart failure, arrhythmia, myocarditis, and myocardial infarction [11][12][13].
There is no exact cure for COVID-19 disease. According to the WHO, the best way to get rid of and slow down the spread of COVID-19 is to be well informed about the disease [14]. This is why education on the disease, including the disease it causes and how it spreads, plays a key role in the prevention and control of COVID-19 disease. An interactive learning app based on a smartphone application has the potential to be used as a health promotion medium [15]. Unfortunately, the learning app containing education on COVID-19 is still not available in Indonesia. This study was primarily focused on designing and developing an interactive learning app on COVID-19 education, including basic knowledge such as pathophysiology, signs and symptoms, treatment options, and prevention methods, as well as the latest news and current information on the number of new cases, death, and recovery rate in all provinces of Indonesia. Hopefully, this learning app can improve the level of knowledge and awareness about COVID-19 disease among Indonesian millennials.

Materials and methods
The type or design of this study was research and development, implemented to create a new product and evaluate its effectiveness [16]. In terms of app development, this study employed the analysis, design, development, implementation, and evaluation (ADDIE) model, which consists of five phases, as the name implies analysis, design, development, implementation, and evaluation [17]. Figure 1 shows the five steps of the ADDIE approach applied in this study.
The project, which started from user needs assessment to evaluation of the app, was carried out from July to December 2021 at the Universitas Muhammadiyah Yogyakarta, Bantul, Indonesia. It involved 25 study participants consisting of 10 each of health and non-health students, 3 medical professionals (1 doctor, 1 nurse, and 1 pharmacist), and 2 IT experts from the Universitas Muhammadiyah Yogyakarta. All subjects owned a Google Android smartphone and were able to run apps on it. Meanwhile, the object of the study was Educovid-19, an app that contains educational material on the prevention of COVID-19. The tested aspects were feasibility of the app in terms of suitability or relevance of educational material, organization of the material, evaluation of the material, language used in the material, effects on learning strategies, software engineering, and visual display or user interface. Data collection was carried out by distributing questionnaires.
The research instruments used in this study were qualitative questionnaires for needs analysis, validation of educational material by health professionals, validation of app performance by IT experts, and quantitative questionnaires for usability assessment by students (endusers). The qualitative data were analyzed by recapitulating data for app improvement purposes, while the quantitative data were analyzed using univariate analysis by calculating the frequency distribution and the percentage of application assessment scores.

Participants
The majority of the participants involved in this study were female (n = 15; 60%), aged 15-20 years old (n = 15; 60%), undergraduate (n = 20; 80%), and college students with both health and non-health background (n = 10; 40%). Table 1 shows the demographic of the study participants with various backgrounds (sex, age, level of education, and backgrounds).

Analysis
During this phase, the study participants (students, health professionals, and IT experts) were asked to provide feedback on COVID-19 preventive functions, features, and educational material. The feedback was provided through a questionnaire distributed using a G-form. It was reported that almost all participants (n = 24; 96%) wanted educational material on COVID-19 prevention, followed by other features including COVID-19 statistics (n = 20; 80%), symptom checker (n = 17; 68%), latest news on COVID-19 (n = 15; 60%), vaccine information (n = 14; 56%), and others. The detail of the results of the user needs assessment is shown in Table 2.
According to the results of interviews with study participants, the app should be made simple and quickly accessible, the display or interface should be eye-catching and not too flashy, the educational material should be made simple and easy to understand, multimedia (image or video) should be added to the educational material, and there should be no ads in the application.

Design
This stage, as the name suggests, was aimed to design a user interface and the function of each app feature. The design approach was carried out using storyboards. The storyboards of the app are shown in Figure 2, while the details of the storyboards are depicted in Table 3.

Development
During this phase, the storyboards for the educational app were developed to be a prototype. The development process of the app was carried out using Android studio. Figure 3 shows a list of screenshots of the user interface of the app. Figure 3(a) shows a screenshot of the main menu of the app containing many features, including educational materials on COVID-19 prevention, a symptom checker, the latest news on COVID-19, and COVID-19 statistics.  The questionnaire was sourced from the "guidelines of the treatment of COVID-19 patients based on symptoms" by The Ministry of Health of The Republic of Indonesia. This page was designed like a calculator, and it operates by providing suggestions on the treatment after the questionnaire was fulfilled by patients or users. The results of the test are shown in Figure 3(e). The symptom checker page consists of information including the patients' place of care, such as self-isolation, health facilities provided by government, field hospital, referral hospital, nonreferral hospital, High Care Unit, and Intensive Care Unit. In addition, the page also has many other useful information, including therapy for no-symptom, mild, moderate, and severe, as well as treatment duration. Figure 3(f) shows a list of vaccine information links, Figure 3(g) shows a screenshot of COVID-19 latest news gathered from many reputable sources, and Figure 3(h) represents the COVID-19 statistics, containing new cases and average numbers of cases in a week. All the data were sourced from Google.

Implementation
During this phase, the prototype (trial version) of the app was distributed to all study participants to validate the app based on many aspects, ranging from medical professionals to students. The participants from medical professionals (n = 3) were asked to carry out an assessment to validate the prototype based on the aspects of content inside the app, while IT experts (n = 2) were on the aspects of app function, and students (n = 20) were on the aspects of the user. To obtain the validity results of the app, a survey was carried out for 1 week. The details of the results of the assessment are shown in Figures 4-6. According to Figure 4, it was reported that the average value of the assessment of the app prototype based on all aspects of material assessment was 4.6 with a percentage value of 92%, indicating that the prototype was very appropriate to be used as an educational app. Similar results were discovered in the aspects of assessment by IT experts (Figure 5), showing that the average value of the assessment of app was 4.25 with a percentage value of   85%, indicating that the prototype was very appropriate to be implemented as an educational app.
The results of the assessment of prototype by users, as shown in Figure 6, found that the average value of the assessment of app prototype based on all aspects of material assessment was 4.7 with a percentage value of 94.2%, indicating that the quality of educational material inside the app and its implementation based on aspects such as the relevance of educational material, the organization of the material, the evaluation of the material, the language used in the material, the effect or impact for the patient education, the software engineering, and the visual display or user interface were very appropriate for use for patient education.

Evaluation
The evaluation of the educational app was carried out in two ways, namely black-box and user acceptance. The details of both tests are explained subsequently.

Black-box testing
The test was primarily aimed to determine the functional specifications of the developed software or app by checking whether the output was as expected or not. It was carried out on an ASUS Vivo-Book S14 S430UN -EB732T laptop (Intel Core i7-8550U 1 processor, 6-4.0 GHz), one Samsung Galaxy Tab S4, and one Samsung Galaxy A7 smartphone. The results of the test showed that the educational app could function well. The details of the results of black-box testing are shown in Table 4.

User acceptance
A user acceptance test was carried out by introducing the features, functions, and methods to run the educational app to all participants, followed up by practicing to run the app directly. The test was conducted at the Muhammadiyah University of Yogyakarta, Bantul, Indonesia, in December 2021 and involved 30 study participants. The participants filled out a questionnaire to provide an assessment of the app. Table 5 shows the details of the results of user acceptance.

Discussion
COVID-19 is currently a serious global health issue, with the number of cases increasing every day. According to previous studies, there are many factors that can lead to the transmission of the virus, one of which is the lack of public knowledge about the virus and its prevention [18][19][20], particularly in Indonesia. This is why the study aimed to design and build an educational app that can be This educational app was developed using the ADDIE approach, which consists of several phases, including analysis, design, development, implementation, and evaluation. Based on the validity results of the app, it was determined that the educational app used in this study was very appropriate to be utilized as a digital medium for patient education. The validity assessment of the app was carried out by medical professionals for the relevance of educational content, the organization of the material, the evaluation of the material, the language used in the material, and the effect or impact on patient education, while IT experts assessed the aspects of the language used in the material, the effect or impact for the patient education, the software engineering, and the visual display or user interface, and the students as users covered all aspects ranging from the relevance of the educational content to the visual display or user interface.
In terms of performance, it was confirmed that all features and pages of the app (main menu, education, symptom checker, a list of vaccine information links, the latest news, and the COVID-19 statistics) worked well. The performance of the app was assessed by utilizing black-box testing, which was found in a previous study [21]. Meanwhile, regarding the user assessment, this study used a questionnaire of user acceptance sourced from a previous study [22]. The findings of the user assessment confirmed that almost all participants strongly agree that the user interface of the app was very interesting, the app was interactive to its users, easily installed and easy to run, and enthusiastic when running the app. From the aspects of educational materials, almost all participants strongly agree that the materials (both text and video) were in accordance with a reliable source, namely the Decree of the Minister of Health of the Republic of Indonesia, No. HK.01.07/MENKES/413/2020 regarding COVID-19 prevention and control guidelines. Meanwhile, regarding the aspects of the impact of the app on its users, it was reported that the educational materials and features of the app were interesting and helped participants to learn COVID-19 prevention easily, therefore motivating them to learn the topic. Since the materials and features of the app were useful, the participants strongly agree that the app could be used as an alternative learning medium for patient education and health promotion.

Strengths and limitations of the study
The strength of this study was all the processes of design, development, and implementation of this educational app involved experts and users of the app (medical professionals, IT experts, and students as users). Therefore, the app has many perspectives from experts or professionals and end-users. In addition, this study was also the first of its kind in Indonesia. However, it should be evaluated in the context of study limitations. First, the study only involved participants with high education background (undergraduate, graduate, master's, and doctoral degree), aged between 15 and 55 years old, and owning a Google Android OS smartphone; thus, the developed app might not reflect the opinion of those with a lower level of education, under the age of 15 or more than 55 years old, and owning a non-Google Android OS smartphone, such as iOS Apple and Blackberry. Another limitation was that the app was solely built for the Google Android operating system, indicating that other users were unable to use the app. Further studies on the development of the app with Apple iOS or other operating systems were highly required.