Development and usability testing of an educational mobile learning app for climate change and health impacts

Introduction

Climate change is defined as “a change of climate that is attributed directly or indirectly to human activity, that alters the composition of the global atmosphere, and that is in addition to natural climate variability over comparable time periods”. Climate change is the greatest challenge of the 21st century, threatening all aspects of the society, and increases risks to human lives and health [1]. The recent and future impacts both on ecological resources and human well-being/impacts have been reviewed extensively [2]. The severity of direct and indirect impacts on human and animal health (One Health, WHO) is increasingly clear, with expected higher heat-related morbidity and mortality, necessitating environmental education. Changing weather patterns increase the risk of many infectious diseases worldwide, e.g., the coronavirus (COVID-19) pandemic, and other newly-circulating ones (HIV/AIDS, hantavirus, hepatitis C, SARS etc). This reflects the combined impacts of rapid demographic, environmental, social, technological and other changes in our ways-of-living. Climate change will also affect infectious disease occurrence. Changes in infectious disease transmission patterns are a likely major consequence of climate change. Air pollution due to greenhouse gas emission, increases the risk of COVID-19 spreading faster and becoming deadlier.

However, due to multidisciplinary nature, complexity of environmental, societal, economic, mitigation/adaptation phenomena; Climate Change education confronts with many challenges [3] such as significant differences in the learner’s level of understanding [4]. Both citizens and governments are invited to focus and participate [5], definitely a collective solution where responsible civic participation is required for mitigation, adaptation and education. Quality environmental education by lifelong learning is the key component of adaptive capacity, the knowledge and skills needed to adapt lives and livelihoods to the ecological, social, and economic realities. For education to be transformative, it must be based on active, inclusive, and participatory learning and teaching process [6].

New opportunities offered by information and communication technologies shift learning and teaching process from traditional environments to online environments, whether optional or compulsory (as in the COVID-19 pandemic process) and distance education activities are becoming widespread. Recent coronavirus crisis (COVID-19) reemphasized the importance of open education and “opened up new doors” for several online platforms, which have changed over the past 20 years with many phases of development: “Online Open Course 2.0” debates still continue [7], [8], [9], emphasis should be placed on lifelong learning [10].

Distance education or distance learning delivers education to students who are not physically “on-site” to receive their education. Learning Management System (LMS) provides learning environment through a specific software [11]. Open-source learning management systems (e.g., A Tutor; Claroline; EBA; e-Study; EduZone; Moodle; Sakai) are continually evolving to meet the demands of learners and teachers [12]. Before the 2020 pandemic, open source learning management systems already actively supported formal education with blended learning approach in secondary and higher education and assisted distance education. It provides many opportunities and facilities to the distance education users: Developing the knowledge, skills and abilities of the learners, helping his/her personal development, choosing the learning style suitable for the learners, reducing transportation costs and other expenses, providing equality of opportunities in education to people especially with poor socio-economic status, opportunity to learn at the learner’s own learning speed, independent and lifelong learning opportunity [13], [14], [15]. Courses requiring physical on-site presence such as exams are hybrid or blended course or program [16]. Flipped learning, a blended learning approach, brings together advantages of face to face teaching and technology enriched online learning environments. In the flipped classroom, course material is presented to the students prior to class via online LMS, then during classtime learning-centered activities build up on the preclass work, accomodating opportunities for individualized education [17]. Furthermore, quality of in class and out of class time is promoted by combining technology, education and science. Diverse audiovisual materials are made available and students are canalized to different activities. Each day, new teaching-learning approaches, models, methods, and different practices appear in the education domain and these practices are becoming more popular in different learning levels.

Mobile apps and social media facilitate access to new generation platforms; synchronously changing Internet user behavior, creating a new era of education in a connected world: digital learning, data rich environments, computer-based assessment, time and location independent, enjoyment during learning, direct access to information; learners can communicate asynchronously, interact and share multimedia contents such as texts, images, videos or animations [18], [19], [20], [21]. Smart mobile devices have emerged as an important tool for self-paced learning. Mobile education is defined as the use of mobile devices to allow learning without time and space restrictions, either alone or in conjunction with other information and communication technologies (ICT), expand access to knowledge [22]. In addition, they encourage learner creativity, problem solving [23]. During the last decade research focused on potential use of mobile applications for education employing mobile augmented reality (MAR) application and similar tools [24], [25], [26]. Impact of mobile applications as a virtual learning platform on students’ comprehension, motivation, and learning satisfaction were reported [27], [28], [29], [30], [31], [32].

Interest in the usability principles of social media technology has increased in general. Moreover, many articles are associated with the health and medical area, suggesting a more mature development of these fields [33]. Advances in smartphone technology are to help reinforce users skills in the health sector. There are various apps focusing on healthcare education: to memorize especially in pharmacology (MedCalX), to help reinforce users diagnosing and threating skills (Prognosis), to provide a vivid and comprehensive look of the human body (Muscle and Bone Anatomy 3D), the flashcard system that questions users on topics users are weak at more than the topics users know very well (Anki) etc. [34]. Similarly, there are some mobile apps to increase awareness about climate change and effects; to control energy use (EnergySmart), current weather forecast in users location to a massive database of weather history (Vodoo Skies Normal or Not), an audiovisual journey through the stunning arctic findings (Chasing Ice), a platform to measure carbon dioxide emissions during everyday commuter activity (Commute Greener), energy map alert (AllertMe Energy Map), pollution levels in users area with an interactive map (Pollution), companies are ranked on how well users address climate change (Climate Counts), to find energy saving tips to protect oceans from climate change (PowerPup). These apps are available on all platforms (iOS, Android, Windows, and MacOSX) [35]. Sullivan et al. [36] reviewed existing smartphone apps to estimate personal carbon emissions/cost and potential health consequences of these actions. Wallace and Bodzin [37] reported a significant increase in science and technology interest with students who used the MobiLAP approach for citizen-science to familiarize them with Climate Change.

Akay et al. [38] reported that the majority of the participants (83.5%) declared training need about Climate Change; knowledge about Climate Change were 41.7%, without 5.1%, partially knowledgeable 53.8%. Participants who had training or any course/seminar concerning health impacts were 8.2%, not trained 91.8%. It is apparent that this vital topic is vastly misunderstood, and steps must be taken to educate those who may not recognize the impacts, conceive Climate Change as a social determinant of health (SDOH) [39]. A survey of literature indicated need for training by life science and one health professionals. Citizens to understand how they contribute to anthropogenic Climate Change and actions that can be taken to mitigate human induced causes. A more user-centered and holistic approach with the target group involvement in developing and testing the usability and usefulness of an app is required [40]. The aim of the study is to develop a training Mobile app on climate–health integrated into all aspects. For this reason, mobile app has been developed and tested the usability and satisfaction of the users. Our hypothesis that Mobile app increases users’ awareness level of Climate Change is tested. While developing the Mobile app, the app is entertaining and informative, suitable for self-paced learning, its visuals are remarkable, and users can evaluate themselves with tests taken during the learning process.

Materials and methods

Participants were surveyed about useability and satisfaction of the mobile app; pedagogical impact was evaluated by measuring their awareness levels in Climate Change before and after the process. Quasi-experimental design, a quantitative research method, was used in five phases: Mobile app design and development, scale development and final decision on the scales to be used, implementation, data collection, and analysis.

Experiment design

Mobile app design and development process

Initial stage

Mobile app development process started with an examination of sample mobile applications freely downloadable via the Google Play and the App Store, accessed with keywords “iklim değişikliği” and “climate change” and scanned during 1–8 September 2019. Four most rich in content, addressing both Climate Change and health impacts, and user-friendly Mobile apps were analyzed: Üretirken Koru [Protect While Producing]; Facts of Global Warming; İklim Değişikliği Muğla [Climate Change Muğla]; NASA Visualization Explorer. Our Mobile app had some similarities between the scanned applications above in terms of aesthetichs, hierarchy, interface, system info. However, ours addresses health impact, energy efficiency, mitigation, and personal measures to combat climate change.

Stage 2

Theoretical content was enriched using the analyzed Mobile apps and reviewed literature on Climate Change and health effects: Clean energy sources, history of Climate Change, global and country situation of Turkey, greenhouse effect, greenhouse gases (GHGs), climate change impacts and renewable energy sources and solutions for individuals to mitigate Climate Change. The texts were examined and necessary corrections were made by specialists (Professor of Biology and Instructor of Geography Education).

Learning objectives of climate change awareness Mobile app are given below:

After using this Mobile app, participants will be able to:

1. Describe the greenhouse effect, global warming and climate change.

2. Explain the concept of climate change.

3. Identify causes of climate change based on scientific evidence.

4. Analyze impacts of climate change on health.

5. Compare adaptation and mitigation strategies.

6. Recognize and internalize ways of lowering climate change impacts at personal and societal levels.

7. Consider international collaboration and action against climate change.

Stage 3

Infographics using the canva.com/tr_tr/ [41] were prepared to release the comprehensive content of Mobile app to users.

Stage 4

Google Forms^® preliminary tests were prepared and directly accessed from the Mobile app content with a link to measure awareness levels and self-valuation. Three-question “mini tests” on mobile application content were at the end of each section. A “Let’s test ourselves” section of 16 questions is accessible from the Mobile app main menu (Figure 1).

Figure 1:

Mini tests used in the mobile application to measure awareness levels and self-valuation of respondents.

Three-question “mini tests“ on mobile application content were at the end of each section. The “Let’s test ourselves“ section of 16 questions is accessible from the mobile app main menu, accommodating self-evaluation while answering the mobile App.

Infographics and mini tests were appraised by field specialists (a biology professor, a geology professor, a biology associate professor, a communication associate professor) before release in Mobile app. The screenshots of the prepared infographics are shown in Figure 2.

Figure 2:

Mobile phone screenshots with the Andorid^® operating system of the MA.

The screenshots present infographics for successful learning using images.

Stage 5

Using the mobiroller.com/tr site [42], a mobile app for use on Android^® operating system mobile devices (mobile phone, tablet etc.) was developed. The operating system was preferred since most operating systems are in this form, are mostly open source and enable interventions.

The size of mobile application is 18 MB and downloadable with file “744046463496.apk” sent to the users via Google Drive^® (https://drive.google.com/open?id=1hEMQwchDgQ7MOmhjAUyDjJRz8CCOAj8J). An “application download directive” has been also sent to users, for easy download.

Data collection tools

Global climate change awareness test (GCCAT)

Mobile app content is depicted above. A draft test of 34 items/questions, consisting of MA’s four sections, has been prepared. Content validity was reviewed by three field experts (two biology, one geographer, and educational expert) and the V1 was revised; the new version was applied to 107 respondents including biology teachers. The awareness test was developed with participants/respondents (including biology teachers) who had prior exposure to the field. After application of the draft test, item analysis was carried out by calculating the difficulty and distinctiveness of the items, validity and reliability of the survey performed. Inappropriate items were excluded, KR-20 reliability coefficient calculated, and the final form of the test was ready.

Item analysis results of the awareness test are given below in Table 1.

Table 1:

Items’ difficulty and distinctiveness indexes.

Question number	Item variance	p	q	Expression, p	r	Expression, r	Decision
1	0.14	0.83	0.17	Moderate	0.40	Good	Used
2	0.08	0.92	0.08	Easy	0.41	Good	Used
3	0.06	0.93	0.07	Easy	0.21	Fair	Used
4	0.25	0.51	0.49	Moderate	0.14	Fair	Not used
5	0.01	0.93	0.07	Easy	−0.03	Poor	Not used
6	0.13	0.85	0.15	Easy	−0.01	Poor	Not used
7	0.25	0.48	0.52	Moderate	0.22	Fair	Used
8	0.24	0.58	0.42	Moderate	0.18	Fair	Not used
9	0.15	0.82	0.18	Moderate	0.05	Poor	Not used
10	0.15	0.19	0.81	Hard	−0.05	Poor	Not used
11	0.14	0.17	0.83	Hard	0.18	Fair	Used
12	0.17	0.79	0.21	Moderate	0.04	Poor	Not used
13	0.25	0.49	0.51	Hard	0.15	Fair	Not used
14	0.25	0.53	0.47	Moderate	−0.03	Poor	Not used
15	0.04	0.96	0.04	Easy	0.32	Good	Used
16	0.23	0.36	0.64	Hard	0.19	Fair	Not used
17	0.12	0.86	0.14	Easy	0.25	Fair	Not used
18	0.15	0.81	0.19	Moderate	0.24	Fair	Not used
19	0.06	0.93	0.07	Easy	0.43	Good	Used
20	0.17	0.78	0.22	Moderate	0.15	Fair	Not used
21	0.04	0.96	0.04	Easy	0.48	Good	Used
22	0.05	0.94	0.06	Easy	0.35	Good	Not used
23	0.16	0.80	0.20	Moderate	0.15	Fair	Not used
24	0.03	0.97	0.03	Easy	0.48	Good	Used
25	0.11	0.88	0.12	Easy	0.29	Fair	Used
26	0.04	0.96	0.03	Easy	0.57	Good	Used
27	0.13	0.84	0.16	Moderate	0.22	Fair	Used
28	0.02	0.98	0.02	Easy	0.46	Good	Used
29	0.18	0.77	0.23	Moderate	0.45	Good	Not used
30	0.15	0.19	0.81	Hard	0.07	Poor	Not used
31	0.15	0.81	0.19	Moderate	0.23	Fair	Used

1. p: difficulty index, q=1−p, r: distinctiveness index.

Item selection based on expert opinions finalized selection by taking, subject content, item difficulty index and item distinctiveness into consideration. Eight items, with distinctiveness of >0.30, (1, 2, 15, 19, 21, 24, 26 and 28) were selected for the final test. Six items, distinctiveness >0.20, (3, 7, 25, 27 and 31), and only one item (11) with distinctiveness >0.10 were revised and final version of 14 items were completed. The GCCAT is composed of 14 items, measures the extent of the basic knowledge in individual prossesses. It composed of 14 items, measures the extent of the basic knowledge an individual possesses. One point is given to a correct answer item, and the highest score is 14 points. Average item difficulty was 0.83, concluding the test as moderately difficult. Average item distinctiveness strength of test was good =0.35. KR-20 reliability coefficient was acceptable: 0.71 [43].

Mobile app usability and satisfaction scales

Two mobile application usability scales were used: Mobile Application Usability Scale (MAUS), developed by Hoehle, Aljafari, Venkatesh [44], tested within a structural measuring model and adapted to Turkish culture, by Güler [45]. Five sub-scales of the total 10 were used: esthetic, color, control, font, and hierarchy. The first scale consisted of total of 20 items and the 7-point Likert type scale was used for data collection. The second scale was Mobile Application Satisfaction Scale (MASS), developed by Namlı [46], a 7-point Likert scale of 15 items. It measures consumer usability directly and in user-satisfaction evaluation, usability values in human-computer interaction.

Results

Overall, the use of Mobile app in health effects of Climate Change has been shown to be effective in enhancing students’ learning experience and well-received in terms of adopting and using such technology for educational purposes. Findings of our study show a positive outcome of the Mobile app; it easily facilitates blended learning, a learning environment that blends online learning with classroom activities; we conclude that our Mobile app is successfully incorporated into the learning context and can be used to supplement, rather than to replace, classroom-based activities, including lectures and active learning exercises such as case studies of heatwaves and extreme weather events during in service training of life sciences and health professionals.

Mean scores of the experimental group before exploring the Mobile app was 10.82 (pre-) and 11.64 (post-) after they examined the Mobile app, a statistically significant difference between the pre-test and post-test scores (p<0.05) (Table 3). Standard deviations were 1.40 and 1.58, respectively. Mobile app significantly increased the participants’ awareness level of global climate change.

After Mobile app was examined by the participants, they were asked to evaluate the Mobile app according to different criteria such as esthetic, color, control etc. The results of Mobile app usability scores are given below (Table 4).

The MAUS results showed that, in general the participants’ views on Mobile app are “usable ( X ‾ =5.87)” (1.00–1.86=strongly unusable, 1.87–2.72= unusable, 2.73–3.58= somewhat unusable, 3.59–4.44= neutral, 4.45–5.30= somewhat usable, 5.31–6.16= usable, 6.17–7.00= strongly usable). The views of the participants about Mobile app’s esthetic ( X ‾ =5.51), color (5.57), control ( X ‾ =6.16), font ( X ‾ =6.05), and hierarchy ( X ‾ =6.05) are usable too. The highest score is the control feature of the Mobile app and the lowest score is the esthetic feature of the Mobile app (Table 5).

Participants’ views about their satisfaction with the Mobile app, MASS results, showed satisfaction “satisfied ( X ‾ =5.83)” (1.00–1.86= strongly not satisfied, 1.87–2.72= not satisfied, 2.73–3.58= somewhat not satisfied, 3.59–4.44= neutral, 4.45–5.30= somewhat satisfied, 5.31–6.16= satisfied, 6.17–7.00= strongly satisfied). In general participants strongly satisfied items results are “the Mobile app is suitable for its purpose ( X ‾ =6.30)”, “the Mobile app is not confusing ( X ‾ =6.45)”, the Mobile app is not very complicated ( X ‾ =6.50)”, and “the controlling Mobile app is easy ( X ‾ =6.41)”. The participants view about the Mobile app needs to be further developed is “neutral ( X ‾ =3.84)”. Likewise, the participants’ views about to concentration/focus to use the Mobile app is “neutral ( X ‾ =4.25)”. In conclusion, the Mobile app is user friendly.

The reliability values of GCCA pre-test and post-test and MAUS and MASS are given below (Table 6).

A reliability coefficient of 0.70 or higher is considered “minimally acceptable” in social sciences [47]. GCCA pre-test (α=0.72) and post-test (α=0.78) reliability scores were higher than 0.70, showing respectable mean scores. The MAUS reliability scores were calculated 0.97 and subscales of esthetic, color, control, font, and hierarchy reliability scores were calculated 0.97, 0.82, 0.95, 0.97, and 0.99, respectively. Usability of reliability scale and subscales are excellent/appropriate, the only exception is the very good reliability of color. The MASS reliability score was calculated 0.85 and mean satisfaction of the scales is very good. Infographics enable more understandable presentation of the data/information, simplifying complex and intensive information by visualization.

Discussion

One of the primary focus of modern citizen science is examining how Climate Change affects life on our planet. Global weather- and Climate Change related disasters of devastating fires (devastating wildfires continue to burn in Australia), floods, and landslides have led to loss of life and property and irreversible ecosystem changes; emphasizing serious physical and mental health consequences. Changing climate impacts contend longer and hotter summers, more frequent and intense storms, sea-level rise, more severe droughts, and poorer air quality, are inextricably linked to poorer health. Similar to mental health problems, child health impacts are numerous and include worsening asthma and allergies; physical trauma from disasters; mental health symptoms, including posttraumatic stress disorder after disasters and anxiety about the future; increased exposure to infectious diseases; and lack of access to adequate food and clean water [39].

Increased interest in Climate Change education and effective education strategies/approaches have been reviewed [48]. Skavanis et al. [49] described mobile app “Climapp” to promote climate change communication to the general public. However, assessment of the application in terms of effectiveness in efficient communication of climate change to the general public was not measured. Effectiveness of the developed Mobile app on climate change and health effects in improving the learning experience of the participants and the positive effects of adopting and using this technology for educational purposes in the Results section. In addition, the data of the study show that the mobile application increases the awareness and learning of the participants. Research has accentuated challenges associated with the complex nature of Climate Change and some educators allude to lack of necessary skills and knowledge to adequately deliver instruction. Here mobile learning can be an alternative since it enables individuals to adjust the education to their needs, expertise, and skills. Similar to our findings, Teri et al. [20] developed and applied a Mobile app (NutriBiochem) in a second year Biochemistry and Metabolism course for biochemistry and nutrition education. Majority of students agreed the Mobile app to be a useful learning tool; others felt it helped them perform better in the course. The study tool was especially effective for students who were comfortable with technology, and access it regularly. Peart et al. [50] introduced Bioscience and Biochemistry students to Mobile app which measured resting heart rate in a research skills module. They concluded that progress from passive classroom learning to more active inquiry-based learning and finally an example of independent problem-based learning, with the aim to encourage critical analysis in bioscience students was achieved. Cell biology teaching is often considered difficult, both by students and by teachers due to difficulty in creating mental models and the scarce resources used by teachers in the didactic transposition makes the process of teaching and learning hard. Cell biology apps were reviewed as potential support for classroom lectures. Realistic models and animations were found useful for exploring the dynamics of the cell functioning [51]. Study of Etcuban and Pantinople [52] determined the effects of using a Mobile app in teaching mathematics among grade 8 students in high school, in the Philippines. The researchers used the quasi-experimental method of research using the pre- and post-test design with two participant groups with 40 students per group and reported an increase in the post-test scores of the control and experimental groups. Use of the Mobile app helped enhance students’ achievement and learning in teaching mathematics.

The Mobile app as a tool for blended learning may first present to the students via mail, WhatsApp etc. Class time is then may use for learning activities that built on preclass work. So, the transfer of knowledge is shifted outside the classroom, social and active learning activities are carried out in the classroom. Educators can increase the quality of in class and out of class time by combining technology. Our results showed participant satisfaction with the Mobile app use, and they like to access information anytime and anywhere. Similarly, student achievements, motivations and attitudes towards learning changed with the support of instructional design with technology [53] and technological tools used by teachers in their lessons positively affected teaching [54]. Results of previous studies agree and support our research results. Traditional online courses are replaced by different income models, as communication between students and lecturers increase and lead to further development of these free and open education models. Besides, mobile communication tools specific to personal use have become an important technological tool in accessing such new generation of open education platforms. In this sense, the development of educational Mobile app and their free access has an important role on the creation of new generation of Open Course materials and eventually induce the rapid and widespread awareness of critical issues such as Climate Change. Another interesting approach to Climate Change and adaptation was collecting and analyzing mobile phone data for the analysis of urban population activities and mobility patterns at high spatial and temporal resolutions. Technologies such as mobile phone data are a main asset for addressing climate-related risk within cities, better addressing the vulnerability of urban communities and managing climate risks in a continuous manner, thereby overcoming the limitations of static planning approaches based on population estimations from census data [55].

The Mobile app can be effectively used for teaching/training mental health providers and their colleagues on Climate Change, disaster preparedness, and how to best help the patients cope with these highly stressful situations [56]. Increasingly need arises for inclusion of Climate Change health impacts in development of curricula in allied life and health disciplines as lifelong learning, in close collaboration with the multidisciplinary specialty experts. The medical and One Heath community needs to collaborate with climate scientists, clean energy leaders, community-based organizations, and behavioral health specialists with expertise in environmental-related health issues. Way further is to resolve potential of smartphone apps to evoke behavior change resulting in Climate Change and health co-benefits [36].

Mobile apps as essential element of blended learning environments are subject of usability testing considering different factors: learnability, efficiency, memorability, errors, satisfaction effectiveness, cognitive workload, interruptibility and simplicity [57]. Along this line, future research can focus on describing pedagogical impact of mobile applications in biochemistry and/or biochemistry laboratory education.

Today’s learners expect an education just in time, just enough and just for them and m-learning is a learning model that corresponds with these demands. With these features, m-learning emerges as a powerful dynamic of lifelong learning and allows the learning process to be realized at every moment of life without interruption [58]. Another advantage of m-learning is the ability to install apps according to the individual’s learning requirements and correspond with the learning requirements [59]. There are millions of free applications on mobile platforms that can support the need for learning or can be used as an auxiliary tool. The ability to download and use the applications needed according to the learning needs on mobile devices allows the mobile devices to be employed as a personalized learning environment that allows learning to continue without interruption. The Mobile app developed in the present study can be revised and modified according to blended learning needs and therefore very flexible tool to support self-training, especially during extraordinary significant threats such as the COVID-19 pandemic.