Mobile learning in a ﬂ ipped classroom: ﬁ ndings from a “ 5-lecture-5 ” blended learning design for large classes Ters

Objective: We aimed to develop and evaluate a mobile learning (m-learning) design for large group application learning based on flipped classroom approach within an integrated curriculum of medical faculty. Materials and methods: Totally 359 students enrolled in a two-credit course during the 2017 – 2018 academic year. An instructional design called 5-Lecture-5 (5L5) was implemented via mobile application. This design is a blend of m-learning and flipped classroom approaches, with pre-, during, and post-lecture elements. Data were analysed using Chi-square, Mann – Whitney U, and Kruskal – Wallis tests. Open-ended questions and course observations were analysed thematically and reported together. Results: Of the participating students, 26.96% who saw the first announcement examined pre-lecture materials and 55.43% attended face-to-face lecture. Achievement test scores of those who attended lecture were significantly higher (p = 0.027). Correct response rates for each question were 99.30, 72.10, 58.70, and 94.40%, respectively. Consequently, students demonstrated positive attitude towards m-learning ( M = 170.15 ± 28.48). The most preferred positive aspects were “ active learning, ” “ interactive, ” “ entertaining, ” “ useful, ” and “ repeatable. ” Conclusions: M-learning activities within a flipped classroom with instant feedback were found to increase participation and interaction levels of students in classroom setting, resulting in their increased enjoyment and engagement.


Introduction
Mobile phone usage, and smartphones in particular, has dramatically increased over the past decade, so much so that it can negatively affect educational learning environments, resulting in issues such as distracted, passive, and unengaged students, especially in the large group learning classroom environment.Involving larger cohorts of students in courses brings many challenges to both faculty teaching staff and university administrators.Educators have used active learning strategies to overcome some of these challenges [1].However, having larger groups of learners can make it even harder to deal with such consequential situations.The flipped classroom is one of the approaches suggested in recent years to aid the transitional shift from the traditional classroom teaching environment to one that involves a higher degree of active learning.
Learning in larger classrooms due to an increased population and lack of attention to lecture-based teaching has become a significant issue [1].In the larger class, students can easily lose attention with regard to what is being taught due to numerous other distractions such as smartphones, peer student disruptions, and general classroom noise level.Thus, capturing and maintaining student attention during a full lecture becomes even harder.
In most integrated curricula, lectures to be attended and followed are intensively and sequentially structured [2].Following up these lessons during the day as well as preparing for courses in advance requires considerable time and effort.Having students already prepared for their lessons in advance increases their level of participation and interaction.However, courses can be negatively affected due to students' excessive course load which can prevent students attending classes pre-prepared [3], as well as causing students' cognitive overload [4].Medical students tend to spend more time selfstudying than other faculty's students, in addition to attending face-to-face lectures.Effective learning strategies are therefore crucial in order to enhance the success and participation of medical education students [5].In order to avoid such issues in knowledge transfer, instructors need to design their lectures with added care.However, today's learners expect a more interactive learning environment due to the numerous distractions that can occur during today's traditional face-to-face lectures [6].
Educational technology seeks to provide solutions to these issues through various approaches such as Internet technology-based e-learning [7], blended learning [8], mobile learning (m-learning) [9], and the flipped classroom [10].Successful results can be achieved through the application of these methods in lessons designed according to established instructional design processes.Additionally, the frequent practising of learnt skills is of vital importance to effectively deliver the required information within a limited time period, whilst ensuring sufficient levels of knowledge retention [11].
The widespread use of mobile technologies and handheld devices (i.e. smartphones and tablet, personal computers) provides opportunities for the application of novel teaching and educational methods for larger teaching groups in higher education.Moreover, the features of these technologies contribute a significant level of freedom through the provision of instant feedback and just-in-time learning, whereby learners can access instruction at any time and from any place.Based on literature regarding active learning strategies applied to large classroom lectures, the current study developed an instructional design called "5-Lecture-5" (5L5) which blends m-learning and the flipped classroom approach in three phases: "pre-lecture," "during lecture," and "post-lecture."The study includes various online activities, as well as pre-and post-tests, and students informed in advance about the topics covered by the lectures.
Therefore, the purpose of the current study is to design, develop, implement, and evaluate an m-learning design based on the flipped classroom approach for a large group teaching and learning scenario within an integrated medical faculty curriculum.With this model, the effective transfer of course content is aimed to be achieved, with a corresponding increase in learning levels, active participation of students, and persistence of learnt knowledge retention.

Methods Sample
The current study was conducted within an "Assisted Reproductive Techniques Laboratory" course, which is part of the Urogenital System internship programme at Ege University's Faculty of Medicine, in Turkey, during the 2017-2018 academic year.The sample of the study was students who enrolled in the course and volunteered to participate in the study.A total of 359 medical undergraduate students enrolled were expected to attend face-to-face lectures.However, due to non-attendance, a lower participation rate of 55.43% was realized for the study.Therefore, the actual study sample consisted of 199 students who attended face-to-face lectures.Since all students in the sample could easily be reached by the researcher, no sampling method was applied.Ethics committee approval was received from Ege University's Scientific Research and Publication Ethics Health Sciences Board prior to the initiation of the study (Protocol No.: 331-2017).
Educational context and instructional design process "Assisted Reproductive Techniques Laboratory" is a two-credit course provided by the Department of Histology and Embryology, included within the context of vertical integration in the Urogenital System internship programme of Ege University's Faculty of Medicine.The course takes place over five sessions due to the rotation of the programme.The outcomes of the course are designed to include teaching objectives at the required level of knowledge and comprehension for undergraduate students and to create an appropriate level of professional awareness for clinical application following their graduation [12].However, this compulsory course includes complex topics such as the legally defined rights of reproduction, as well as advanced methods for eliminating disruptions as a general medical practitioner working in the field.Therefore, understanding the most effective and efficient method to transfer knowledge is crucial to the education of prospective medical practitioners.
The researchers of the current study designed the 5L5 method, which consists of three phases (see Figure 1).In the first phase (pre-lecture), the course material was designed to be reviewed by students in a 5-min period and the content learnt in the information step.Material support resources were sent to the students via a mobile application 1 day prior to the scheduled face-to-face lecture via online app notification and by e-mail.In this process, the students were expected to examine pre-course materials in preparation for their subsequent lesson.
In the second phase (during lecture), the students are expected to attend a face-to-face lecture and participate in activities prepared by the instructor using a mobile application.Interactive presentation was implemented using the Moodle Learning Management System (LMS) [13], and students replied to questions via the mobile application.The content of the face-to-face lecture was adapted according to the students' levels of interaction and knowledge.
Figure 2 shows a screenshot of student responses to questions posed during the face-to-face lecture.Based on the students' responses, the instructor readjusted the subsequent lesson content and presentations for further discussion on the topic.The mobile application's activities were designed based on the clicker method in order that class success rates could be instantly evaluated.
Four questions were designed to evaluate four learning goals considered crucial to the course.Questions answered during the face-toface lecture were then immediately discussed with the instructor.Where questions received high correct response rates, the corresponding part of the lesson was passed through quickly with only a brief mention; else, the topic was explained and elaborated upon further by the instructor.In the third phase (post-lecture), at the end of the lesson, the instructor assigned post-lecture activities in the form of 5-min interactions in which students were expected to complete simple activities and achievement tests.In addition, the students were then asked to complete a "Student Information Form" and "Attitude Scale Towards Mobile Learning" as data collection tools.All online activities were monitored and logged on LMS.
The researchers informed the students via LMS announcements, e-mail, mobile app notification, and through face-to-face lectures prior to the flipped classroom activity.The LMS logs were examined in order to ensure adequate numbers of students were informed about reviewing online material prior to their face-to-face lectures.Through this method, the researchers were able to notify the students on several occasions.

Instruments
Student information form: The questionnaire was prepared by the researchers with a total of 11 questions.Five of the questions were based on demographic data, while the remaining questions sought the students' opinions with regard to the 5L5 design.
Attitude scale towards mobile learning: The scale was previously developed by Demir and Akpınar [14] and consists of 45 items (including seven reverse-scored) and was designed as a 5-point, Likert-type scale ("5-Totally Agree," "4-Agree," "3-Partially Agree," "2-Disagree," and "1-Totally Disagree").The Cronbach's alpha coefficient was computed by the scale's authors as 0.950 [14].Scores ranged from 73 to 197 (where a higher number equates to higher achievement).The four factors of the scale are as follows: "Satisfaction level towards mobile learning" (F1), "Effect of mobile learning on learning" (F2), "Representatives of the motivation towards mobile learning" (F3), and "Representatives of the usefulness of the mobile learning" (F4).The Cronbach's alpha coefficient of the scale for the current study was 0.964, which indicates that the scale is considered highly reliable [15].

Course observation form:
The instrument was developed by the researchers of the current study and used by the experts during the course to observe the function of the teaching method.
Achievement test: The test was created by the course instructor in order to evaluate the students' knowledge and achievements.The test consists of five multiple-choice questions, each with five answer options interrelated to the course content.
Online system records: Data such as time of entry, students' task completion status, and their duration in the system were retained by the LMS.Records were subsequently examined throughout the process in order to evaluate the students' course attendance.

Statistical analysis
Frequency tables and descriptive statistics were calculated.Chi-square test was used to examine the relationship between the categorical variables.Shapiro-Wilk normality test was used to determine whether or not continuous data were distributed normally.As the continuous data such as attitude scale total score and achievement test score were not normally distributed, the Mann-Whitney U and Kruskal-Wallis tests were then used in order to establish whether or not the comparisons between the groups were statistically significant at a significance level of 0.05.Statistical analyses were performed using SPSS Statistics for Windows, Version 25.0 (IBM SPSS Statistics for Windows, Version 25.0.Armonk, NY: IBM Corp) [16].Additionally, open-ended questions and course observations were analysed thematically and reported together.

Findings related to 5L5 instructional design
Pre-lecture Of the 359 students, 199 (55.43%) attended to the face-toface training.According to LMS records, 72 (26.96%) of 267 students who entered the LMS after the first announcement examined the material prior to the face-to-face classroom (see Table 1).

During lecture
The number of students participating in the activities upon which the four questions were based, according to student interaction data via mobile devices were 149, 154, 150, and 143, respectively, of the total 199 students attending face-toface lectures.During the lectures, questions were asked via the interactive environment utilizing mobile tools.Results were instantly shared with the students during the face-toface lecture.The rate of correct responses for each question was 99.30% (148 students), 72.10% (111 students), 58.70% (88 students), and 94.40% (135 students), respectively.Of the 199 students, 32.30% (54 students) responded correctly to all questions.There was no statistically significant difference found between the scores of the students who answered all questions correctly and those who did not (p = 0.679).Although the scores of the students who responded correctly to all questions were higher than the scores of those who did not, no statistically significant difference was found between the two groups (median full: 4.55, incomplete: 2.73, p = 0.437).

Post-lecture
Among the face-to-face lecture attendants, some volunteered to complete the post-lecture activities, which included "Achievement Test," "Student Information Form," and "Attitude Scale Towards m-Learning." The student information form was completed by 30 students (of which 16 were female; 53.30%).The median age was 22 years.All students possessed a smartphone, and 96.70% (29 students) had a median of 4 GB mobile Internet package.Additionally, 40% (12 students) had a tablet pc, and 73.30% (22 students) had a laptop pc.
When the students were asked about their computer use knowledge level, 70% (21 students) described their level to be good or above.The computer use knowledge level of male students was significantly higher than that of the female students (good or above; female: 50.00%, male: 92.90%, p = 0.017).Of the students, a total of 60% (18 students) stated that they would like to take some courses within the 5L5 instructional design environment and some without.Regarding the 5L5 method of instruction, 86.7% (26 students) considered that the course was suitable for their education using the 5L5 instructional design environment, and 83.30% (25 students) stated being satisfied with the course being applied in the 5L5 environment.For the pre-class activities, 53.30% (16 students) were satisfied at having participated in the pre-class activities using the interactive m-learning environment; 86.70% (26 students) were satisfied with the activities during the face-to-face lecture; and 73.30% (22 students) were satisfied with the post-lecture activities.
In the post-lecture achievement test, 45 (49.50%) of the 91 students who completed the test had a median score of 75.50.Although the students who responded correctly to all four questions asked during the face-to-face lecture were more successful than those who did not, there was no statistically significant difference between the achievement test scores of the two groups (median full: 90.00, incomplete: 76.00, p = 0.148).Similarly, although the students who examined the pre-lecture material had higher achievement test scores than those who did not, this difference was not found to be statistically significant (median examined: 80.75, not examined: 73.00, p = 0.394).However, the achievement test scores of the students who participated in the face-to-face lecture were found to be significantly higher than those who did not (median participant: 77.50, nonparticipant: 40.75, p = 0.027).

Attitudes and opinions of students on 5L5 instructional design
Of the students who responded to the "Attitude Scale Towards m-Learning," 33 were found to have positive attitudes (M = 170.15± 28.48; see Table 2).There were no statistically significant differences between the total attitude scale scores of the students who answered the all four questions correctly and those who did not (median full: 171.00, incomplete: 172.00, p = 0.644).
According to the observation forms which were completed simultaneously by two independent researchers, there were no issues noted except for Internet connection problems.The students (73.33%, n = 22) summarized their most preferred positive aspects of the 5L5 design as active learning, interactive, entertaining, useful, and repeatable.One student highlighted the positive aspects of the 5L5 design as "It was good in terms of active participation.It is also useful to have questions that led to the analysis and use of the information we learnt."Another student highlighted the instructional design key principles as drill and practise, stating: The pre-course practical preparation and the mini quizzes, which allow us to repeat the courses that are followed after the lesson, have allowed us to repeat more than once, and considering the medical education, both clinical and basic science courses are suitable to be followed and repeated online.
One student emphasized that they were able to participate in the face-to-face lecture interactively and visually as, "The videos and quizzes that can be visualized enable us to participate actively and not be distracted during the faceto-face lecture.In addition, the site allows for easy access to repeat the lesson at another time."Moreover, in the classroom observers' notes, analysis showed similar results in regard to the students' views in terms of interaction and active participation.
On the negative side, the students identified technological and Internet problems due to the use of mobile phones.Additionally, the novelty of the method was found to be a bit difficult to adapt, with one student mentioning that, "I had difficulty because I've never experienced this kind of lesson before.Yet, if it's more common, everyone can get used to it, soon."Although recurrent announcements were made, one student mentioned issues regarding the lecture activities, stating, "We can forget to do the preand post-lecture activities, but I liked to use it [the app] during the course."Moreover, occasional Wi-Fi connection problems occurred during the course observations.
In order to improve the course quality through the design of the 5L5 environment, suggestions were made with regard to increasing the number of questions asked during the course, strengthening the signal or increase the speed of the university Wi-Fi "eduroam" connection, and adding visual materials such as videos to courses.It was also emphasized that the technological readiness of lecturers is paramount as a factor for course success.

Discussion
Instructional design based on the m-learning and flipped classroom approaches was applied to an "Assisted Reproductive Techniques Laboratory" course in three phases.According to the results, students were able to attend the lesson content at pre-, during, and post-lecture phases.The results indicated that the students' interaction was higher in face-to-face lectures.Although the participation in face-to-face lectures was expected to be much higher, in reality, almost half of the students did not attend the face-to-face lectures as attendance was not mandatory.
In the post-lecture activities, students reviewed the course content and took a short quiz, and high levels of achievement were observed amongst those who took the quiz.
For the first phase (pre-lecture), the students' participation in the pre-lecture activities was found to be low.This issue had also been reported previously as a challenge for the flipped classroom approach [17].A similar trend was observed in the current study, although various notification channels were used to remind students and to encourage their participation.This result can be considered as an indication of the students' obligations towards a course such as attendance or grading.On the other hand, there were students who did not participate in any activities before or during face-to-face lectures, but completed the post-lecture activities.Since the lesson content was open to all enrolled students, having been uploaded to the LMS, those students who did not attend the face-to-face lecture were able to review the course content online.This approach provided the students with the opportunity to study the lesson asynchronously if they missed or were unable to attend the lesson.
Student interaction with lessons is a key indicator of their success.In the design of the current study, a comparison was made of the pre-, during, and post-lecture activities, and it was found that students who interacted the most achieved better results in the post-lecture quiz.Similar studies, not only in faculties of medicine, have indicated that the flipped classroom approach significantly improved knowledge acquisition, demonstrated greater effectiveness with students more open to projects, and delivered better academic performances when compared to the traditional lecture-based teaching approach [18][19][20][21].Moreover, a comparison was also made in the current study of in-class (during lecture) activities and post-lecture quiz results.Those students who participated during the face-to-face activities scored better on the post-lecture quizzes.This result was similar to that reported in a study by Hwang, Lai, and Wang [22], where students interacted in a seamless learning experience through the use of mobile devices.Having a phone in the palm of students can be turned into an effective course teaching medium.In this context, students were found to focus better on lessons that were interactive, were better motivated in maintaining their attention, and found the interactive course to be more interesting.Therefore, mobile technology can be said to be useful as an enabler for student' active participation in the classroom.
In another study [21], there were no statistically significant differences found between the lecture-based and flipped classrooms regarding the methodology, student satisfaction, long-term learning, or time spent for exam preparation.According to the current study's results, one significant finding was that students who attended face-to-face lectures were statistically more successful at the post-lecture quizzes.Therefore, utilizing technology within instruction does not guarantee academic success without some level of instructor and peer learner social interaction such as the discussion of a problem within the classroom environment [23].Although students took courses for the first time using the 5L5 method, they managed to adapt easily since Moodle is a widely used LMS [24], as was the faculty's mobile application that was used [13].In this way, mobile devices used during face-toface lectures can be transformed into a useful instrument to be utilized within the course.
According to the attitude scale results, the students had positive attitudes towards m-learning and were found to be interested in the enrichment of course materials with videos, lectures, and questions asked after lessons.They indicated that their active participation in the course was better than for a traditional classroom lecture.Similar results were also reported with positive attitudes towards the flipped classroom in other studies [18].Similarly, in another study [25], the flipped classroom approach was found to be more satisfying when compared to lecturebased learning.
In the current study's implementation, short-term technology-related limitations regarding Internet access, and the lack of grading for the lesson could be considered as limitations of the study.This also affected student participation in activities related to the lesson in general.Among the weaknesses of the instructional design that need some further development, there was a weaker level of participation seen at the pre-lecture phase compared to the during lecture or post-lecture phases.

Conclusion
With the addition of technology to the instructional design process, the performance, participation, and learning skills of students increased in courses where educational technology was actively employed.The application of mlearning activities within a flipped classroom approach with instant feedback increased the participation and interaction levels of students in the classroom, which resulted in more a sociable and enjoyable learning environment, and improved engagement of the students.

Figure 1 :
Figure 1: Three phases of 5L5 method and data collection procedures.

Figure 2 :
Figure 2: Screenshot of students' responses for interactive question during face-to-face lecture.

Table  :
Descriptive statistics according to students' answers.

Table  :
Descriptive statistics on factor/total score of attitude scale towards m-learning.