The flipped classroom: a novel approach to physical examination skills for osteopathic medical students

Context: Medical students are faced with the challenge of synthesizing large volumes of information quickly. With the increasing accessibility of technology, a “flipped classroom” allows students to learn foundational material independently. Class time is instead devoted to in-depth skill building with instructors, promoting an active learning environment. This method of content delivery is also relevant given the current COVID-19 pandemic. Objectives: To comprehensively evaluate the benefit of adopting a flipped classroom approach in teaching physical exam skills (PES) to first-year osteopathic medical students. Methods: A cohort study was conducted comparing firstyear osteopathic students in the class of 2022 (n=201), who had taken the PES course traditionally, with the class of 2023 (n=203), who experienced the flipped classroom approach. Objective metrics such as cumulative grades, objective structural clinical examination performance (OSCE), and practical exam performance were compared using nonparametric Mann–Whitney U rank sum tests. Subjective measures such as student course evaluations were used to analyze course perceptions using independent sample t-tests assuming unequal variances. A faculty survey was administered to faculty who taught both cohorts to assess instructor attitudes toward the flipped classroom approach. Due to the COVID-19 pandemic, Spring 2020 quarter data was not included, given the transition of all classes to an online learning environment. Results: The flipped classroom approach significantly improved objective student performance for the Fall (p=0.009) andWinter (p<0.001) student cumulative grades and the History-Taking OSCE (p=0.010). Performance on Fall and Winter practical exam scores had no significant association with the flipped classroom. General student perceptions of the course remained unchanged between both cohorts. Faculty survey results from 10 faculty members showed that six (60%) faculty members preferred the traditional classroom, while four (40%) preferred the flipped classroom. Conclusions: The flipped classroom approach showed some statistically significant improvement in student PES. Further studies are needed to evaluate the benefits of a flipped classroom approach using skills-based assessments styles to measure student performance, with a focus on standardization of in-classroom groupwork.

In a 'flipped classroom' approach, pre-coursework is completed by students at home to encourage foundational instruction [1], and class time is then spent skill building to further apply concepts in greater depth. Flipped classroom models have traditionally been associated with interventions in K-12 grade levels [2]. However, with increasing costs of medical education and the growing need for medical students to synthesize large volumes of information quickly, the flipped classroom could allow medical students to learn at their own pace and utilize class time to work through complex concepts [3]. Flipped classroom models also maximize instructors' time, allowing them to provide enriched learning experiences rather than passive lectures [4][5][6], which have been shown to be ineffective for learning outcomes; active participation in class increases student performance [7,8].
For a hands on course such as Physical Exam Skills (PES), where students learn to conduct a comprehensive history and physical exam with osteopathic components, having additional class time to actively practice skills in groups with physician faculty could increase student competence in performing these skills with standardized patients on exams. The knowledge achieved in this firstyear course supports students throughout their medical careers, increasing their comfort in performing physical exams during rotations [9].
Previous research has assessed the impact of flipped classroom approaches in medical school settings. For example, a systematic review of 118 research studies [10] showed subjectively positive perceptions of flipped classrooms from students, yet the objective measures were inconsistent. Of the studies covered in that review, only nine had controlled methodology [10]. However, another research study [11] compared performance in a medical certificate writing course for fourth year undergraduate medical students who received instruction in a traditional model (n=141) with performance from students who participated in the following year's flipped classroom approach (n=142). That study [11] found that students in the flipped classroom scored significantly higher than their traditionally taught peers on a medical certificate writing assessment (mean score [SD], 10.97 [1.25] vs. 8.87 [1.70]), respectively. Using a similar model to the latter study, our prospective cohort study was designed to help address existing discrepancies in the literature. By incorporating multiple quantitative measuressuch as cumulative grades, objective structural clinical examination (OSCE) scores, and practical exam scoresto objectively analyze the impact of flipped classroom initiatives, we sought to evaluate student performance between both cohorts in several contexts. We also assessed subjective data, such as course evaluations and faculty survey responses. We hypothesized that the flipped classroom format would yield improvements in both quantitative grade outcomes and faculty and student perceptions of the course. Through comprehensive evaluations like ours, support for the flipped classroom approach can potentially be expanded in medical school settings.

Methods
This study was reviewed by the Midwestern University Institutional Review Board, which deemed it exempt. Data collection began after review and exemption.

Course descriptions
PES is a year-long weekly laboratory course that introduces first-year osteopathic medical students to skills for taking a comprehensive history and performing a full physical exam. This course parallels the Patient Symptom Presentation (PSP) course, which provides a traditional lecturebased overview of topics aligned with the PES workshop.
The class of 2022 attended PES in a traditional method, which included a total of 31 sessions of 55 min each: a 15-20 min introduction by the course director demonstrating weekly physical examination skills, followed by small group teaching with clinical faculty for the remaining 35-40 min. Session introductions from the class of 2022 were recorded and available for the class of 2023 students to view in the 2019-2020 year on Canvas (Instructure, Inc.), a web-based learning management system used for all courses at Midwestern, before attending PES. The class of 2023 attended PES in a flipped classroom format. Students started each of the 31 workshops with a five-question quiz based upon the corresponding class of 2022 recorded session introduction video and a weekly outline. The remainder of the 55 min session was spent in small groups.
Clinical faculty members rotated through the small groups during each quarter for both cohorts (2022 traditional and 2023 flipped). There were eight small groups, each with one clinical faculty member. The faculty corresponding to each small group did change, at times based on physician availability, but generally faculty rotated their small groups on a quarter-to-quarter basis. Faculty were assigned to their small group students at random. The faculty for the class of 2023 was not specifically trained in flipped classroom delivery but was made aware of the new curriculum plan of transferring to a flipped classroom education. With the flipped classroom approach, the faculty was given more time to expand on teaching PES to the class of 2023 students, but did not receive any unique or specific training that differed from the class of 2022. Faculty for both years participated in a preworkshop faculty "huddle" to ensure consistency in the teaching of PES. The length and content of these preworkshop faculty "huddles" did not change between the cohorts. One author (B.P.) served as faculty for both the class of 2022 and the class of 2023.

Sample
Demographic information for both cohorts was recorded by Midwestern University's Dean's Office upon Fall enrollment. Data was used for every student who completed a given assessment.

Objective measures: examinations
Items such as Fall and Winter cumulative grades, history-taking OSCE scores, and Fall and Winter practical exam scores were used to compare student performance between the class of 2022 and class of 2023 cohorts. The curriculum was kept standard for both cohorts, with the same topics and skills being covered in the same weeks of each quarter. The methods of assessment and grading rubrics for the practical exams and OSCEs were mostly consistent between both cohorts, with the addition of the weekly 5-point quiz for the class of 2023. Full physical OSCE scores, Spring quarter practical exams, and Spring cumulative grades were removed from the analysis due to the class of 2023 experiencing instruction off-campus due to coronavirus 2019 (COVID-19) online learning modifications.

Subjective measures: students and faculty
Each student cohort received a PES course evaluation from Midwestern University after course completion. Surveys were accessible on each student's Midwestern University online portal and were anonymous. Student evaluations opened the week after the final exam for the respective quarter. Spring quarter course evaluations from the class of 2022 were compared to Winter quarter course evaluations from the class of 2023, due to COVID-19 restrictions during Spring quarter. Students were asked to agree or disagree with the following items on a 5-point Likert-type scale, where 1 represented strongly disagree and 5 represented strongly agree: "The course objectives were clear."; "The course content was well organized."; "The evaluation methods adequately sampled information I was expected to learn."; "Overall, I rate this course highly." The course evaluation focused on general perceptions of the course. In this research design, we refrained from asking students direct questions about the flipped classroom. To our knowledge, students had not experienced a PES-like course as first-year medical students, thus students may have lacked an accurate baseline comparison for judging traditional vs. flipped classroom teaching in this course. Students could also bring potential biases from previous experiences with flipped classrooms not in the context of PES.
Faculty who taught both cohorts were asked to complete a fiveitem survey (Supplemental Material) sent via Midwestern University e-mail and data was recorded anonymously using the REDCap (Vanderbilt University) electronic data capture tools, one week after the end of courses for the class of 2023 cohort [12,13]. This survey focused on the instructor perception of the traditional vs. flipped classroom and which method instructors felt was better for promoting student learning. The first four items were multiple choice questions directly comparing the traditional vs. flipped classroom instruction. The last item on the survey provided an open-ended comments section.

Data collection
Individual student OSCE scores, practical exam scores, and cumulative grades were collected by the Department of Clinical Integration and available to the research team through the principal investigator's (B.P.) role as the Course Director for PES. The research team (S.B., B.P.) performed statistical analysis and worked with deidentified student data. The Course Director (B.P.) had access to the deidentified student data on Excel, which was then updated to include only raw scores arranged according to grade from low to high, which were then sent to the other author (S.B.) for student and statistical analysis. Data was not sent in alphabetical order to ensure confidentiality. Aggregate data such as participant number, mean scores, and standard deviations from student course evaluations was provided by the Department of Clinical Integration. The faculty survey was administered to core faculty via Midwestern e-mail and survey response data were collected using Midwestern University's REDCap electronic data capture tools to ensure confidentiality [12,13].

Data analysis
To compare objective measures such as cumulative grades, OSCE scores, and practical exam scores, mean and median scores for both cohorts were examined. Nonparametric Mann-Whitney U rank sum tests were used at a 95% confidence level to determine statistically significant differences, as objective data lacked normal distribution. Mean scores were examined to compare subjective feedback from the Spring course evaluations for the class of 2022 and the Winter course evaluations for the class of 2023. Independent sample t-tests assuming unequal variances were used to detect statistically significant differences between the two cohorts at a 95% confidence level. For analyzing the subjective measures from the faculty survey, the number and percent of faculty who responded to each answer choice were calculated. All statistical analyses for the objective measures was performed using Microsoft Excel for Office 365 (2016). Analysis for the subjective student course evaluations was performed using the OpenEpi website (Version 3.01) due to the research team not having access to individual student data points. Thus, aggregate data was used instead to calculate the independent sample t-tests using the OpenEpi free software [14].  Table 1.

Impact on student performance
Mean and median cumulative course grades by semester and type along with mean and median examination scores for the traditional cohort and the flipped classroom cohort are shown in Table 2

Impact on student perception
Student course evaluations from both cohorts were compared to understand whether general course perceptions were impacted by the flipped classroom curriculum. Twenty-seven students (13.6%) from the class of 2022 and 40 students (19.7%) from the class of 2023 participated in the course evaluations. Mean response scores to the four survey questions administered to students (on a 5-point Likert-type scale) and p values are shown in Table 3; there were no statistically significant differences in response values between the two cohorts on the student evaluation items.

Impact on faculty perception
Of 16 core faculty, 10 (62.5%) participated in the survey (Supplemental Material). Of the 10 respondents, four (40%) preferred video instruction (flipped classroom) while six (60%) preferred in-person instruction (traditional approach). Four (40%) believed students were more comfortable and prepared in class because of prior video instruction. Nine (90%) felt their time as an instructor was used more effectively with having the full time dedicated to group sessions with the flipped classroom. Four faculty members (40%) responded that they would prefer the flipped classroom for the future class of 2024, while six (60%) would prefer the traditional approach. Six (60%) wrote comments in response to the final open-ended  question, with mixed perceptions reported. In support of the flipped classroom, one faculty member wrote, "With video, student knew what to expect," and another commented "I believe that all time in lab should be dedicated to in-person instruction without any interruption." A primary concern raised against the flipped classroom by three faculty members was that students came to class without watching the videos entirely. Another faculty member wrote "… I prefer flipped classroom approach, but I believe we would need to find a way to better standardize sessions between instructors," emphasizing an additional consideration with the flipped classroom implementation.

Discussion
The results from our study showed that objective student performance slightly improved with the flipped classroom approach. Fall and Winter cumulative grades, along with history-taking OSCE scores, were significantly higher for the class of 2023 compared with the class of 2022. The addition of the 5-point weekly quizzes in the class of 2023 may have impacted the statistical significance of Fall and Winter cumulative grades, but the practical and OSCE grading criteria were unchanged in both cohorts. While differences in other exam outcomes were not statistically significant, in every instance where there was significant difference between both cohorts, the flipped classroom cohort outperformed the traditional cohort. The results of our study are inconsistent with previous literature, which is not surprising given the wide range of results from previous studies. A meta-analysis [15] using 28 studies with similar methodology of prerecorded videos in health professions settings showed that student perception and performance were both higher in flipped classroom cohorts. Another meta-analysis of 11 studies [16] comparing traditional vs. flipped classroom education in pharmacy school settings found no significant difference between final examination scores. In flipped classroom research by Moffett and Mill [5] in a veterinary professional skills course, student performance declined with the flipped classroom. In Heitz et al.'s flipped classroom study [17] in an emergency medicine clerkship, there were no statistically significant improvements with the flipped classroom. However, in Belfi et al.'s flipped classroom research [18] in a radiology introductory clerkship, students participating in the flipped classroom ("blended" classroom) had significant improvements in posttest results compared with control groups. In each of these previous studies, however, student performance was evaluated through written examinations only. The evaluation methods for PES in this study are based on both written exams and graded evaluation of PES performance with standardized patients.
Regarding subjective student course perceptions, less than 20% of students from both cohorts completed the course evaluations. Previous medical education flipped classroom studies have shown that students generally have positive perceptions of the flipped classroom [10]. However, those studies gauged positive perception by asking students direct questions about the flipped classroom, a method avoided in our current study to limit bias [19,20]. Future researchers of this topic are encouraged to find a way to assess student course perceptions with participation from most students without directly asking students about flipped vs. traditional classroom approaches. Furthermore, studies to establish the efficacy a "hybrid" flipped classroom model would be beneficial, with some content delivered outside the classroom and some content delivered traditionally.
In our study, faculty members in both cohorts participated in a "huddle" before PES, where the roles of the faculty were clarified before each session to ensure consistency in PES teaching. An additional improvement to this structure could be to have faculty members watch the instructional videos provided to students, to ensure all faculty are in alignment with the weekly session content. However, the fact that 90% of faculty reported feeling their time was used more efficiently with the flipped classroom showcases how this approach has potential to improve the faculty teaching experience in PES with some modifications.

Osteopathic considerations
Implementation of the flipped classroom in osteopathic medical education is important for modeling lifelong learning and critical for enhancing professionalism in students. Future research could replicate the current study and accumulate more support for implementation of the flipped classroom by comparing skills-based assessments with standardized patients rather than solely using written examinations as an objective means of comparison between cohorts. Increasing evidence supporting the flipped classroom using skill-based assessment performance as a measure can benefit the osteopathic medical student experience in multiple areas. OMM coursework in osteopathic medical schools also uses hands on performance during practical examinations as a means of student evaluation [21]. Expanding research around the flipped classroom model using skills-based assessments as an objective measure is essential to expand the osteopathic medical student curriculum not just in PES, but in other preclinical coursework.

Limitations
A potent limitation of this study was the shift to off-campus learning during Spring quarter for the class of 2023 due to COVID-19 online learning modifications. Thus, data from the Spring cumulative grades, Spring practical exam scores, and full physical OSCE exam scores were not compared between the two cohorts. However, despite these limitations, this study had five strong objective measures where student performance from both cohorts were compared to evaluate the traditional vs. flipped classroom approach.
Another limitation of our study was the assumption that all students in the class of 2023 completely participated in the flipped classroom protocol. We attempted to control for this concern by administering a 5-point quiz on content covered in the prerecorded videos and outline to start each PES session. Despite this control, some faculty members believed students did not watch the full instructional videos, driving the preference of 60% of faculty for the traditional classroom. Future studies should consider developing further means of measuring student participation and adherence to the flipped classroom protocol. This could be accomplished through administration of a student survey, similar to the one described Tune et al.'s research [22], where students would be able to anonymously report whether they adhered to watching the instructional videos.
Another limitation of our study design was that two cohorts from two separate academic years were analyzed. An alternative research design would be to have the same cohort participate in both the traditional method of teaching and the flipped classroom curriculum within the same academic year to potentially limit any confounding variables. However, we refrained from adopting this design because providing each student with the same PES experience is important and we wanted to avoid any negative impact on student education.

Conclusions
A flipped classroom approach yielded some statistically significant improvement in student performance of PES. Data from student and faculty perceptions were both limited in response rate. Given the COVID-19 pandemic's effect on medical education, including a need to develop innovative methods to deliver osteopathic physical examination content, the flipped classroom provides an opportunity to balance virtual with in-person learning. Further studies are needed to expand the support for evaluating the benefits of a flipped classroom approach using skills-based assessments styles to measure student performance with a focus on standardization of in-class groupwork.