During the COVID-19 pandemic, many clinicians quickly adapted their way of practicing patient care by offering telehealth and virtual office visits while simultaneously having to minimize direct patient care. The shift in direct clinical learning opportunities provided to third- and fourth-year medical students required a shift in the educational curriculum to develop learner skills around the appropriate use of telehealth in patient care.
The aim of this project was to provide exposure to students so they could learn the telemedicine equipment and best practices, and how to identify infectious diseases to improve access to care and meet the needs of the patient.
In July and August of 2020, the Indiana Area Health Education Centers Program partnered with Marian University College of Osteopathic Medicine (MUCOM) to support a 1 day telehealth simulation (online curriculum, group lecture, and two standardized patient encounters) into their clerkship curriculum. We utilized a retrospective pretest-posttest to assess changes in learner knowledge around telehealth after the program. At the conclusion of the telehealth training program, students were asked to complete a retrospective pretest-posttest assessing their level of preparedness to utilize telehealth equipment, their preparedness to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy, their confidence to utilize telehealth for identification of infectious diseases, and their confidence to utilize telehealth to identify proper treatment plans.
A total of 96 learners completed the program in 2020. Posttest results demonstrate a statistically significant (p<0.05) improvement for learners’ self-reported level of preparedness to utilize telehealth equipment, their preparedness to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy, their confidence to utilize telehealth for identification of infectious diseases, and their confidence to utilize telehealth to identify proper treatment plans.
Our telehealth curriculum involving a video, interactive learning session, and two standardized patient experiences provided osteopathic medical learners with realistic simulated case scenarios to work through in effort to improve their knowledge and self-efficacy around the utilization of telehealth in practice.
Telehealth connects patients to vital healthcare services through videoconferencing, remote monitoring, electronic consults, and wireless communications to improve access to health care . The goal of telehealth is to increase access to physicians and specialists in order to help ensure that patients receive the right care, at the right place, at the right time . Although telehealth technology has the potential to revolutionize healthcare delivery, providers were slow to participate in telemedicine prior to the pandemic . With the emergence of the COVID-19 pandemic, the healthcare system demonstrated the agility to apply telehealth approaches yet exposed the gap in widespread telehealth implementation [3, 4]. Simultaneously, over the past several years, medical specialty organizations have developed specialty-specific guidelines and tips for optimal telehealth use, yet formal curricular education for medical students appears to lag against other advancements in telehealth . In order to develop a health workforce willing and able to practice telehealth, early exposure throughout undergraduate medical education is necessary to prepare learners to practice telehealth .
As the COVID-19 pandemic unfolded, many clinicians quickly adapted their way of practicing patient care by offering telehealth and virtual office visits—especially for vulnerable populations in rural and medically underserved communities . In addition, the federal government, state Medicaid programs, and private insurers expanded coverage for virtual telehealth care services . During the first quarter of 2020, the number of telehealth visits increased by approximately 50% compared with the same period in 2019, with a 154% increase in visits during surveillance week 13 in 2020 compared with the same time period in 2019 . This appeared to be closely related to the rise in COVID-19 – related evaluations . Although telehealth is becoming a standard component in the practice of medicine, telehealth has not universally been a component of medical school training. One 2019 review noted that over one-fourth of allopathic schools reported in 2015–2016 that they had preclinical telehealth training .
Both the American Medical Association (AMA)  and the American Association of Colleges of Osteopathic Medicine (AACOM)  recommend training medical students in telehealth services. On March 17, 2020, the Association of American Medical Colleges (AAMC) and AACOM recommended that medical students be paused from clinical rotations due to the increase in COVID-19 cases and concerns over student safety and an inadequate amount of personal protective equipment. As a result, medical learners were left with a gap in the curriculum that could be utilized in other manners in which to augment clinical knowledge and skills.
Simulation is a widely utilized model of medical training. Simulations allow hands-on training in the presence of faculty oversight and feedback to the learner. The program described in this article closely aligned with the Indiana Area Health Education Center Network (IN-AHEC) goal to enhance workforce training around the use of telehealth technologies as a response to the COVID-19 pandemic. This article describes the medical student telehealth training 1 day program developed at Marian University College of Osteopathic Medicine (MUCOM) in partnership with the IN-AHEC Network during the 2020 COVID-19 pandemic year and the initial positive student-reported outcomes after its first-year of implementation.
In July and August of 2020, IN-AHEC partnered with MUCOM to support a telehealth simulation demonstration project into their medical education curriculum. Experience in telehealth is beneficial to future physicians because telehealth plays a role in patient care when geography, physician shortages, or safety concerns limit the availability of face-to-face visits. The purpose of the project was to improve osteopathic students’ knowledge, confidence, and preparedness for utilizing telehealth equipment to follow best practices, identify infectious disease, and develop appropriate treatment plans.
This study was reviewed and approved with exempt status (10638) by the Indiana University Institutional Review Board. The program was conducted for educational purposes, not research purposes, thus written consent was not obtained. Rather, we utilized the existing education data to assess the impact of the program. The program and related study were funded by the U.S. Department of Health and Human Services, Health Resources and Services Administration (HRSA), Area Health Education Centers COVID program (grant T1KHP39172). Students did not receive any compensation for participating in the educational program.
The MUCOM telehealth program provided a hybrid of didactic modules, in-person lecture, and simulated telehealth with realistic case studies to expose learners to appropriate and inappropriate uses of telehealth for patient care. The 1 day learning session utilized an online didactic curriculum, in-person lecture, and two standardized patient encounters. Participation in the telehealth curriculum was mandatory for all third- and fourth-year osteopathic medical students. For part of the curriculum, all students were required to watch the American College of Physicians (ACP) comprehensive review of telemedicine . On the day of the program, groups of 15–20 students engaged in an interactive 45 min small group lecture (facilitated by faculty) on telemedicine followed by a virtual encounter with a standardized patient utilizing telemedicine. All students had two separate 30 min encounters with a standardized patient offsite via the computer. The encounter was typical of what would be seen in a family medicine telemedicine encounter. The two cases included an uncomplicated urinary tract infection and chest pain. The chest pain case emphasized that not all encounters can be conducted via telemedicine. In this case, students were trained to recognize that the patient should be referred to the Emergency Department immediately. After each case, students received feedback from the standardized patient and participated in a post-encounter debrief with program faculty. Following the program, the learners were given the external program survey to share their experience in a manner separate from their academic course grade. Although all students engaged in the same specific cases, they were not permitted to share any knowledge of the cases with their peers. Figure 1 outlines the curricular topics of the telehealth curriculum.
We utilized a retrospective pretest-posttest to assess changes in learner knowledge around telehealth after the program. The retrospective pretest-posttest method is a simple method of assessing changes in self-efficacy because the act of asking both contemporary and retrospective answers at the same time ensures that the participant’s frame of reference for interpreting a given question will remain the same . At the conclusion of the telehealth training program, students were asked to complete a retrospective pretest-posttest assessing their level of preparedness to utilize telehealth equipment, their preparedness to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy, their confidence to utilize telehealth to identify infectious diseases, and their confidence to utilize telehealth to identify proper treatment plans. Students responded utilizing a five-point Likert scale ranging from strongly disagree to strongly agree.
The survey completed by students was voluntary and not connected to their academic scores in the curriculum. All demographic and program evaluations were completed on a self-reported form by the students, which allowed for them to self-identify their race and ethnicity (with open formatting if they did not think that the provided HRSA-identified federal categories did not fit their identity). Demographic information around race and ethnicity was collected according to federal requirements associated with the funding because the AHEC program strives to enhance opportunities for health profession students from underrepresented-in-medicine backgrounds. The inclusion criteria for the study required the student both to be engaged in the telehealth curriculum and to complete the post-event AHEC program evaluation tool. To test the potential statistical significance in the hypothesis, we conducted a paired-sample t-test to assess a difference in the means in the student’s pre- and posttest. All data were collected with FormAssembly software and storage in a cloud-based Salesforce platform. All statistical analysis was conducted utilizing IBM SPSS 26. We considered two-sided p values of < 0.05 as statistically significant.
In July and August of 2020, 95 osteopathic medical students participated in a training program to develop their preparedness and confidence to utilize telehealth and completed the AHEC program evaluation tool. As highlighted in Table 1, the majority of our study population self-identified as white (74; 77.9%) and female (55; 57.9%). Three-fourths (72; 75.8%) of our participants reported coming from a rural (38; 40.0%) and/or disadvantaged background (34; 35.8%). The results of the telehealth preparedness assessments are highlighted in Table 2 below, which depicts the pretest-posttest means, paired t-test, sample mean, and standard deviation. We asked the learners to self-report their self-efficacy on feeling prepared for utilizing telehealth equipment. The mean pretest score for utilizing telehealth equipment was 1.91 (where zero indicates strongly disagree and four is strongly agree), while the posttest score was 3.03. The t-test results showed a statistically significant gain (t=−9.677; n=76; p<0.05), and the overall results indicate an increase in self-efficacy around utilizing telehealth equipment.
|Black or African American||3||3.2|
|More than one race||7||7.4|
|Hispanic or Latinx||5||5.3|
|Self-identified as coming from a rural background||38||40.0|
|Self-identified as coming from a disadvantaged background||34||35.8|
|Academic year – 1||0||0.0|
|Academic year – 2||1||1.1|
|Academic year – 3||52||54.7|
|Academic year – 4||42||44.4|
|Pretest mean||Posttest mean||Overall change in mean||Standard deviation||t (df)|
|I feel prepared to utilize telehealth equipment.||1.91||3.03||−1.12||1.013||−9.677 (76)a|
|I feel prepared to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy.||1.95||2.99||−1.04||1.156||−7.934 (77)a|
|I feel confident in my ability to utilize telehealth for identification of infectious diseases.||1.62||2.79||−1.17||0.909||−11.283 (76)a|
|I feel confident in my ability to utilize telehealth to identify proper treatment plans.||2.05||3.05||−1.00||0.940||−9.398 (77)a|
We asked the learners to self-report their self-efficacy on feeling prepared to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy. The mean pretest score for demonstrating “telehealth best practices” was 1.95, and the posttest score was 2.99. The t-test results showed a statistically significant gain (t=−7.934; n=77; p<0.05), and the overall results indicate an increase in self-efficacy of how to utilize telehealth in a manner consistent with telehealth best practices.
We asked the learners to self-report their self-efficacy on utilizing telehealth to identify infectious diseases. The mean pretest score for utilizing telehealth to identify infectious diseases was 1.62, and the mean posttest score was 2.79. The t-test results showed a statistically significant gain (t=−11.283; n=76; p<0.05), and the overall results indicate an increase in self-efficacy in the ability to utilize telehealth visits to identify infectious diseases.
Lastly, we asked the learners to self-report their self-efficacy to utilize telehealth to identify the proper treatment plans. The mean pretest score for utilizing telehealth to identify the proper treatment plans was 2.05, and the mean posttest score was 3.05. The t-test results showed a statistically significant gain (t=−9.398; n=77; p<0.05), and the overall results indicate an increase in self-efficacy on identifying the proper treatment plans when utilizing telehealth equipment and services.
Our findings demonstrate a statistically significant improvement for learners’ self-reported level of preparedness to utilize telehealth equipment, preparedness to demonstrate “telehealth best practices” in a manner consistent with protecting patient (and data) privacy, confidence to utilize telehealth to identify infectious diseases, and confidence to utilize telehealth to identify proper treatment plans. In further analysis, we found no significant differences based on the gender, rural background, disadvantaged backgrounds, or academic year of the learners.
Given the dynamic shift to better incorporate telehealth into professional practice as result of the COVID-19 pandemic, medical education has a duty to enhance the academic curriculum to ensure that learners have the skills to utilize telehealth into practice. Medical learners need to be equipped with the tools needed to utilize telehealth to address the AHEC mission to increase access to care by enhancing the health workforce. During the project, we discussed with the students that telemedicine was being much more utilized in private practice due to the pandemic. The continued utilization of telehealth will ensure that providers can meet the needs of their patients in rural and urban underserved communities, especially for those with challenges associated with transportation, distance, mobility, or time (associated with employment and travel) .
This project showed significant improvement in the learner’s self-reported knowledge and confidence around utilizing telehealth in practice. Coupling simulation as an immersive teaching and learning tool with active learning methodology allows students to move from rote memory to acquiring the cognitive schema to apply memorized information into practice . Existing research around simulation-based learning reported that students felt that the model improved their clinical skills, their ability to retain learning materials, their clinical decision-making, and their communication skills with patients .
The inclusion of the telehealth training program provided osteopathic medical learners with realistic simulated case scenarios to work through to improve their skills utilizing telehealth in practice. While we recognize that correlation does not necessarily equate to causation, we believe in this case that the core components of the program provided learners with an increased opportunity to apply the didactic materials in a valuable manner to increase skills and self-efficacy around how to utilize telehealth equipment, learn best practices, and utilize the technology to identify infectious diseases to develop appropriate patient care plans.
The aim of this project was to provide exposure to students so that they could learn the telemedicine equipment and the best practices, and to learn how to identify infectious diseases to improve access to care and meet the needs of the patient. This meets the mission of osteopathic physicians to provide care that includes osteopathic principles and practice, which is the integration of osteopathic philosophy into healthcare practices. Moving forward, osteopathic principles and practice could be further integrated into the telemedicine curriculum by expanding the learning objectives to include the identification of chief complaints, diagnostic maneuvers, and techniques that may be addressed and taught via telemedicine (such as the Galbreath technique, a manipulative treatment for otitis media).
The study faced several limitations that may impact the application of our study. Our findings are based on that of one osteopathic medical school curriculum, therefore participation in similar programs across the nation may result in varying experiences for the learners. While the program was mandatory for all third- and fourth-year medical students, we had several learners who did not complete the external program evaluation provided by the AHEC program, therefore those students were not included in the study results looking at the 95 students. A further limitation is the self-reported nature of the evaluations. The program results rely on the students’ self-reported knowledge and self-efficacy rather than observable assessment by a faculty. Additionally, although we believe that the program has a strong potential to positively influence learner self-efficacy pertaining to utilizing telehealth, we cannot discount the bias related to the self-reported nature of the evaluation.
The landscape of providers utilizing telehealth to improve access to care and subsequent patient outcomes will continue to evolve, and the undergraduate medical education curriculum needs to evolve simultaneously. Our telehealth curriculum involving a video, interactive learning session, and two standardized patient experiences in telehealth improved students’ self‐reported knowledge in telehealth. Future studies should observe learning sessions and assess applied skills to develop a health workforce that is willing and competent to utilize evolving and emerging technology to improve patient health outcomes.
Funding source: Bureau of Health Professions, Health Resources and Services Administration
Award Identifier / Grant number: T1KHP39172
Research funding: The project described was supported by Bureau of Health Professions, Health Resources and Services Administration Grant Number T1KHP39172. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the U.S. Department of Health and Human Services, Health Resources and Services Administration, Area Health Education Centers COVID program.
Author contributions: All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; all authors drafted the article or revised it critically for important intellectual content; all authors gave final approval of the version of the article to be published; and all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Competing interests: None reported.
Ethical approval: This study was reviewed and approved with exempt status (10638) by the Indiana University Institutional Review Board.
1. American Hospital Association. Telehealth fact sheet: AHA. Available from: [Accessed 18 Mar 2021.Search in Google Scholar
2. Talal, AH, Sofikitou, EM, Jaanimägi, U, Zeremski, M, Tobin, JN, Markatou, M. A framework for patient-centered telemedicine: application and lessons learned from vulnerable populations. J Biomed Inf 2020;112:103622. https://doi.org/10.1016/j.jbi.2020.103622.Search in Google Scholar
3. Camhi, SS, Herweck, A, Perone, H. Telehealth training is essential to care for underserved populations: a medical student perspective. Med Sci Educ 2020;30:1–4. https://doi.org/10.1007/s40670-020-01008-w.Search in Google Scholar
4. Wegermann, K, Wilder, JM, Parish, A, Niedzwiecki, D, Gellad, ZF, Muir, AJ, et al.. Racial and socioeconomic disparities in utilization of telehealth in patients with liver disease during COVID-19. Dig Dis Sci 2021:1–7. https://doi.org/10.1007/s10620-021-06842-5.Search in Google Scholar
5. Walker, C, Echternacht, H, Brophy, PD. Model for medical student introductory telehealth education. Telemed J e Health 2019;25:717–23. https://doi.org/10.1089/tmj.2018.0140.Search in Google Scholar
6. Tapper, EB, Asrani, SK. The COVID-19 pandemic will have a long-lasting impact on the quality of cirrhosis care. J Hepatol 2020;73:441–5. https://doi.org/10.1016/j.jhep.2020.04.005.Search in Google Scholar
7. U.S. Health Resources and Services Administration. Billing for telehealth during COVID-19. Available from: [Accessed 2 Jul 2021].Search in Google Scholar
8. Koonin, LM, Hoots, B, Tsang, CA, Leroy, Z, Farris, K, Jolly, T, et al.. Trends in the use of telehealth during the emergence of the COVID-19 pandemic - United States, January-March 2020. MMWR Morb Mortal Wkly Rep 2020;69:1595–9. https://doi.org/10.15585/mmwr.mm6943a3.Search in Google Scholar
10. Kelly, J, American Medical Association. AMA encourages telehealth training for medical students, residents. Available from: [Accessed 25 Jun 2021].Search in Google Scholar
11. American Association of Colleges of Osteopathic Medicine. Updated message from AACOM regarding students returning to clinical activities. Available from: [Accessed 25 Jun 2021].Search in Google Scholar
12. American College of Physicians. Telemedicine: a practical guide for incorporation into your practice. Available from: [Accessed 2 Jul 2021].Search in Google Scholar
13. Chang, R, Little, TD. Innovations for evaluation research: multiform protocols, visual analog scaling, and the retrospective pretest-posttest design. Eval Health Prof 2018;41:246–69. https://doi.org/10.1177/0163278718759396.Search in Google Scholar
14. Woodall, T, Ramage, M, LaBruyere, JT, McLean, W, Tak, CR. Telemedicine services during COVID-19: considerations for medically underserved populations. J Rural Health 2021;37:231–4. https://doi.org/10.1111/jrh.12466.Search in Google Scholar
16. El Naggar, MA, Almaeen, AH. Students’ perception towards medical-simulation training as a method for clinical teaching. J Pak Med Assoc 2020;70:618–23. https://doi.org/10.5455/JPMA.6481.Search in Google Scholar
© 2021 Jennifer Taylor et al., published by De Gruyter, Berlin/Boston
This work is licensed under the Creative Commons Attribution 4.0 International License.