Skip to content
Publicly Available Published by De Gruyter January 5, 2023

A systematic review of videoconferencing in health professions education: the digital divide revisited in the COVID-19 era

  • Jennie C. De Gagne , Paula D. Koppel , Emily J. Wang , Sharron Rushton , Leila Ledbetter , Sandra S. Yamane , Eunhee Lee , Kimberly Manturuk and Dukyoo Jung ORCID logo EMAIL logo

Abstract

Objectives

During the COVID-19 pandemic, educators shifted from traditional lectures to videoconferencing. This systematic review explored the use of videoconferencing as a teaching tool in response to the pandemic as well as issues related to digital equity and inclusion.

Content

The review was conducted using the Joanna Briggs Institute for Systematic Reviews methodology and reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 statement.

Summary

A total of nine studies met eligibility criteria. The participants in the included studies were medical students from various parts of the world. Technical difficulties and lack of human interactions were identified as barriers to learning through videoconferencing.

Outlook

To achieve full success, pedagogical videoconferencing must prioritize digital equity and a universal design for learning. Although useful for maintaining education during the pandemic, in the future, videoconferencing will present challenges related to the digital divide as well as opportunities as a teaching tool for nurse educators globally.

The COVID-19 pandemic presented enormous challenges for institutions of higher education. Educators and students, from all health professions including nursing, were pushed to adopt remote pedagogy quickly as they transitioned from the physical classroom to virtual learning (Crawford et al., 2020; UNESCO, 2020). In contrast to planned and resourced online learning environments, remote learning classrooms implemented abruptly by untrained faculty often felt chaotic (SUNY Online, 2021). Although synchronous virtual lectures via a videoconferencing tool (hereafter videoconferencing pedagogy [VCP]) became a common alternative to face-to-face lectures (Correia et al., 2020), many educators and students found VCP frustrating or even daunting, especially when this pedagogical choice had been imposed with limited time to prepare. In addition to the stress of an unfamiliar learning environment and the impact of a global pandemic, some users faced the challenge of the digital divide or digital inequality (Correia, 2020). This study aimed to synthesize evidence of the use of videoconferencing as a teaching tool in health professions education during the COVID-19 pandemic viewed through the lenses of the digital divide and inclusive pedagogy. The specific review questions were the following:

  1. What were the outcomes of VCP during the COVID-19 pandemic?

  2. What were the facilitators and barriers to VCP and its implications for practice?

Background

Digital divide

The digital divide refers to the gap between those who benefit from digital accessibility and those who do not (McLean, 2021). This divide is grounded in broader societal and civic conditions (McKenzie, 2021). The COVID-19 pandemic accentuated existing inequalities among college students (June, 2020) which had long been the focus of campus-based support systems and resources to close the digital divide (Digital Future Society, 2019). The move to VCP forced students to be reliant on technology, including computers and Wi-Fi resources (McKenzie, 2021), to which they no longer had on-campus access. The level of access to technology required for successful VCP is multifaceted and dependent on devices, technical specifications, and internet connection (Williamson et al., 2020). These factors, when adequately addressed, allow all participants to be involved in the classroom equally; however, unequal access to technology and resources creates disparities in the educational experience, as when students with limited broadband access experience difficulties navigating the digital ecosystem efficiently (González-Betancor et al., 2021; Stamler & Upvall, 2021). Minoritized students are more likely to struggle with video technologies used in virtual learning (Pacansky-Brock et al., 2020).

Inclusive pedagogy

Inclusive pedagogy comprises teaching strategies that ensure flexible, accessible, and sustainable learning for all students (Cotán et al., 2021). Inclusive pedagogy entails humanizing the learning experience through commitment to inclusivity, diversity, and accessibility (Mehta & Aguilera, 2020). Mehta and Aguilera (2020) explored an ideological approach to humanizing digital pedagogies to promote a more inclusive learning environment across virtual contexts. Although digital learning often incorporates Universal Design for Learning (UDL), a framework founded on humanizing pedagogical principles and a commitment to addressing learners’ diverse needs, it has been suggested that factors of power, privilege, and ideology need to be addressed (Mehta & Aguilera, 2020). Although many universities use digital technologies to support students, erroneous assumptions built into technology algorithms and programs can exacerbate educational inequities, social injustice, and culturally marginalizing practices (Mehta & Aguilera, 2020). A critical and humanizing framework cannot be delineated by a set of regulations but should be recontextualized to meet the full needs of the learners it serves (Iturbe-LaGrave et al., 2021; Mehta & Aguilera, 2020). Pacansky-Brock and colleagues (2020) have provided a model that integrates the theoretical foundations of UDL, culturally responsive teaching, social presence, and validation theory. This model uses humanizing strategies and highlights the importance of connection and empathy to facilitate instructor–student relationships and community in online teaching.

Videoconferencing as a teaching tool

Although videoconferencing has been used since the 1960s (Correia et al., 2020), this methodology was impractical for pedagogical purposes until the rise of the internet and the development of accessible and easy-to-use technology (Correia et al., 2020). When paired with student learning objectives, VCP can eliminate geographic boundaries and improve learning outcomes (McGonigle & Mastrian, 2021). Videoconferencing offers a powerful tool for the nurse educator because the addition of a collaborative component to the online education setting can enhance the learning experience for students whose learning styles differ (Hebda et al., 2019). Given its increased presence and potential challenges, studies have demonstrated the need for VCP guidelines (Correia et al., 2020); such guidelines should (a) consider the impact of cultural context on teaching and learning experiences, and (b) include supportive means of mitigating identified inequities that may be introduced through the use of videoconferencing (Mehta & Aguilera, 2020). Strategic teaching that emphasizes the promotion of self-determined learning in students through the development of metacognitive strategies is key to making VCP an adaptable tool that bolsters student engagement during the pandemic and beyond (Mehta & Aguilera, 2020).

Methods

The nature of our study’s questions led to the selection of a systematic review as its methodology; this was guided by the Joanna Briggs Institute for Systematic Reviews (Aromataris & Munn, 2020). The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement (Page et al., 2021) was used to report this review (Appendix I). An a priori protocol was developed and followed. The authors’ institutional review board determined that the protocol of this study (Pro00108625) meets the definition of research not involving human subjects.

Search strategy

In the initial phase, a preliminary search was conducted to determine whether existing reviews on this topic exist and to identify appropriate keywords. The search found no systematic reviews that examined VCP in health professions in response to the COVID-19 pandemic. This process was verified and validated by MEDLINE (PubMed) and Cumulative Index to Nursing and Allied Health Literature (CINAHL). The initial phase informed the second phase of the search process, in which the strategy was finalized and tailored to each information source. The search was developed and conducted by a professional medical librarian (LL), with expertise and input from the authors, and included a mix of keywords and subject headings representing videoconferencing, education, and digital divide. Ultimately, five databases were searched: MEDLINE (via PubMed), CINAHL (via EBSCOhost), ERIC (via EBSCOhost), Education Full Text (H.W. Wilson) – (via EBSCOhost), and Embase (via Elsevier).

Search hedges or database filters were used to remove (a) virtual reality and other online educational technologies outside the scope of our research question; and (b) publication types such as editorials, letters, case reports, comments, book chapters, and abstracts, as appropriate for each database. The search was conducted on March 16, 2021, with updates on June 3, 2021 and March 30, 2022. Complete reproducible search strategies, including date ranges and search filters for all databases, are detailed in the Supplementary Materials (Appendix II). From April 1–11, 2022, the reference lists of the included articles were reviewed and citation tracking in Web of Science and Scopus was used to identify relevant studies, which were added for full-text review.

Eligibility criteria

The PICOTTS (Population, Intervention, Comparison, Outcome, Timeframe, Type of Study, and Setting) framework (Haynes et al., 1997) was used to identify the inclusion and exclusion criteria and select literature. The review selected the broad inclusion criteria of pre-licensure health professions students rather than focusing on one health discipline to capture as much data as possible about this sudden increase in VCP. We anticipated a need and benefit to learning from the experiences of interdisciplinary health professions research. As such, inclusion criteria were as follows: (1) participants had to be composed of only pre-licensure health professions students; (2) studies evaluating the use of videoconferencing in place of face-to-face lecture due to COVID-19; (3) outcomes of course evaluation could include any of Kirkpatrick’s (2016) four levels of evaluation; and (4) any data-based studies conducted in an academic setting since the onset of COVID-19. Exclusion criteria were as follows: (1) studies with a mixed population (e.g., class that includes both pre-licensure and other types of health professions students); (2) studies that focused on evaluation of social media or simulation; (3) studies that did not report any student or course evaluation; (4) articles such as editorials, reviews, protocols, and conference presentations; (5) studies involving clinical learning environments. The detailed eligibility criteria are presented in Supplementary Material (Appendix III).

Selection of evidence sources

After the search, all identified studies were uploaded into Covidence (Veritas Health Innovation, Melbourne, Australia), a software system for managing systematic reviews; duplicates were removed by the software. Study selection was carried out independently by two of the authors (JCD & PDK). Studies were excluded if they did not clearly meet inclusion criteria based on title and/or abstract review. Conflicts between the two reviewers were resolved by a third reviewer (SR). At the full-text screening stage, papers were reviewed in detail by two independent reviewers (JCD & PDK) and excluded if they did not meet inclusion criteria. Conflicts between the two independent reviewers were resolved through discussion. If papers not published in English met the inclusion criteria during the title/abstract screening, the abstracts were reviewed for usable data. Due to restrictions in funding, we chose not to have these papers translated, and they were excluded at the full-text screening phase.

Data extraction and synthesis

The data were extracted by two independent reviewers (SR/EJW and PDK), reviewed by a second team member with discrepancies resolved by discussion and when necessary, a third reviewer (JCD). Data were summarized in a tabular form. In alignment with the objective of this study, we reported variables including characteristics of included studies and outcomes. Extracted elements included study characteristics, intervention, videoconferencing platform, and barriers as well as facilitators to the implementation of VCP. Pedagogical outcomes were assessed according to Kirkpatrick’s four levels of evaluation (i.e., Reaction, Learning, Behavior, Results). Kirkpatrick’s Level 1 refers to the level of students’ reaction to instruction, and Level 2 indicates the changes in students’ learning after participation in the instruction. Level 3 assesses whether the instruction changed learners’ behavior, and Level 4 measures the instruction’s effectiveness or direct results against the institution’s business goals (Kirkpatrick & Kirkpatrick, 2016).

Quality assessment

A formal methodological quality assessment was performed using the Medical Education Research Study Quality Instrument (MERSQI). This widely used tool for quality appraisal in educational research consists of six domains: (1) study design, (2) sampling, (3) types of data, (4) validity of evaluation instrument, (5) data analysis, and (6) outcomes (Cook & Reed, 2015). Each domain carries a maximum score of 3 and a minimum of 1 (except the sampling domain, in which the minimum score is 0.5). The total score of the MERSQI can range between 5 and 18. Although there is no available cutoff for a good MERSQI score, a higher score on the scale indicates a higher quality paper. Two independent reviewers (SR/EJW and PDK) performed the MERSQI scoring, and discrepancies were resolved by discussion between them or by a third reviewer (JCD).

Results

A total of 231 potential studies were identified from five databases. After removing 69 duplicates, the remaining articles were reviewed, and those that did not meet the inclusion criteria were excluded. A total of 162 articles were screened by title and abstract, of which 122 articles were excluded. A total of 40 full-text articles were downloaded and assessed, from which 35 were excluded with reasons. Reference and citation tracking were performed on the remaining five articles, resulting in four additional articles. In total, nine studies were selected for this study. A flow chart of the study selection process is presented in Figure 1.

Figure 1: 
Flow diagram depicting the study selection process.
Figure 1:

Flow diagram depicting the study selection process.

Sample characteristics

A total of nine studies met the criteria and were included in the analyses. Methodological quality assessed from the MERSQI scores ranged from 7 to 13.5 (out of a possible 18) with an average of 10.4 (see Appendix IV). Most of the studies were designed as a single group study (n=7), and two studies were conducted using a nonrandomized 2-group study design. The included studies were published between 2020 and 2022 and emanated from various parts of the world. Eight studies were conducted in single countries: Germany (n=2), UK (n=1), the United States (n=1), Israel (n=1), Poland (n=1), Brazil (n=1), and Cyprus (n=1). The remaining study involved participants from Qatar, Saudi Arabia, Peru, and Argentina. The most frequently used platform was Zoom (n=4), followed by Blackboard Collaborate (n=2), Microsoft Teams (n=1), and Google Meet (n=1). One study did not report the platform used.

The participants in all studies were medical students. The most common course topics taught by VCP in these studies involved basic medicine (e.g., histology, pathology, histopathology) (n=7). Other topics included medical informatics (n=1), and first aid, physical examination, and clinical science (n=1). Similar to pre-pandemic courses, the VCP courses described in the studies consisted of multi-sessions with activities including lectures, quizzes or assignments, discussions, student presentations, and simulations; however, high-fidelity simulations, clinical training, and practice tests that required face-to-face activities were canceled or provided in a simplified form (Skrzypek et al., 2020; Uraiby et al., 2022). The courses in six studies were mandatory and provided on the same schedule as before the Covid-19 pandemic, while the elective courses (n=3) were taught over a relatively short timeframe (2–5 weeks) (Behrends et al., 2020; Guiter et al., 2021; Samueli et al., 2020).

Various learning resource materials such as photographic images, e-books, videos, tests using virtual games, and virtual simulations were used to provide VCP course activities (see Table 1). If a learning management system (LMS) had been established and utilized prior to the COVID-19 pandemic, the course was provided by linking various learning materials and synchronous learning platforms based on the existing LMS. The evaluation system in the included studies consisted mainly of course evaluations by students and evaluations of student performance by instructors. A structured questionnaire or narrative feedback was used for the course evaluation by students; student performance was evaluated based on grades for tests, homework completion, and course participation.

Table 1:

Characteristics of included studies (n=9).

Authors (year)/country Discipline/participants, sample size (n) VC platform Course topic and activities Course description
Behrends et al. (2020)/Germany Medical students (n=12) Virtual room (not specified)
  1. Topic: Medical informatics

  2. Activities: 14 collaborative work assignments, forums, presentations with dialogue, wikis

  1. Type: Elective

  2. Course length: 5 weeks

  3. Session details: 4 virtual meetings

  4. Resources: University Learning Management System (LMS)

Darici et al. (2021)/Germany Undergraduate medical students in 2nd and 3rd semesters (n=392) Zoom
  1. Topic: Histology (cell biology, tissue histology, organ histology)

  2. Activities: Quizzing, slide review, presentations, suggested readings, small group work in breakout sessions

  1. Type: Mandatory

  2. Course length: 1 semester

  3. Session details: 2 online sessions, 3 h per week; students were randomly assigned to breakout sessions.

  4. Resources: Annotated histological images, e-books, presentation slides, test proctoring, virtual microscopy

Guiter et al. (2021)/Qatar, Saudi Arabia, Peru, Argentina Undergraduate medical students (n=29) Zoom
  1. Topic: Oncologic pathology

  2. Activities: Case studies, discussion, flipped classroom with independent student preparation, questions, zoom polling

  1. Type: Elective

  2. Course length: 2 weeks

  3. Session details: 3 sessions per week 90–120 min

  4. Resources used: clinical pathological cases (vignette, diagnostics information – image findings, laboratory findings, gross specimens, and immunohistology), LMS (canvas), University of Leeds’s Virtual Pathology Library for pathology slides

Hernandez et al. (2021)/USA First-year undergraduate medical students

Course evaluation:
  1. In-person (n=67)

  2. Remote (n=64)

Exam results:
  1. In-person (n=140)

  2. Remote (n=139)

Zoom
  1. Topic: General pathology

  2. Activities: Recorded lectures, lab demonstrations, computer-based exercises, patient vignette-based lab exercises, quizzes, final exam, practical lab exam

  1. Type: Mandatory

  2. Course length: 4 weeks

  3. Session details: 17 h of lectures, 15 h of lab-based instruction, lab with a review of specimens, group computer-based vignette exercises, weekly quizzes, final exam, practical examination

  4. Resources: Echo360, blackboard learn, desktop document camera

Nikas et al. (2022)/Cyprus First and third-year medical students and second-year biomedical sciences program students (n=173) Blackboard collaborate
  1. Topics: Histology, pathology

  2. Activities: Weekly lectures, laboratory sessions with virtual microscopy, and quizzes; independent lab assignment and midterm and final examinations

  1. Type: Mandatory

  2. Course length: 13 weeks

  3. Session details: Weekly lecture and laboratory. Lectures included virtual microscopy and student participation with microphone, chat, or polling.

  4. Resources: Respondus lockdown browser, blackboard collaborate, Google Forms, Textbooks, YouTube, WhatsApp

Rodrigues et al. (2022)/Brazil Third and fourth-year undergraduate medical students (n=90) Google Meet
  1. Topic: Pathology

  2. Activities: Live lectures, case-based discussions, formative assessments, games on duty of doubts sessions, asynchronous activities (recorded lectures, question tutorials)

  1. Type: Mandatory

  2. Course length: Not reported

  3. Session details: Weekly synchronous activities (2 h), asynchronous activities on moodle (LMS), evaluation at end of each module (pass/fail) based on tasks uploaded to Turnitin, and synchronous participation

  4. Resources: Moodle (LMS), Turnitin, Textbooks, scientific articles, Kahoot (virtual game)

Samueli et al. (2020)/Israel Undergraduate medical students (n=59) Zoom
  1. Topic: Diagnostic pathology

  2. Activities: Diagnostic assignments using digitized slides with quizzes, lectures on zoom, required and optional readings, review sessions

  1. Type: Elective

  2. Course length: 2 weeks

  3. Session details: 8 modules

  4. Resources: Digitized slides-whole slide images, moodle (LMS); virtual private network service and remote desktop access; zoom recordings

Skrzypek et al. (2020)/Poland Medical students from multiple years (n=not reported) Microsoft Teams
  1. Topics: First aid; history and physical examination; advanced history and physical examination; trauma care; clinical science (each taught as separate courses)

  2. Activities: Lectures, simulations of vital signs and venipuncture, practicing physical assessment at home, virtual simulations, role-playing with a standardized patient, debriefing; problem-based learning

  1. Type: Mandatory

  2. Course length: 1 semester

  3. Session details: Not reported

  4. Resources: Films showing procedures, recordings of auscultatory sounds, standardized patients, prepared scenarios, simulated monitor

Uraiby et al. (2022)/UK Fourth-year undergraduate medical students (n=90) Blackboard collaborate
  1. Topics: Histopathology

  2. Activities: Simulation in history taking, interpreting annotated virtual histology slides, online discussions, polls, remote synchronous tutor facilitated discussion sessions with a multidisciplinary team

  1. Type: Mandatory

  2. Course length: Not reported

  3. Session details: Groups of 12–15 students in a virtual learning environment, 4–6 h for asynchronous work (4–6 h), synchronous sessions (2–3 h)

  4. Resources: Google classroom for asynchronous participation, Hamamatsu NanoZoomer S210 for digital slide scanning, Phillips Xplore for virtual slide viewing

  1. VC, videoconferencing.

Key findings

A summary of the key findings is provided in Appendix V. VCP in response to the COVID-19 pandemic was generally evaluated positively (Behrends et al., 2020; Darici et al., 2021; Guiter et al., 2021; Nikas et al., 2022; Rodrigues et al., 2022; Samueli et al., 2020; Skrzypek et al., 2020; Uraiby et al., 2022). Students were mostly satisfied with VCP as they perceived benefits, including unlimited access to lectures and flexibility in study time and place (Rodrigues et al., 2022; Samueli et al., 2020). Although students’ satisfaction with videoconferencing itself was high, relative satisfaction with VCP compared to face-to-face education before the COVID-19 pandemic was not evaluated very positively (Nikas et al., 2022; Rodrigues et al., 2022). Students described excessive workloads (Behrends et al., 2020), technical difficulties (Guiter et al., 2021; Samueli et al., 2020), and low motivation (Rodrigues et al., 2022; Uraiby et al., 2022). Perceived stress for VCP was higher among students who attended classes outside their home country or those with poor or average internet connections (Nikas et al., 2022). Student performance as assessed by instructors met or exceeded pre-defined competencies or learning outcomes (Darici et al., 2021; Guiter et al., 2021; Samueli et al., 2020). When assessed, knowledge acquisition was reported to have significantly increased after course completion (Hernandez et al., 2021; Uraiby et al., 2022), and no significant difference was reported between face-to-face classroom education and VCP (Hernandez et al., 2021).

Kirkpatrick’s outcome evaluation

All the studies measured student response to the VCP (Level 1) through survey feedback on a course evaluation. Guiter et al. (2021) evaluated student feedback on satisfaction, engagement, interaction, and overall course experience of and Nikas et al. (2022) reported responses regarding motivation, perceived stress, engagement, and course quality gathered from student feedback. Seven of the nine studies (78%) evaluated student learning according to student knowledge level (Level 2) assessed through tests such as quizzes, written examinations, and practical examinations. None of the studies evaluated the higher levels of learning outcomes required to evaluate behaviors (Level 3) or results (Level 4). A description of each study’s evaluative tools and levels, as well as relevant findings, can be found in the Supplementary Materials (see Appendix V).

Facilitators and barriers

The development of a learning platform was identified as a factor that can potentially affect the implementation of VCP. Synchronous learning platforms such as Zoom and Microsoft Teams were able to deliver most of the learning activities traditionally offered in face-to-face education (Skrzypek et al., 2020). Some of the benefits of VCP, such as a flexible environment with less constraint on time and space, contributed to positive learning experiences for both students and instructors (Rodrigues et al., 2022; Samueli et al., 2020). Students indicated that breakout sessions were useful if the group was small and the session lasted less than 15–20 min (Darici et al., 2021). Students appreciated the integration of active learning strategies such as role playing, anonymous questions/answers, and games (Guiter, 2021; Rodrigues et al., 2022); the ability to review recordings of lectures (Samueli et al., 2020); and the opportunity to disable their cameras (Darici et al., 2021).

An analysis of the included studies identified several barriers to implementing videoconferencing in health professions education including (a) technical issues, (b) lack of human interaction, and (c) limitations on some skills training. Technical difficulties included loss of internet and website connectivity (Darici et al., 2021; Samueli et al., 2020), which contributed to course dissatisfaction (Guiter et al., 2021; Samueli et al., 2020) and stress (Nikas et al., 2022). Some students missed having in-person interactions in a physical classroom (Rodrigues et al., 2022; Samueli et al., 2020) and found it more difficult to stay motivated and to separate study activities from home responsibilities (Rodrigues et al., 2022). Courses requiring face-to-face delivery, such as skills training or skills tests, were not replaced by videoconferencing because of logistic and technical issues. For example, objective structured clinical examination and high-fidelity simulation were canceled during the COVID-19 pandemic (Skrzypek et al., 2020).

Discussion and implications for education

The students surveyed in the studies included in this review described their VCP experiences positively, similar to results reported in other studies of VCP during the COVID-19 pandemic with 5th-year medical students in a pediatric rotation (Fatani, 2020), dental students (Amir et al., 2020), and post-secondary non-health professions students (Nikou, 2021; Van Wart et al., 2020). In contrast, Vandenberg and Magnuson (2021) found nursing students and faculty far less satisfied with learning on a videoconferencing platform, reporting that only 12% of students enjoyed using Zoom. Other studies of post-secondary student perceptions have described mixed feelings among students. In a study by Serhan (2020) of student attitudes towards Zoom, 48% disagreed with the statement that videoconferencing was enjoyable, and 55% disagreed with the statement that they would like to use Zoom in other courses. In both the studies by Serhan (2020) and Vandenberg and Magnuson (2021), students were more likely to describe a preference for in-person learning when asked to compare VCP with pre-pandemic in-person education.

Videoconferencing facilitators

Despite mixed perceptions of VCP, learners appreciated its unlimited access to lectures and course materials as well as the flexibility afforded regarding when and where to study. This was true for students who were satisfied with VCP (Rodrigues et al., 2022; Samueli et al., 2020) as well as among students who indicated that they did not enjoy using Zoom (Serhan, 2020; Vandenberg & Magnuson, 2021). Van Wart et al. (2020) explored factors leading to satisfaction in online classes. Their regression analysis suggests that the social components of online learning are less important to student satisfaction than functional or structural course aspects that assist learning (e.g., organization of the course, clarity of presentations, course materials, access to instructors, quality of feedback). Most of the studies reviewed described use of an LMS, sophisticated course materials, and other strategies to engage learners and provide feedback.

Videoconferencing barriers

The challenges with VCP reported by students in the studies in our review have been highlighted in other studies (Candarli & Yuksel, 2012; Vandenberg & Magnuson, 2021). For example, Vandenberg and Magnuson (2021) reported that 52% of students in their study faced technological barriers with Zoom. Interestingly, multiple studies in our review included the use of virtual microscopy, a technology requiring high-speed broadband, and students did report lags and connectivity issues. For example, Guiter (2021) described a loss of internet in 1 of 24 sessions and a loss of virtual microscopy website connectivity during 2 of 24 sessions, suggesting the need for alternative plans when connectivity issues arise during synchronous class meetings. Similarly, Darici (2021) described the need for real-time course technical support. Connectivity was an important consideration for many learners. McLean (2021) described surveys conducted in the United States in which 57% of college students felt that obtaining access to stable, high-speed internet would prove challenging if courses continued to be offered only online. Although some of the studies in our review identified connectivity issues, the source of internet issues did not appear to be explored in detail.

Decreased motivation due to lack of feedback from instructors and peers has been identified as one explanation for decreased enrollment in higher education during the COVID-19 pandemic (McKenzie, 2021). In the studies in our review, lack of motivation was mentioned as a challenge for some students (Nikas et al., 2022; Rodrigues et al., 2022). Students who scored less than 80% on their final examination reported less engagement and more difficulty studying in an online course (Nikas et al., 2022). Skrzypek et al. (2020) reported that students requested more feedback during remote simulation exercises.

Recent research has demonstrated that VCP allows teachers to engage learners and create opportunities for student interaction (Mehta & Aguilera, 2020) through approaches such as case-based discussion (Fatani, 2020) and flipped classroom (Hew et al., 2020). Studies in our review reported that students appreciated active learning strategies such as role playing and games (Guiter, 2021; Rodrigues et al., 2022). Uraiby et al. (2022) indicated that synchronous sessions created a sense of collaboration and alleviated pre-existing anxieties. Interestingly, recent research indicates that teacher presence and interactive activities are more important determinants of student satisfaction and course effectiveness then technical challenges if students feel engaged in the course (Almarzooq et al., 2020; Fatani, 2020).

Although there is evidence that synchronous remote learning through videoconferencing promotes engagement (Mehta & Aguilera, 2020), there was concern during the pandemic that students who chose to leave their cameras off during meetings would minimize this benefit (Castelli & Sarvary, 2021). Reasons stated by students for leaving their cameras off included increased stress from personal appearance, the need to guard the privacy of their personal spaces, and increased distraction from seeing classmates’ backgrounds (Castelli & Sarvary, 2021; Nicandro et al., 2020). Requiring students to have their cameras on to enhance engagement can be especially difficult for minoritized students and can lead to digital inequity and exclusion (Nicandro et al., 2020). Numerous studies have recommended flexibility and clear guidelines related to camera usage as well as strategies to promote engagement without requiring the use of cameras (Castelli & Sarvary, 2021; Hogan & Sathy, 2020; Mehta & Aguilera, 2020; Nicandro et al., 2020). In our review, students surveyed in the study by Darici et al. (2021) were appreciative of opportunities to leave their camera off.

Perceived stress was explored in one of the studies we reviewed (Nikas et al., 2020) and has been identified in other studies as a barrier to learning (Castelli & Sarvary, 2021; Vandenberg & Magnuson, 2021). Nikas et al. (2022) reported high perceived stress with videoconferencing among students who attended outside their home country or with poor or average internet connections. In addition to these stress-inducing factors, Vandenberg and Magnuson (2021) reported that in their study, 76% of students felt disconnected from peers and instructors, and 46% felt increased stress or anxiety with VCP. Several studies have noted that students missed the opportunity to have in-person encounters in a classroom (Rodrigues et al., 2022; Samueli et al., 2020).

During the pandemic, the conversion of clinical skills training and evaluation techniques presented a unique challenge. A majority of studies in this review were focused on histology or pathology and entailed similar technology. Teaching strategies were generally evaluated positively by students (Darici et al., 2021; Guiter et al., 2021; Hernandez et al., 2021; Nikas et al., 2022; Rodrigues et al., 2022; Samueli et al., 2020; Uraiby et al., 2022), and resulted in positive learning outcomes (Darici et al., 2021; Guiter et al., 2021; Samueli et al., 2020; Uraiby et al., 2022). It appears that histology and pathology are content that is well suited for VCP; however, the study by Skrzypek et al. (2020) evaluated courses covering a wide range of content for medical students including first aid, history and physical examination, trauma care, and advanced physical examination. Students were largely satisfied with the courses but requested more feedback when learning hands-on skills (e.g., physical assessment) and were reluctant to participate in teaching strategies that required them to practice physical assessment skills on their friends and family (Skrzypek et al., 2020). The literature contains reports of innovative strategies for teaching clinical nursing skills online including the use of video podcasting (Stone et al., 2019), open-source simulations, virtual patient encounters, and assessment technology products (Konrad et al., 2021). These resources, used together with debriefing in breakout groups, chat, and polling as part of VCP, have been described as useful in supporting clinical competency (Konrad et al., 2021).

State of course evaluation

The studies in this review assessed learners’ reactions to educational offerings as well as their growth in knowledge, attitude, confidence, and commitment (Kirkpatrick levels 1 & 2); however, demonstration of how courses were received and whether key knowledge and skills were learned does not confirm that knowledge has been transferred into behaviors or outcomes of value to the learner’s organization or future career path (Kirkpatrick levels 3 & 4). A full demonstration that VCP is as effective as in-person instruction would require these additional levels of evaluation (Kirkpatrick et al., 2016). This assertion is not meant as a criticism of the evaluative outcomes identified in the reviewed studies; in each case, outcomes measures appeared to align with the course objectives. Rather, we suggest that an evaluation of whether the outcomes of VCP translate into behavioral practices for all students and meet professional standards of competency is important as VCP appears destined to become a mainstay in health professions education. The original Kirkpatrick model has been criticized for making evaluation of Levels 3 and 4 difficult, but the revised version provides a broader perspective of these evaluative steps (Moreau, 2017).

Inclusive pedagogy and the digital divide

Inclusive pedagogy involves humanizing the learner’s experience by intentionally creating activities and materials that are sensitive to inclusivity, diversity, and accessibility (Mehta & Aguilera, 2020). A variety of teaching strategies can be used to meet diverse learner needs and incorporate UDL principles, many of which were utilized in the courses described in this review; however, there was no evidence of intentional selection based on inclusivity, diversity, and accessibility needs. Moreover, although most of the studies included described the impact of internet connectivity on course satisfaction by noting challenges related to access to technology, how these challenges were related to learning outcomes was left largely unexplored. Sieck et al. (2021) have developed a potentially useful framework for addressing this need; their approach evaluates a health care system’s shift to virtual care during the pandemic with consideration of the Five A’s of access for technology equity: (1) availability, (2) accessibility, (3) accommodation, (4) affordability, and (5) acceptability. The access criteria most frequently identified the studies in our review was acceptability, defined by Sieck et al. (2021) as the end user’s attitude toward and comfort with tools and workflows. Most of the studies we reviewed evaluated the overall acceptability of videoconferencing, its tools (i.e., virtual microscopy, breakout rooms, chat, camera usage), and workflows (i.e., ability to ask questions, getting feedback, viewing recordings). As the remaining elements of the access framework were not addressed in these studies, a gap in our understanding about these important issues persists.

Availability, described by Sieck et al. (2021) as the relationship between the services offered and the needs and abilities of the end user, was not directly evaluated by the studies. Although some students identified that they were less motivated, engaged, or confident (Nikas et al., 2022; Rodrigues et al., 2022; Uraiby et al., 2022), the etiology of this problem was not investigated. Lack of motivation or engagement could have been related to (a) the unavailability of services or resources to meet a student’s particular needs or abilities, (2) the student’s inability to navigate the platform and resources (accommodation), or (3) the student’s limited digital skills and literacy (accessibility). Likewise, internet connectivity or technical difficulty was described as a challenge for at least some students in several of the studies reviewed (Guiter et al., 2021; Nikas et al., 2022; Samueli et al., 2020); however, these difficulties were not delineated, making it impossible to know whether they were related to affordability (ability to afford high-speed broadband), availability (lack of broadband in their geographic location), or accessibility (adequate technical skills) (Sieck et al., 2021). As noted by McKenzie (2021), there are many layers of the digital divide that impact postsecondary educational success. To address these barriers, educators must first identify them. Given higher education’s commitment to addressing existing inequalities among the student population, one might expect that the Five A’s of access would be closely assessed during course evaluation.

Application to nursing education

Though nurse educators are proficient or even experts in other pedological applications and teaching strategies, VCP is unique and necessitates educators invest in their professional development to acquire the knowledge, skills, and attitudes to effectively teach using this approach. Brenner’s Novice to Expert Model (2009) applies not only to nursing students but also to the importance of ongoing professional development for nurses and nurse educators when they are placed in a situation in which they have little experience. As experienced during COVID-19, nurse educators suddenly found themselves as “novice” using videoconferencing methods for the first time. Brenner’s model not only validates the need for nurse educator’s professional development but also a structure for building levels of mastery. This approach was successfully demonstrated by Thomas and Kellgren (2017) in their program to prepare faculty to use simulation. In addition to prioritizing VCP professional development for nurse educators, the lack of nursing studies in this review highlights a gap in the profession’s scholarly evaluation of VCP in nursing education. While the experience of other health professions research is useful when designing and implementing interdisciplinary curriculum, further research is needed to evaluate nurse educators’ knowledge and practice of VCP, as well as outcomes of VCP in nursing education. In addition, the use and effectiveness of teaching strategies based on diversity, inclusivity, and accessibility needs to be explored by nurse scholars.

Limitations

This review had several limitations. The search identified only a small number of studies, and as these pertained only to medical students, there is a possibility of selection bias and limited generalizability to other health disciplines such as pre-licensure nursing students. The inability to include studies published in non-English languages could lead to increased publication bias and decreased applicability of the findings. There are likely socio-cultural factors that may affect the ways in which students from non-Western backgrounds experience VCP, thus the interpretation of findings and implication for practice should be considered accordingly.

Conclusions

The aim of this systematic review was to explore evidence of the use of VCP in response to the COVID-19 pandemic with a view toward digital equity and inclusion. Specifically, we sought to understand facilitators and barriers to and outcomes and practice implications of VCP viewed through the lenses of humanizing pedagogy and UDL. We expect our findings to (a) contribute to bridging the VCP knowledge-practice gap in nursing education, and (b) assist educators globally to broaden their ideas for designing inclusive VCP curricula that provide all students with equal opportunities to learn. The future of videoconferencing as a teaching tool holds challenges and new opportunities for nurses and health professions educators across the globe. Future research should consider the importance of the interplay between educators’ attitudes toward and knowledge and practice of UDL guidelines as well as VCP student learning outcomes and competencies to the creation of equitable, inclusive, and accessible learning spaces.


Corresponding author: Dukyoo Jung, PhD, RN, Professor, College of Nursing, Ewha Womans University, Seoul, South Korea, Phone: 82 10 4310 4522, Fax: 82 2 3277 2875, E-mail:

Acknowledgments

The authors thank Donnalee Frega, PhD, for editorial assistance.

  1. Research funding: This was funded by a 2020–2022 Duke Learning Innovation’ Carry the Innovation Forward Grant awarded to JCD.

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Competing interests: Authors state no conflict of interest.

  4. Informed consent: Not applicable.

  5. Ethical approval: The authors’ institutional review of board (IRB) has determined that the protocol of this study (ID No. Pro00108625) meets the definition of research not involving human subjects as described in 45 CFR 46.102(f), 21 CFR 56.102(e) and 21 CFR 812.3(p) and satisfies the Privacy Rule as described in 45CFR164.514.

References

Almarzooq, Z. I., Lopes, M., & Kochar, A. (2020). Virtual learning during the COVID-19 pandemic. Journal of the American College of Cardiology, 75(20), 2635–2638. https://doi.org/10.1016/j.jacc.2020.04.015 Search in Google Scholar PubMed PubMed Central

Amir, L. R., Tanti, I., Maharani, D. A., Wimardhani, Y. S., Vera, J., Sulijaya, B., & Puspitawati, R. (2020). Student perspective of classroom and distance learning during COVID-19 pandemic in the undergraduate dental study program Universitas Indonesia. BMC Medical Education, 20, 1–8. https://doi.org/10.1186/s12909-020-02312-0 Search in Google Scholar PubMed PubMed Central

Aromataris, E., & Munn, Z. (2020). JBI manual for evidence synthesis. https://jbi-global-wiki.refined.site/space/MANUAL Search in Google Scholar

Behrends, M., Hoffmann, I., Marschollek, M., & HiGHmeducation, C. (2020). Teamwork, communication and exchange despite Covid-19-experiences from a digital elective in human medicine studies as part of the HiGHmed project. GMS Journal for Medical Education, 37(7), 1–6. https://doi.org/10.3205/zma001379.Search in Google Scholar PubMed PubMed Central

Candarli, D., & Yuksel, G. H. (2012). Students’ perceptions of video-conferencing in the classrooms in higher education. Procedia – Social and Behavioral Sciences, 47, 357–361. https://doi.org/10.1016/j.sbspro.2012.06.663 Search in Google Scholar

Castelli, F. R., & Sarvary, M. A. (2021). Why students do not turn on their video cameras during online classes and an equitable and inclusive plan to encourage them to do so. Academic Practice in Ecology and Evolution, 11(8), 3565–3576. https://doi.org/10.1002/ece3.7123 Search in Google Scholar PubMed PubMed Central

Cook, D. A., & Reed, D. A. (2015). Appraising the quality of medical education research methods: The Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education. Academic Medicine: Journal of the Association of American Medical Colleges, 90(8), 1067–1076. https://doi.org/10.1097/ACM.0000000000000786 Search in Google Scholar PubMed

Correia, A. P. (2020). Healing the digital divide during the COVID-19 pandemic. Quarterly Review of Distance Education, 21(1), 13–21.Search in Google Scholar

Correia, A., Liu, C., & Xu, F. (2020). Evaluating videoconferencing systems for the quality of the educational experience. Distance Education, 41(4), 429–452. https://doi.org/10.1080/01587919.2020.1821607 Search in Google Scholar

Cotán, A., Aguirre, A., Morgado, B., & Melero, N. (2021). Methodological strategies of faculty members: Moving toward inclusive pedagogy in higher education. Sustainability, 13(6), 3031. https://doi.org/10.3390/su13063031 Search in Google Scholar

Crawford, J., Butler-Henderson, K., Rudolph, J., Malkawi, B., Glowatz, M., Burton, R., Magni, P., & Lam, S. (2020). COVID-19: 20 countries’ higher education intra-period digital pedagogy responses. Journal of Applied Learning & Teaching, 3(1), 1–20.10.37074/jalt.2020.3.1.7Search in Google Scholar

Darici, D., Reissner, C., Brockhaus, J., & Missler, M. (2021). Implementation of a fully digital histology course in the anatomical teaching curriculum during COVID-19 pandemic. Annals of Anatomy-Anatomischer Anzeiger, 236, 151718. https://doi.org/10.1016/j.aanat.2021.151718 Search in Google Scholar PubMed PubMed Central

Digital Future Society. (2019). Bridging digital divides: A framework for digital cooperation. https://futuribile.org/project/digital-cooperation-to-bridge-digital-divides/ Search in Google Scholar

Fatani, T. H. (2020). Student satisfaction with videoconferencing teaching quality during the COVID-19 pandemic. BMC Medical Education, 20(1), 396. https://doi.org/10.1186/s12909-020-02310-2 Search in Google Scholar PubMed PubMed Central

González-Betancor, S. M., López-Puig, A. J., & Cardenal, E. M. (2021). Digital inequality at home. The school as compensatory agent. Computers & Education, 168, 104195. https://doi.org/10.1016/j.compedu.2021.104195.Search in Google Scholar

Guiter, G. E., Sapia, S., Wright, A. I., Hutchins, G. G., & Arayssi, T. (2021). Development of a remote online collaborative medical school pathology curriculum with clinical correlations, across several international sites, through the Covid-19 pandemic. Medical Science Educator, 31(2), 549–556. https://doi.org/10.1007/s40670-021-01212-2 Search in Google Scholar PubMed PubMed Central

Haynes, R. B., Richardson, W. S., Rosenberg, W., & Sackett, D. L. (1997). Evidence based medicine: How to practice and teach EBM. Churchill Livingstone.Search in Google Scholar

Hebda, T., Hunter, K., & Czar, P. (2019). Handbook of informatics for nurses & healthcare Professionals (6th ed.). Pearson.Search in Google Scholar

Hernandez, T., Fallar, R., & Polydorides, A. D. (2021). Outcomes of remote pathology instruction in student performance and course evaluation. Academic Pathology, 8, 1–9. https://doi.org/10.1177/23742895211061822.Search in Google Scholar PubMed PubMed Central

Hew, K. F., Jia, C., Gonda, D. E., & Bai, S. (2020). Transitioning to the “new normal” of learning in unpredictable times: Pedagogical practices and learning performance in fully online flipped classrooms. International Journal of Educational Technology in Higher Education, 17(1), 57. https://doi.org/10.1186/s41239-020-00234-x Search in Google Scholar PubMed PubMed Central

Hogan, K. A., & Sathy, V. (2020). 8 Ways to be more inclusive in your zoom teaching. Chronical of Higher Education. https://www.chronicle.com/article/8-ways-to-be-more-inclusive-in-your-zoom-teaching/ Search in Google Scholar

Iturbe-LaGrave, V., Sponsler, L. E., Paguyo, C. H., & Alvarez, L. D. C. (2021). The pedagogical imperative of changing the culture of teaching. Journal of the Professoriate, 12(1), 150–180.Search in Google Scholar

June, A. J. (2020). The unequal costs of the digital divide. The Chronicle of Higher Education, 67(4). https://www.chronicle.com/article/the-unequal-costs-of-the-digital-divide Search in Google Scholar

Kirkpatrick, J. D., & Kirkpatrick, W. K. (2016). Kirkpatrick’s four levels of training evaluation. Association for Talent Development.Search in Google Scholar

Konrad, S., Fitzgerald, A., & Deckers, C. (2021). Nursing fundamentals – Supporting clinical competency online during the COVID-19 pandemic. Teaching and Learning in Nursing, 16(1), 53–56. https://doi.org/10.1016/j.teln.2020.07.005 Search in Google Scholar PubMed PubMed Central

McGonigle, D., & Mastrian, K. (2021). Nursing informatics and the foundation of knowledge (5th ed.). Jones & Bartlett Learning.Search in Google Scholar

McKenzie, L. (2021). Bridging the digital divide: Lessons from COVID-19. https://www.insidehighered.com/content/bridging-digital-divide-lessons-covid-19 Search in Google Scholar

McLean, D. (2021). Failed connection: The broadband gap. The Chronicle of Higher Education. https://connect.chronicle.com/rs/931-EKA-218/images/FailedConnectionInisghtsReport.pdf Search in Google Scholar

Mehta, R., & Aguilera, E. (2020). A critical approach to humanizing pedagogies in online teaching and learning. International Journal of Information & Learning Technology, 37(3), 109–120. https://doi.org/10.1108/ijilt-10-2019-0099 Search in Google Scholar

Moreau, K. A. (2017). Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Medical Teacher, 39(9), 999–1001. https://doi.org/10.1080/0142159X.2017.1337874.Search in Google Scholar PubMed

Nicandro, V., Khandelwal, A., & Weitzman, A. (2020). Please, let students turn their videos off in class. The Stanford Daily. https://www.stanforddaily.com/2020/06/01/please-let-students-turn-their-videos-off-in-class/ Search in Google Scholar

Nikas, I. P., Lamnisos, D., Meletiou-Mavrotheris, M., Themistocleous, S. C., Pieridi, C., Mytilinaios, D. G., Michaelides, C., & Johnson, E. O. (2022). Shift to emergency remote preclinical medical education amidst the COVID-19 pandemic: A single-institution study. Anatomical Sciences Education, 15(1), 27–41. https://doi.org/10.1002/ase.2159 Search in Google Scholar PubMed PubMed Central

Nikou, S. A. (2021). Web-based videoconferencing in online teaching during the COVID-19 pandemic: University students’ perspectives. In 2021 International Conference on Advanced Learning Technologies (pp. 431–435).10.1109/ICALT52272.2021.00137Search in Google Scholar

Pacansky-Brock, M., Smedshammer, M., & Vincent-Layton, K. (2020). Humanizing online teaching to equitize higher education. Current Issues in Education, 21(2), 1–21.Search in Google Scholar

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 Search in Google Scholar PubMed PubMed Central

Rodrigues, M., Zornoff, D., & Kobayasi, R. (2022). Remote pathology teaching under the COVID-19 pandemic: Medical students’ perceptions. Annals of Diagnostic Pathology, 56, 151875. https://doi.org/10.1016/j.anndiagpath.2021.151875 Search in Google Scholar PubMed PubMed Central

Samueli, B., Sror, N., Jotkowitz, A., & Taragin, B. (2020). Remote pathology education during the COVID-19 era: Crisis converted to opportunity. Annals of Diagnostic Pathology, 49, 151612. https://doi.org/10.1016/j.anndiagpath.2020.151612 Search in Google Scholar PubMed PubMed Central

Serhan, D. (2020). Transitioning from face-to-face to remote learning: Students’ attitudes and perceptions of using Zoom during COVID-19 pandemic. International Journal of Technology in Education and Science (IJTES), 4(4), 335–342. https://doi.org/10.46328/ijtes.v4i4.148 Search in Google Scholar

Sieck, C. J., Rastetter, M., Hefner, J. L., Glover, A. R., Magaña, C., Gray, D. M., Joseph, J. J., Panchal, B., & Olayiwola, J. N. (2021). The five A’s of access for techQuity. Journal of Health Care for the Poor and Underserved, 32(2S), 290–299. https://doi.org/10.1353/hpu.2021.0064 Search in Google Scholar

Skrzypek, A., Perera, I., Szeliga, M., & Cebula, G. (2020). Modification of teaching during the COVID-19 pandemic at the Department of Medical Education of Jagiellonian University Medical College. Folia Medica Cracoviensia, 60(4), 113–121.Search in Google Scholar

Stamler, L. L., & Upvall, M. J. (2021). Global inequity in nursing education: A call to action. International Journal of Nursing Education Scholarship, 18(1), 20210131. https://doi.org/10.1515/ijnes-2021-0131.Search in Google Scholar PubMed

Stone, R., Cooke, M., & Mitchell, M. (2019). Undergraduate nursing students’ use of video technology in developing confidence in clinical skills for practice: A systematic integrative literature review. Nurse Education Today, 84, 104230. https://doi.org/10.1016/j.nedt.2019.104230.Search in Google Scholar PubMed

SUNY Online. (2021). Remote teaching vs. online teaching. https://onlineteaching.open.suny.edu/page/remote-teaching Search in Google Scholar

Thomas, C. M., & Kellgren, M. (2017). Benner’s novice to expert model: An application for simulation facilitators. Nursing Science Quarterly, 30(3), 227–234. https://doi.org/10.1177/0894318417708410 Search in Google Scholar PubMed

UNESCO. (2020). COVID-19 educational disruption and response. https://en.unesco.org/news/covid-19-educational-disruption-and-response Search in Google Scholar

Uraiby, H., Grafton-Clarke, C., Gordon, M., Sereno, M., Powell, B., & McCarthy, M. (2022). Fostering intrinsic motivation in remote undergraduate histopathology education. Journal of Clinical Pathology, 75(12), 837–843. https://doi.org/10.1136/jclinpath-2021-207640.Search in Google Scholar PubMed

Van Wart, M., Ni, A. Y., Ready, D., & Shayo, C. (2020). Factors leading to online learner satisfaction. Business Education Innovation Journal, 12(1), 14–24.Search in Google Scholar

Vandenberg, S., & Magnuson, M. (2021). A comparison of student and faculty attitudes on the use of Zoom, a video conferencing platform: A mixed-methods study. Nurse Education in Practice, 54, 103138. https://doi.org/10.1016/j.nepr.2021.103138 Search in Google Scholar PubMed PubMed Central

Williamson, B., Eynon, R., & Potter, J. (2020). Pandemic politics, pedagogies and practices: Digital technologies and distance education during the coronavirus emergency [Editorial]. Learning, Media and Technology, 45(2), 107–114. https://doi.org/10.1080/17439884.2020.1761641 Search in Google Scholar


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/ijnes-2022-0068).


Received: 2022-06-26
Accepted: 2022-11-18
Published Online: 2023-01-05

© 2022 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 9.12.2023 from https://www.degruyter.com/document/doi/10.1515/ijnes-2022-0068/html
Scroll to top button