Efficacy of implementing intermittent STOP THE BLEED® reviews on long term retention of hemorrhage control skills of first year medical students.

CONTEXT
Some medical schools integrate STOP THE BLEED® training into their curricula to teach students how to identify and stop life threatening bleeds; these classes that are taught as single day didactic and hands-on training sessions without posttraining reviews. To improve retention and confidence in hemorrhage control, additional review opportunities are necessary.


OBJECTIVES
To investigate whether intermittent STOP THE BLEED® reviews were effective for long term retention of hemorrhage control skills and improving perceived confidence.


METHODS
First year osteopathic medical students were asked to complete an eight item survey (five Likert scale and three quiz format questions) before (pretraining) and after (posttraining) completing a STOP THE BLEED® training session. After the surveys were collected, students were randomly assigned to one of two study groups. Over a 12 week intervention period, each group watched a 4 min STOP THE BLEED® review video (intervention group) or a "distractor" video (control group) at 4 week intervals. After the 12 weeks, the students were asked to complete an 11 item survey.


RESULTS
Scores on the posttraining survey were higher than the pretraining survey. The median score on the five Likert scale items was 23 points for the posttraining survey and 14 points for the pretraining survey. Two of the three knowledge based quiz format questions significantly improved from pretraining to posttraining (both p<0.001). On the 11 item postintervention survey, both groups performed similarly on the three quiz questions (all p>0.18), but the intervention group had much higher scores on the Likert scale items than the control group regarding their confidence in their ability to identify and control bleeding (intervention group median = 21.4 points vs. control group median = 16.8 points).


CONCLUSIONS
Intermittent review videos for STOP THE BLEED® training improved medical students' confidence in their hemorrhage control skills, but the videos did not improve their ability to correctly answer quiz-format questions compared with the control group.

Medical students are expected to develop hemorrhage control skills pertaining to basic wound management and traumatic injury as a core competency during their emergency medicine rotation [1]. Additionally, recognizing emergent situations and applying basic life support skills are considered core competencies needed for entering residency [2,3]. For this reason, integrating STOP THE BLEED ® training into medical school education is critical because it provides medical students with a practice based learning approach that prepares them for possible mass casualty events and contributes to their development as healthcare providers and community leaders. STOP THE BLEED ® training teaches participants how to effectively intervene during bleeding emergencies; approximately 1.5 million individuals around the world have completed this training [4]. These training sessions generally consist of a didactic session and a hands-on practice session. In one study [5], 92% (n=348) of medical students reported that they felt comfortable using the skills learned during a proprietary bleeding control basics course. Although emergency hemorrhage control training can be effectively integrated into the medical school curriculum, student confidence and retention of such skills after an extended amount of time are rarely assessed.
Research suggests retention of skills is improved by refresher training [6]. In an emergency nursing study [7], retention of tourniquet application skills was assessed after participation in a bleeding control course. After initial training, 100% (n=46) of participants successfully demonstrated tourniquet skills. After 6 months, 39% (18 of 46) were unable to successfully apply a tourniquet, and 26% (12 of 46) were unable to control life-threatening bleeding. These data suggest that regular review training should be implemented to increase retention. Research has also suggested that current hemorrhage control training should be improved for better retention; a previous randomized controlled trial [8] showed high rates of failure in tourniquet application in both the intervention (79%; 167) and control (81.9%; 138) groups even after successfully completing the tourniquet training. Although at least one prior study has investigated emergency bleeding training [9], to our knowledge, no prior studies have examined specifically the efficacy of refresher courses on increasing long-term retention and confidence in medical students. Because acquired skills can be forgotten over time, reviews of those skills could be beneficial.
When assessing life-saving skills like hemorrhage control, it is more important to consider long term outcomes rather than short term. The ability to show mastery and improved confidence in learned skills is ultimately the goal. But "long term" does not have a clear definition in the literature. Some medical educations consider long term as 4 weeks [10] or 15 weeks [11]. Others define it as 5 months [12] or 6 months [13][14][15][16]. For this reason, we elected to consider a timeframe of 1 month to 6 months as long term.
Also, knowledge alone does not lead to life saving moments; rather, a bystander's confidence and willingness to act also have important roles in emergency situations. In a study of STOP THE BLEED ® training for laypeople [17], participants reported improved confidence and willingness to use a tourniquet (before training, 64.2% [140 of 218]; after training, 95.6% [194 of 203]). One way to improve people's likelihood of taking action is to increase their self confidence and perceived self efficacy in hemorrhage control skills [18]. Therefore, we investigated whether intermittent STOP THE BLEED ® reviews were effective for long term retention of hemorrhage control skills and improvements in perceived confidence. We hypothesized that the intervention group would have higher confidence levels and higher scores on quiz questions than the control group.

Participants
The current study used a pre-post survey design, and participants were first year osteopathic medical students at A.T. Still University School of Osteopathic Medicine in Arizona (ATSU-SOMA). The study was conducted from February to June 2020. As part of the curriculum, all students from a single class completed a STOP THE BLEED ® training and a simulation day, spaced 1 week apart. As part of the study, students completed surveys immediately before the training (pretraining), immediately after the training (posttraining), and after a 12 week intervention period (postintervention). The pretraining and posttraining surveys were completed on paper, and responses were later entered into a spreadsheet. The postintervention survey was completed electronically. No data were collected on the simulation day.
Students were invited to participate in the study by announcement when they arrived for STOP THE BLEED ® training. Students who agreed to participate were given the pretraining and posttraining surveys. After completion of the training, students were assigned to one of two intervention groups, sorted randomly by ATSU-SOMA's curriculum support manager, and were sent an e-mail inviting them to participate in the intervention part of the study. The randomization process consisted of putting all the participants onto a spreadsheet, randomizing (shuffling) the entire list, then splitting participants into the two groups (intervention and control); the top half of the list was assigned to the control group, while the bottom half was assigned to the intervention group. The e-mail invitation described the purpose of the study and included a link to one of two videos (intervention or control). By clicking on the link, participants were informed that they were affirming their consent to participate. Participation was voluntary, and all survey responses were collected anonymously. Moreover, students were informed that participation or nonparticipation would not impact their grades, and no compensation was provided for participation. The institutional review board at ATSU in Arizona deemed the current study exempt.

STOP THE BLEED ® training day
STOP THE BLEED ® training was included as part of the cardiopulmonary curriculum for first year osteopathic medical students at ATSU-SOMA. The training consisted of a didactic portion delivered in a classroom setting, followed by hands-on practice in smaller groups with guidance from trained volunteers. The entire training lasted approximately 3 h.
On the day of the training session, students completed an 8-question survey (Supplemental Material) immediately before (pretraining) and after the training (posttraining). The two surveys were identical and designed specifically for the current study to assess participants' confidence and ability to correctly answer questions before and after emergency hemorrhage control training. Although STOP THE BLEED ® training was mandatory, students were informed that completing these surveys was optional.
Five of the eight questions used a 5 point Likert scale, where 1 was "strongly disagree" and 5 was "strongly agree." These five items included the following: (1) At the present time, I know how to identify life threatening bleeding; (2) At the present time, I feel confident in my ability to limit life threatening bleeding; (3) At the present time, I know how to properly apply a tourniquet to stop bleeding; (4) At the present time, I know how to properly pack a wound to control bleeding; and (5) If there was a serious bleeding incident at this very moment, I would step in and take action. Based on the wording of these Likert scale items, a response with a greater value implied higher confidence in hemorrhage control than response with a lower value.
The remaining three questions were presented in a quiz format to assess student competency in hemorrhage control knowledge and the correct steps to stop a bleed. The questions were written to have one correct answer. The specific questions were: (1) Before attempting to stop a life threatening bleed, what is the first step? (Answer: attempt to remove yourself and victim, if possible, from danger); (2) Where should you apply the tourniquet to stop a life threatening bleed? (Answer: about 2-3 inches above the bleeding site); and (3) Which is an example of a properly applied tourniquet? (Answer: tightened to a point that causes pain and lack of perfusion).

STOP THE BLEED ® simulation day
One week after the STOP THE BLEED ® training, students were required to attend a simulation day as part of the curriculum. The simulation was meant to complement the training from the previous week. During the event, teams of four or five students experienced a simulated emergency situation pertaining to trauma and hemorrhage. No data were collected on this day as part of the current study.

Intermittent review training
After completing the STOP THE BLEED ® training and simulation day, participants were randomly assigned to one of two groups as previously described, then provided with email links to video review training. Each group watched a 4 min STOP THE BLEED ® review video (intervention group) or a "distractor" video (control group) at 4 week intervals over 12 weeks. The first video was e-mailed 4 weeks after the simulation day, and subsequent videos were e-mailed twice more every 4 weeks after that. Both groups received identical e-mails with the same instructions; only the link to the assigned video was different.
We used an online platform for the video reviews because it was more flexible and reduced the burden on participants. The STOP THE BLEED ® review video was taught by a trauma surgeon at the University of California San Diego Health and summarized the training in a quick and concise manner [19]. The distractor video was a promotional video for ATSU-SOMA had no information relevant to hemorrhage control training. After the 12 week intermittent review period, participants were sent an e-mail that asked them to answer a final survey (postintervention survey). This 11-item survey included the same the eight questions that were in the pretraining and posttraining surveys, plus three additional questions about the students' actual participation in the study and possible prior knowledge of hemorrhage control (Supplemental Material). These participation questions were designed to determine how often participants watched the assigned video and to evaluate participation levels between the two groups.
All survey questions utilized in the study were written specifically for this project and validated by the same emergency room physician who coordinated the STOP THE BLEED ® training at the institution (C.W.).

Data analysis
Cronbach α was calculated for all five Likert scale items to check the reliability of the survey. Total scores on all five Likert items were calculated for the pretraining and posttraining surveys and were summarized using median and interquartile range (IQR). A Wilcoxon rank sum test was used to compare the median score between pretraining and posttraining surveys. For the postintervention survey, intervention group data was deleted for one participant who didn't attend any training or simulation, five participants who didn't watch the review video, and one participant who did neither of these. No data was deleted from the control group, as all participants in this group attended the training and simulation, so there was no need to exclude participants based on number of videos watched, since the video content was unrelated. Total scores for Likert items on the postintervention survey for the intervention and control groups were also summarized using median (IQR), and scores were compared between groups using a Wilcoxon rank sum test. Survey responses for other questions on all surveys were summarized using frequency and percentage. A χ 2 test was used to compare the percentage of correct responses for the three quiz format survey questions between the pretraining and posttraining surveys and between the intervention and control groups. Scores for all questions on all surveys were also summarized using mean and standard deviation (SD). All analyses were performed using SAS version 9.4 (SAS, Inc.) and p<0.05 was considered statistically significant.

Results
Of a possible 160 first year osteopathic medical students enrolled in this STOP THE BLEED ® program, 153 (95.6%) participated in at least one component of the current study. Of those 153 students, 147 (89.4%) completed the pretraining survey, 153 (100%) completed the posttraining survey, and 100 (65.4%) completed the postintervention survey. Of the 100 students who completed the posttraining survey, we excluded seven students due to not meeting study criteria (not attending both the STOP THE BLEED ® training and simulation, and not watching at least one of the three videos sent to them throughout the 12 week study period). Of the remaining 93 students who completed the postintervention survey and met the inclusion criteria, 44 (47%) students were from the intervention group (who watched at least one of the three videos) and 49 (53%) students were from control group (who may or may not have watched the distractor videos). The Cronbach α for all five Likert scale items was 0.93, which indicated good reliability. Survey responses are depicted in Table 1.
For the five Likert format questions on the postintervention survey, median (IQR) total score for intervention group was significantly higher than that for the control group (intervention group, 21 [20][21][22][23][24][25] vs. control group, 16 [14][15][16][17][18][19][20]; p<0.0001; Table 2). No differences were found in percentage of correct responses between the intervention and control groups for the three quiz format questions on the postintervention survey (all p>0.13). We utilized the three  quiz format questions as an indicator of knowledge in hemorrhage control skills, while the five Likert format questions were meant to indicate students' confidence in their skills and confidence to intervene when faced with an emergency. By this definition, the comparison between the intervention and control groups suggested that the intermittent review opportunities improved confidence but did not improve the retention of the knowledge. Responses for the three additional survey questions (Supplemental Material) after the 12 weeks of review training are reported in Table 3. Most students in both groups attended the STOP THE BLEED ® training and simulation day: 42 (95.5%) from the intervention group and 47 (95.9%) from the control group. The majority of students reported watching two or more of the videos assigned to them: 39 (90%) from the intervention group and 39 (80%) from the control group. Most students reported that they had not been previously taught to control life threatening bleeding: 40 (91%) from the intervention group and 44 (89.8%) from the control group.

Discussion
The first year osteopathic medical students who participated in our study reported improved confidence in hemorrhage control both after the STOP THE BLEED ® training and after the 12 week review training intervention. Most students in both groups attended the STOP THE BLEED ® training and simulation days and most students had not been taught the I feel confident in my ability to limit life threatening bleeding I know how to properly apply a tourniquet to stop bleeding I know how to properly pack a wound to control bleeding If there was a serious bleeding incident at this very moment, I would step in and take action  procedure before, which suggested that our results were not attributable to differences in baseline training. Our inclusion of the three quiz format questions in the survey was intended to measure student ability to correctly choose the steps for proper hemorrhage control, and we found no difference between the intervention and control groups for these questions after the 12 week review training intervention.
Our finding of no difference in the percentage of correct responses for the three quiz format questions between the intervention and control groups after the 12 weeks of review training was unexpected. Although the intervention group had higher mean scores on the three quiz format questions than the control group, the failure of the scoring difference among the two groups to reach the level of statistical significance was unexpected. Further, the intervention group had much higher confidence in their bleeding control abilities and felt more inclined to take action if they encountered an emergency hemorrhage situation. It is unknown why the STOP THE BLEED ® review videos improved student confidence but not the ability to correctly answer the quiz format questions. From these results, we posit two possible explanations: (1) the STOP THE BLEED ® training and simulation was sufficient to deliver the content in a way that was retained long term regardless of intermittent review videos, or (2) confidence was increased in the intervention group because the videos provided further reassurance of their skills.
Because the majority of students attended the STOP THE BLEED ® training and simulation day and because most participants in both groups watched two or more of the review videos, there is likely another factor contributing to the quiz results. It is possible that we did not include enough quiz format questions. With more assessment questions, results may have been different. Although the survey was designed to be completed quickly and reduce participant drop out, a future study should consider including more quiz format questions with wider scope to assess competency over a longer intervention period to make the group differences more robust.

Limitations
Our study had several limitations. For instance, we had a higher attrition rate than expected. The first two surveys had 147 and 153 student responses for the pretraining and posttraining, respectively, but after the 12 weeks of review training, only 44 in the intervention group and 49 in the control group completed the survey. Providing an incentive to complete the survey after the 12 week review training period may improve the response rate in future studies. Another possible limitation is that the timing of the last survey occurred at the end of the academic year when students were completing final exams, preparing to move away for clinical rotations to various cities across the country, and dealing with the burden of the novel coronavirus 2019 pandemic. Without these hardships, our response rate may have been higher.
Self-reporting bias may have been another limitation. Although we expected the survey to be answered truthfully and there was no coercion from study researchers to answer in any particular way, participants completed the surveys without supervision; without proctoring, students could have potentially utilized outside resources to answer knowledge based questions. Other limitations included small sample size and data collected at a single institution, making it difficult to generalize to other populations.
Lastly, the study period should be discussed. Our study used 12 weeks as a defining measure of "long term" retention, but it is difficult to ascertain whether this a valid end point. Based on the literature, the idea of assessing long term retention was widely scattered. In medical education articles, the timeframe ranged from 4 weeks to 6 months [10][11][12][13][14][15][16], but some studies were longer [20,21]. Due to our time constraints with attempting to collect our data before students began moving to their clinical sites, we believed it would be reasonable to conduct our research using 12 weeks as our marker of long-term retention of hemorrhage control abilities.
Despite these limitations, our results highlighted some important points. The intervention and control groups had no statistically significant difference in correct answers to quiz questions. Therefore, having STOP THE BLEED ® trainees participate in additional reviews after training may be more important for improving and maintaining a high confidence level that is crucial during emergency bleeding incidents than for reinforcing knowledge. Additional research should be performed to compare the intervention and control groups longitudinally to determine whether retention rates are still statistically similar.
The idea of the STOP THE BLEED ® training and the protocol of this study is rooted in various osteopathic philosophies. As stated in the tenets of osteopathic medicine, the body is capable of self-regulation, self-healing, and health maintenance, but in the face of trauma or masscasualty events, the body may require additional assistance. STOP THE BLEED ® teaches the skills needed to assist the body in these extreme situations as the maintenance of blood volume is thrown out of homeostatic levels. Furthermore, training in hemostatic control is closely related to the respiratory-circulatory model of care.
This study also draws on the osteopathic tenet that rational treatment is based on an understanding of the basic principles of body unity and the interrelationship of structure and function. Students with higher confidence levels in hemostatic control are more inclined to make rational treatment decisions in life threatening situations and learn from this training that hemostasis is an essential function within the body to be conserved so that it can continue to function for oxygen delivery and other metabolic processes.

Conclusions
Our study investigated whether providing STOP THE BLEED ® review training would improve retention and confidence in hemorrhage control skills among medical students. Although we found no differences in hemorrhage control skills after 12 weeks of review training, our results suggested that review training positively impacted first year osteopathic medical students' confidence in their emergency preparedness. Students in our intervention group reported higher confidence in their hemorrhage control skills and indicated they would be more likely to step in and help during a severe bleeding incident. One of the most important things an emergency responder must do in a life threatening situation is act quickly. By improving student confidence after the intervention, first year medical students may theoretically act more quickly in these situations than they would have if they had not completed the review training.