Development of Osteopathic Neuromusculoskeletal Medicine (ONMM) residency curriculum guidelines to meet Accreditation Council for Graduate Medical Education (ACGME) milestones

Context: A memorandum of understanding was reached between the Accreditation Council for Graduate Medical Education (ACGME), the American Osteopathic Association (AOA), and the American Association of Colleges of Osteopathic Medicine (AACOM) in 2014 outlining the course for a single accreditation system for graduate medical education. This process was completed in 2020 and has included the transition of AOA-accredited neuromusculoskeletal and Osteopathic Manipulative Medicine (OMM) programs into programs now termed “ Osteopathic Neuromusculoskeletal Medicine ” (ONMM) under the single accreditation system. Progress through ONMM residency is evaluated on the basis of 15 ACGME milestones that encompass six core competencies. However, there are no curricular guidelines to help guide the achievement of these milestones. Objectives: Theprimarypurposeofthisstudywastodevelop a proposed structure and content for an ONMM residency curriculum that is based on (1) the alignment of residency curriculum with ACGME milestones in one ACGME-accredited ONMM residency program, and (2) the perceived needs of residents and faculty for an ONMM residency curriculum. Methods: A mixed-methods exploratory sequential approach with embedded design was utilized. Qualitative analysis of didactics curriculum content for the past 2 years was coded according to themes identified in the residency curriculum content, which were further coded according to ACGME milestones.Curriculumtopicsidentifiedinqualitativeanalysis were utilized to create a questionnaire that was administered to residents and faculty (n=24) in the ONMM residency program toexamine guidelines and submitted to the American Academy of Osteopathy (AAO) for consideration. They are presented here as a resource for ONMM residencies to develop a program curriculum in alignment with individual program needs.

guidelines and submitted to the American Academy of Osteopathy (AAO) for consideration. They are presented here as a resource for ONMM residencies to develop a program curriculum in alignment with individual program needs.
A memorandum of understanding was reached between the Accreditation Council for Graduate Medical Education (ACGME), American Osteopathic Association (AOA), and American Association of Colleges of Osteopathic Medicine (AACOM) in 2014 outlining the course for a single accreditation system for graduate medical education [1][2][3]. Sponsoring institutions could begin to apply for institutional accreditation in April 2015 and receive the designation of pre-accreditation, after which AOA-approved residency programs could apply for initial accreditation in July 2015. This process was completed in 2020 and has included the transition of Neuromusculoskeletal and Osteopathic Manipulative Medicine (OMM) programs that were previously accredited by the AOA into programs now termed "Osteopathic Neuromusculoskeletal Medicine" (ONMM). There are presently 27 ONMM residencies and one combined family medicine and ONMM residency program accredited by the ACGME [4,5]. The AACOM defines the ONMM residency as: A primary residency disciplined in the neuromusculoskeletal system, its comprehensive relationship to other organ systems, and its dynamic function of locomotion. The principle focus of the discipline is osteopathic and patient-centered; specifically, it embodies structural and functional interrelation, body unity, self-healing, and self-maintenance. [6] The ACGME adopts this definition in their program requirements for ONMM, and further indicates that ONMM specialists must have specialized knowledge that encompasses basic and clinical sciences, as well as the evaluation and management of neuromusculoskeletal disorders. They must also have knowledge of Osteopathic Manipulative Treatment (OMT) and its indications, benefits, and risks [7].
Progress through ONMM residency is evaluated based on 15 ACGME milestones that encompass six core competencies (Supplementary Material 1) [8]. These competencies are patient care, medical knowledge, systems-based practice, practice-based learning and improvement, professionalism, and interpersonal and communication skills. The six core competencies were approved by the ACGME in 1999 [9,10]. Prior to the single accreditation system, the core competencies were also implemented in the AOA basic standards for residency education to allow alignment of all residency specialties in the United States within the same organized structure [11,12]. The AOA included an additional core competency, Osteopathic philosophy and OMM, which was meant to be incorporated in all seven AOA core competencies [11]. Issues arose over time with operationalization of the core competencies, and the milestones were developed to provide a developmental framework upon which to assess resident performance of knowledge, behavior, and skills in the core competencies [10]. Some programs began to implement the milestones in 2009, and all subspecialties including ONMM began to report milestone data in 2015 [8,10].
Although the milestones allow the evaluation of stages of professional development during the course of residency, they are not a representation of the complete curriculum content of a residency. Some specialties, such as family medicine and internal medicine, have published residency curriculum guidelines [13][14][15]. However, no guidelines exist to help guide the achievement of milestones in ONMM residencies. The purpose of this study was to develop a proposed structure and content for an ONMM residency curriculum based on: (1) the alignment of current curriculum with ACGME milestones in one ACGME-accredited ONMM residency; and (2) the perceived needs of residents and faculty for an ONMM residency curriculum. Secondary goals of the study were to examine: (1) the definition and purpose of an ONMM residency program; and (2) differences in perception in curriculum needs between residents and faculty.

Methods
Kern's six-step approach to curriculum development for medical education was utilized as the theoretical foundation for this study [16]. The first step of this approach is to conduct a general needs assessment to determine the scope of a health problem and examine the current approach being utilized. The second step is a targeted needs assessment of the learners and learning environment. The methods described below were developed as a general and targeted needs assessment utilizing a mixed methods exploratory sequential approach with embedded design (Supplementary Material 2) [16,17].
Mixed methods are a field of methodology utilized in social, behavioral, and health science that is approximately 30 years old. Qualitative and quantitative data are gathered, and rather than utilizing the results in parallel, both qualitative and quantitative data are integrated to draw conclusions utilizing the strengths of each set of findings. A mixed methods exploratory sequential approach begins with qualitative data that are developed into an instrument or intervention that is then tested, the results of which allow more contextualized conclusions to be drawn.
The general needs assessment consisted of qualitative analysis of didactics curriculum content for the past 2 years in one ACGME-accredited ONMM residency program. The targeted needs assessment was comprised of an open-and closed-question survey developed on the basis of the qualitative analysis and examined the perceived importance of curriculum content among faculty and residents. Open-ended questions were embedded in the survey to investigate the definition and purpose of an ONMM residency. The study protocol was submitted under CARRIE (Centralized Algorithms for Research Rules on IRB Exemption) through Medical City Fort Worth; the project was determined to be exempt from Institutional Review Board oversight (reference # 2020-753) on October 1, 2020.
General needs assessment: qualitative analysis of the residency curriculum ONMM residency didactics calendars for the past two academic years (July 2018-July 2020) were reviewed and hand-coded according to themes and subthemes [18]. Curriculum topics were matched to each of the 15 ACGME milestones that residents are expected to move through during residency [8,18].

Targeted needs assessment
Study questionnaire: An open-and closed-ended question survey was developed based on the curriculum topics, themes, and subthemes identified in the qualitative analysis (Supplementary Material 3) (EB, DM). The perceived importance of each curriculum topic identified in the qualitative analysis was collected on a five-point Likert scale. Openended questions inquiring about the definition of ONMM and the purpose of an ONMM residency curriculum were included. Because this survey has not been previously validated, two external reviewers in the field of OMM outside of the program being surveyed were asked to examine the questionnaire for content and transferability to other programs [19]. Reviewers were purposively chosen; the first reviewer is a recent graduate of an ONMM residency program, and the second reviewer has more than 30 years of experience as a faculty member. Reviewers were sent a summary of the background, purpose, and design of the study, as well as a copy of the survey for review. Cronbach's alpha was utilized to evaluate the internal reliability of curriculum topics in the survey when grouped according to the themes and subthemes identified in qualitative analysis of the residency curriculum and for the overall survey as evidence for establishing future validity [20][21][22][23]. A Cronbach's alpha of more than or equal to 0.7 was considered satisfactory.
Study population: A purposive sample of past participants in didactics in one ONMM residency program were recruited into the study, including current residents, current faculty, recent graduates, and past faculty. Participants were eligible to participate in the study if they had participated in ONMM residency didactics in the past two academic years (July 2018-July 2020), if they were able to give written informed consent, and if they were over the age of 18. A list of potential study participants was developed based on records of didactics from the past two academic years (n=48), of whom 25 responded to the survey. One study participant was excluded from analysis due to conflict of interest.
Study questionnaire procedures: The survey was administered online via Qualtrics between October 21, 2020 and November 12, 2020 [24]. Potential study participants were emailed a link to complete the survey along with three subsequent reminder emails. Potential participants were asked to complete a consent form on Qualtrics. The study was completely voluntarily, and individuals not wishing to complete the survey could simply close the link. The survey took approximately 20 min to complete. No financial compensation was provided.
Quantitative analysis of study questionnaire: SPSS ® Statistics, Version 25.0 was utilized for quantitative data analysis [25]. Counts and frequencies were obtained for categorical data. Averages and ranges were obtained for continuous variables. Counts and percentages of participants who ranked each curriculum topic as either very important or extremely important were calculated. An independent sample t-test compared average Likert scale scores between faculty (past and present, n=12) and residents (current residents and recent graduates, n=12). The null hypothesis was there would be no significant difference in perception in importance of curriculum topics between residents and faculty with a significance level of α=0.05.
Qualitative analysis of open-ended questions embedded in the study questionnaire: Dedoose, Version 7.0.23, a mixed methods software, was utilized to code responses to the open-ended question "How do you define ONMM?" [26] The ACGME definition of ONMM (please see introduction) was divided into five themes: (1) ONMM is a primary and unique residency; (2) ONMM requires specialized knowledge; (3) ONMM requires understanding of the neuromusculoskeletal system; (4) ONMM incorporates the Tenets of Osteopathy; and (5) ONMM encompasses knowledge and application of OMT [27]. Each participant's response to the question "How do you define ONMM?" was coded according to these five themes independently by two authors, and any differences in coding were reconciled. Themes were also identified for the question "What do you think should be the purpose of an ONMM residency curriculum?"

Mixed methods data integration
Integration of results from the general and targeted needs assessments: Counts and percentages of participants who ranked each curriculum topic as either very important or extremely important were tabulated according to: (1) themes and subthemes identified in the qualitative analysis of curriculum content; and (2) by alignment with ACGME milestones [8].
Integration of results from embedded qualitative analysis in the study questionnaire: Kuder-Richardson 20 was calculated to determine the construct reliability of participant responses when coded according to the five themes identified in the ACGME definition of ONMM [20]. The percent consensus of each participant's response with the ACGME definition of ONMM was calculated. An independent sample t-test determined whether there was a significant difference between average percent consensus between the faculty and residents. The null hypothesis was that there would not be a difference in the average percent alignment of faculty and resident definitions of ONMM with the ACGME definition. Significance was set at α=0.05.

General needs assessment: qualitative analysis
The five themes identified in qualitative analysis of the curriculum content over the past two academic years were: (1) OMM laboratory topics, defined as didactics topics that involve the hands-on application of psychomotor skills; (2) faculty-led activities and lecture topics; (3) resident-led  (5) training courses and volunteer activities. Seven subthemes were identified in the qualitative analysis of OMM laboratory topics covered during residency didactics, which were: (1) physical examination; (2) OMT techniques; (3) maternal and child health; (4) anatomy-specific topics; (5) injections; (6) sports medicine; and (7) performing arts medicine (Table 1). OMM laboratory topics predominantly represented the ACGME milestones of patient care and medical knowledge [8].
Five subthemes were identified in the qualitative analysis of faculty-led activities and lecture topics, which were: (1) functional anatomy; (2) discipline-specific topics; (3) practice management; (4) gross anatomy laboratory; and (5) integrative and functional medicine ( Table 2). Faculty-led lectures and activities covered a multitude of ACGME milestones, including medical knowledge, patient care, systems-based practice, practice-based learning and improvement, professionalism, and interpersonal and communication skills. Board review was identified as a subtheme in qualitative analysis of resident-led activities (Table 3). Residentled activities covered several ACGME milestones including patient care, medical knowledge, and professionalism. No subthemes were identified in the qualitative analysis of research curriculum (Table 4) or training courses and volunteer activities (Table 5). Research topics covered several ACGME milestones including systems-based practice, practice-based learning and improvement, and interpersonal and communication skills. Training courses and volunteer activities provide opportunities to meet ACGME milestone requirements for patient care, medical knowledge, professionalism, and interpersonal and communication skills.

OMM laboratory topics
All 24 survey respondents felt that osteopathic structural exam and upper and lower extremity orthopedic exam were either very important or extremely important (Table 1). Direct methods, indirect methods, osteopathic cranial manipulative medicine, and pediatric OMT were also considered either very important or extremely important by all study participants. Residents reported that acupuncture was a significantly more important OMM laboratory topic (mean=3.58, SD=0.996) compared to faculty (mean=2.33, SD=0.985), t (22) =−3.091, p=0.005. There were no other significant differences in the responses between residents and faculty.

Faculty-led activities and lecture topics
The most important perceived faculty-led lecture topics were common upper and lower extremity injuries (n=24) and low back pain (n=24) ( Table 2). The residents found that facultyled activities and lecture topics related to integrative and functional medicine were significantly more important than the faculty. The residents found acupuncture (mean=3.75, SD=0.965) to be a significantly more important faculty-led lecture topic than the faculty (mean=2.58, SD 1.165), t (22) =−2.672, p=0.014. Residents found yoga (mean=3.50, SD=0.905) to be a significantly more important topic than the faculty (mean=2.50, SD=1.087), t (22)=−2.449, p=0.023. The residents also felt that functional medicine (mean=4.00, SD=0.853) was significantly more important than the faculty (mean=2.58, SD=1.084), t (22)=−3.559, p=0.002.

Resident-led activities and lecture topics
The most important perceived resident-led activity was the written exam board review (n=22, 91.7%) ( Table 3). There were no significant differences in the perceived needs for resident-led activities between faculty and residents.

Research
The most important perceived research topics by the residents and faculty were how to conduct a literature review (n=16, 66.7%), journal club (n=16, 66.7%), and oral and poster presentation guidelines (n=16, 66.7%) ( Table 4). The faculty and residents did not differ significantly in their perceived needs for research curriculum.

Training courses and volunteer activities
The most important perceived need was a 40 hour introductory cranial course (n=22, 91.7%), followed by AAO Convocation (n=21, 87.5%) ( Table 5). There were no significant differences in the perceived needs for training courses and volunteer activities between faculty and residents.
Targeted needs assessment: analysis of open-ended questions embedded in the study questionnaire

Definition of ONMM
The study participants most commonly described ONMM in terms of the specialized knowledge required for the discipline (n=19, 79.2%) and the Tenets of Osteopathy (n=17, 70.8%). For example, one participant described ONMM as A specialty that incorporates training in anatomy and neuromusculoskeletal physiology to diagnose, manage, and treat musculoskeletal complaints of various etiologies including rheumatological, neurological, and psychological diseases, and chronic pain with a patient-centered approach.
Twelve study participants (50.0%) described ONMM in terms of the neuromusculoskeletal system, 11 participants (45.8%) described ONMM as a primary or unique specialty, and seven participants mentioned OMT (29.3%) as part of their definition of ONMM. Twelve participants (50.0%) described ONMM in terms of use of the hands without necessarily mentioning OMT. For example, one participant described ONMM as "medicine with our hands." On average, the participants mentioned three out of the five themes noted in the ACGME definition of ONMM (60.0%) [7]. There were no significant differences in the responses between faculty (mean=65.0%, SD=22.8) and residents (mean=55.0%, SD=21.1), t (22)=2.1, p=0.276. The five themes noted in the ACGME definition of ONMM did not have good construct reliability (KD20=0.041). An acceptable level of internal reliability (KD20=0.747) was reached when only the variables "knowledge" and "neuromusculoskeletal system" were left in the model.

Purpose of an ONMM curriculum
When asked what the purpose of an ONMM residency curriculum should be, the participants indicated that gaining knowledge (n=20, 83.3%) and becoming a competent physician (n=19, 79.2%) were the most important aspects of a residency program. A smaller theme noted was that the purpose of an ONMM residency is to provide continuation of this specialty and that ONMM continues to exist (n=3, 12.5%).

Discussion
One of the purposes and functions of the ACGME milestones is to guide curriculum development [10]. The ACGME milestones were utilized as a framework in this study to develop a curriculum proposal with specific objectives for each ACGME milestone that was submitted to the AAO and is undergoing review. Supplementary Material 4 provides a sample of the most current version of the curriculum. The intention of the curriculum is to allow ACGME-accredited ONMM residencies to have a resource to help guide the achievement of milestones. The research leading to the development of this curriculum is presented here.
The high overall perceived need for curriculum topics and the minimal differences between faculty and residents underscore the importance of the curriculum topics identified in qualitative analysis of residency curriculum content. Faculty-led activities covered at least 10 of 15 ACGME milestones, and emphasize the importance of providing adequate time and resources for faculty to engage in curriculum development [28]. Likewise, attendance at conferences, training courses, and volunteer events support educational opportunities related to multiple ACGME milestones (6 out of 15 milestones). The residents and faculty indicated that the most important of these activities was a 40 hour introductory cranial course (n=22, 91.7%). Although not currently a required aspect of ONMM residency education by ACGME, attendance at a 40 hour cranial course is required for board certification by the American Osteopathic Board of Neuromusculoskeletal Medicine (K. Sedrick, email communication, April 2021) [7].
Residents found that acupuncture was a significantly more important OMM laboratory topic than faculty, as well as for most integrative and functional medicine faculty-led topics, including acupuncture, functional medicine, and yoga. Integrative medicine emphasizes evidence-based practices that combine both conventional and complementary approaches to treatment, and focuses on the mind, body, and spirit [29]. Functional medicine looks at the dynamic processes that lead to disease and seeks to find the root cause of dysfunction [30]. The ideas underlying these models intersect with aspects of the Tenets of Osteopathy [27]. Although the differences between faculty and residents may simply indicate a difference in perspectives within the residency program surveyed, further research would be helpful to see if this may be a wider generational shift in understanding of the scope of practice of ONMM.
As Andrew Taylor Still defined, Osteopathy is: a scientific knowledge of anatomy and physiology in the hands of a person of intelligence and skill, who can apply the knowledge to the use of man when sick or wounded by strains, shocks, falls, or mechanical derangement or injury of any kind to the body. [31] Knowledge of the neuromusculoskeletal system (n=19, 79.2%) was central to resident and faculty definitions of ONMM. The Tenets of Osteopathy were developed by the Kirksville College of Osteopathy and Surgery in 1953. and they described (1) the body as a unit; (2) the body as selfregulatory; (3) structure and function as inter-related, and (4) appropriate therapy as based on the first three tenets [27,32,33]. These concepts were also central to study participants' definition of ONMM (n=17, 70.8%). OMT was only mentioned by seven participants (29.3%), but it was more broadly described in terms of utilizing the hands to diagnose and treat (n=12, 50.0%). This emphasizes that ONMM is not simply the application of OMT. While the five themes identified in the ACGME definition of ONMM had low internal reliability, they may simply indicate unique defining aspects of ONMM, including knowledge of the neuromusculoskeletal system, the Tenets of Osteopathy, and the use of one's hands to diagnose and treat. Of note, the term ONMM is new to graduate medical education. Prior to the single accreditation system, ONMM residencies were referred to as neuromusculoskeletal medicine and OMM residencies in the AOA basic standards for residency education [12]. Where possible, the components of the residency curriculum from existing evidence-based medical and postgraduate educational programs that are applicable across ACGME residency specialties, such as conflict resolution, residents as teachers, and quality improvement, should be included in the residency curriculum [34][35][36][37]. Kern's sixstep approach to medical education was utilized in the development of the present research and resulting proposed curriculum guidelines, which much like the milestones is an approach that is meant to be iterative and allow for revision of the curriculum to meet individual program needs [10,16].

Limitations
The present study examined the curriculum of one ONMM residency program and therefore may not be reflective of curriculum in other ONMM residencies. However, 2 years of curriculum content were evaluated to reach saturation of the content covered, and the study questionnaire was reviewed by external ONMM reviewers to ensure transferability to other programs [19]. Reviewers were not blinded in the study. The questionnaire had good internal reliability (Cronbach α=0.967). Future application of the survey to other studies would be helpful in developing further evidence for the validity of this instrument. The curriculum resulting from this research has been reviewed by multiple committees of the AAO including the Resident AAO, the Postdoctoral Training Committee, and the Education Committee, ensuring representation from a wide range of ACGME-accredited residency programs, faculty, and residents.
The results are presented on the basis of current ACGME milestones for ONMM [8]. The ACGME recognizes the need for review and revision of milestones as more experience and evidence becomes available through their implementation over time [10,38]. Many specialties and subspecialties have completed the revision of the milestones; therefore, the suggested curriculum content may need to be periodically reviewed to ensure continued alignment with the milestones as they are revised [39]. Other considerations include the need to adjust curriculum content based on the pathway residents take to enter ONMM postgraduate education. ONMM curriculum content may need to be delivered in 1 year for residents entering the ONMM3 pathway (a 12-month ONMM residency entered after completion of another residency), in 2 years for the ONMM2 pathway (a 24-month residency following completion of a transition year), in 3 years for the ONMM1 pathway (a 36-month residency), or in 4 years in a combined family and ONMM residency [4].

Conclusions
The present findings were employed in the development of proposed ONMM residency curriculum guidelines and submitted to the AAO for consideration. They are presented here as a resource for curriculum content and topics that can be applied to ONMM residency curriculum by level of importance and according to alignment with ACGME milestones.