Establishing osteopathic assessments to fulfill osteopathic recognition standards: a model for dermatology and other subspecialties

The 5 year transition period for American Osteopathic Association (AOA) training programs to apply for and receive Accreditation Council for Graduate Medical Education (ACGME) accreditation (i.e., the single graduate medical education system) was completed June 30, 2020. Of the previously AOA accredited programs that applied for or received osteopathic recognition (OR), only 24.5% are nonprimary care specialty programs according to the ACGME. The reluctance of specialty programs to apply for OR may be because osteopathic principles and practices (OPP) are not assessed. In order for programs to receive OR, they must have a standard method of assessment to assess osteopathic knowledge, including OPP and osteopathic manipulative treatment. In this Commentary, based on our assessment of the results of a literature review, we propose a model to provide a focused osteopathic assessment for the purposes of maintaining OR within residency training based on the ACGME six core competencies. Examples of multiple choice and essay questions are provided, as is a rubric for grading. The model is applied to the field of dermatology in this article and could serve as a blueprint to other subspecialties. With this framework, collaboration among programs will streamline the process to obtain OR in the ACGME single accreditation system.


Abstract:
The 5 year transition period for American Osteopathic Association (AOA) training programs to apply for and receive Accreditation Council for Graduate Medical Education (ACGME) accreditation (i.e., the single graduate medical education system) was completed June 30, 2020. Of the previously AOA accredited programs that applied for or received osteopathic recognition (OR), only 24.5% are nonprimary care specialty programs according to the ACGME. The reluctance of specialty programs to apply for OR may be because osteopathic principles and practices (OPP) are not assessed. In order for programs to receive OR, they must have a standard method of assessment to assess osteopathic knowledge, including OPP and osteopathic manipulative treatment. In this Commentary, based on our assessment of the results of a literature review, we propose a model to provide a focused osteopathic assessment for the purposes of maintaining OR within residency training based on the ACGME six core competencies. Examples of multiple choice and essay questions are provided, as is a rubric for grading. The model is applied to the field of dermatology in this article and could serve as a blueprint to other subspecialties. With this framework, collaboration among programs will streamline the process to obtain OR in the ACGME single accreditation system.
The 5 year transition period for American Osteopathic Association (AOA) training programs to apply for and receive Accreditation Council for Graduate Medical Education (ACGME) accreditation (i.e., the single graduate medical education [GME] system) was completed on June 30, 2020. Osteopathic recognition (OR) is a formal designation conferred by the ACGME. The Osteopathic Principles Committee (OPC) tracks residency programs' commitment to "teach and assess Osteopathic Principles and Practice (OPP) at the graduate medical education level" [1]. With the cessation of American Osteopathic Association (AOA) graduate medical education activities on June 30, 2020 as a result of the single GME system, a focus on maintaining osteopathic identity via OR is paramount for the future of the osteopathic profession [2,3]. Programs with ACGME preaccreditation status can apply for OR but will not be reviewed by the OPC until they have achieved ACGME accreditation [4]. To support the process of ACGME accreditation, the AOA's Application Assistance Program offers free consultations to training programs requesting application assistance [3]. As former AOA residency programs continue to receive full ACGME accreditation, the next logistical step will be to obtain OR.
As of December 27, 2020, 743 formerly AOA accredited residency programs had applied for accreditation through the ACGME; 253 (34.1%) of those programs applied for OR [5], meaning that 490 (65.9%) formerly AOA accredited programs do not maintain OR as of this report. Despite the downward trend in osteopathic presence, prior surveys showed that 67.9% of 728 respondents composed of osteopathic medical students, faculty, and residents at Michigan State University College of Osteopathic Meidcine [6] and 68.2% of 1,038 third year medical students who responded to a survey [7] valued the importance of OPP in their graduate medical education.
Although interest in maintaining OPP throughout training remains, residency programs with osteopathic distinction are evaporating during the transition to single accreditation. Barriers to applying for OR have been referenced in the literature [6] and may contribute to this evaporation of osteopathic training programs. The most common barriers cited include available administrative time to complete applications for OR, availability of trained faculty, and the number of requirements related to OR [6,8]. Furthermore, specialties that do not routinely perform osteopathic manipulative treatment (OMT), such as dermatology and ophthalmology, may also find it difficult to maintain relevance of OPP within their field.
Of the 253 programs that have received or are in the process of applying for OR, 191 (75.5%) are primary care programs as defined by the American Osteopathic Association: 155 in family medicine, 32 in internal medicine, and three in pediatrics. Primary care programs are more likely to have already established OPP curricula, while other specialty programs-such as dermatology, ophthalmology, otolaryngology, and radiology-may have difficulty incorporating osteopathic principles into an ACGME curriculum. Exemplifying the lack of osteopathic principles being applied in practice, a survey of osteopathic physicians throughout Ohio in 2003 showed that 44% of 871 respondents reported not using OMT [9]. Due to the focused nature of some specialties, such as dermatology and many others referenced in the same survey, OMT is not practiced as frequently in them as it is in other disciplines with many associated musculoskeletal conditions such as family medicine, physical medicine and rehabilitation, and sports medicine [9]. In the survey cited earlier [9], no dermatology respondents had practiced OMT in the prior week compared with 165 out of 360 (46%) family medicine respondents. Nonetheless, OPP can still be maintained within all specialties, as exemplified in a recently proposed dermatology curriculum that incorporates the five core osteopathic principles: biomechanical-structural model, metabolic-nutritional model, respiratory-circulatory model, neurological model, and behavioral-biopsychosocial model [10,11]. For specialties that do not routinely utilize OMT but value maintaining an OPP curriculum within a residency program, it may be difficult to create an objective evaluation of residents' osteopathic abilities. One aspect of an osteopathic curriculum not routinely discussed is a grading system employed to evaluate residents' progression in osteopathic training. For example, a prior study [12] of 3,424 physicians who took the Medical Council of Canada clinic skills examination showed that communication scores of two standard deviations below the mean were associated with 1.17 more retained complaints per 100 physicians per year [12]. Similarly, a standardized assessment of osteopathic education is essential for maintaining OR within residency training programs, ensuring patient safety, and improving quality of care.
With that background in mind, in this Commentary, we provide an outline of objective curricular evaluation techniques and examples of questions that could be used to assess osteopathic knowledge amongst residents in programs with a with a non primary care focus based on the six ACGME core competencies: patient care, medical knowledge, professionalism, interpersonal and communication skills, practice based learning and improvement, and systems based practice [13]. The model is applied to the specialty of dermatology in this Commentary, but may serve as a blueprint for other subspecialties. The model is proposed with the hope that the process to obtain OR in the single GME accreditation system might be streamlined by using it.

Literature search
Utilizing the 2009 PRISMA guidelines [14], a literature search was performed on February 23, 2020 by one author (S.M.) in MEDLINE/PubMed database with the following search string: "osteopathic" and "recognition" ( Figure 1); 219 articles were found. There were no restrictions regarding date of publication. Titles and abstracts were individually reviewed by one author (S.M.). Articles were included if they referenced residency graduate medical education in the title or abstract (11; 5.0%) and excluded if they did not (208; 95.0%). After application of the inclusion and exclusion criteria, 11 full text articles were individually reviewed by two authors (S.M., Y.L.L.) for relevance to the structure of residency programs with osteopathic recognition, at which point another article [15] (1; 0.5%) was eliminated because it did not directly relate to the challenges faced by training programs with a more narrow specialty scope. Publicly available ACGME OR requirement documents from the ACGME website were separately reviewed.

Synthesizing core competency requirements
After reviewing requirements for OR and ACGME core competencies, the two domains were synthesized by all three authors to present evaluation methods for OPP and OMT. Using the ACGME core competencies as a framework, examples of objective formative assessments were created to evaluate residents on the basis of OPP and OMT (Table 1). Evaluations were based on commonly used techniques already established in many residency programs, including clinical encounters, multiple choice questions, and essay questions. A quantitative grading scale was created to reflect residents' level of maturation in OPP/OMT skills through subjective examinations such as clinical encounters and essay questions.
The requirements set forth by the OPC outline a framework for programs to design appropriately focused curricula of Osteopathic Principles, Practice, and Therapy [1]. Taking each aspect of the criteria for OR, parts A-D described in Figure 2 are likely achievable based on programs' previous experiences in osteopathic GME. One barrier for specialty programs is an objective formative assessment of osteopathic knowledge and procedural skill (i.e., OPP and OMT). Although many specialties take standardized in service exams through their respective board certification services to assess educational progress, there is no national examination to reference for the evaluation of OPP. The Comprehensive Osteopathic Medical Licensing Examination of the United States (COMLEX-USA) Level 3 examination is often taken during a student's internship or early in residency and subsequently cannot be used longitudinally to evaluate OPP in specialty programs. Finally, in service examinations often do not include an evaluation of OPP, leaving each program to devise its own testing material. In the absence of established evaluations for OPP, we have provided examples of proposed assessments for OPP/OMT within each ACGME competency domain through multiple choice questions, essay questions, or clinical evaluation (Table 1).
Programs may elect to utilize multiple choice exams with questions created by board certification organizations, faculty members, or senior residents with knowledge of OPP. Multiple choice questions should be targeted to address certain aspects of the ACGME core competencies while also incorporating osteopathic concepts, and we provide examples of multiple choice questions based on the ACGME core competencies ( Table 2).

Principles in an examination
When applying for OR, there are specific requirements to implement a clinical or written osteopathic examination. As noted in the OR requirements, "The program must provide assessment of the resident in application of OPP in each of the ACGME Competencies… There must be objective formative assessment of osteopathic medical knowledge and procedural skills" [16]. These requirements must Screening Included Eligibility Identification Figure 1: Preferred reporting items for systematic reviews and metaanalyses [14] flowchart documenting the literature review performed as the proposed curriculum was designed. also include assessment of OPP knowledge and skill proficiency in OMT, as applicable to the specialty [16]. As noted previously, dermatology does not typically utilize OMT in general practice, as exhibited in a survey that included Ohio dermatologists; 0% (n=0) had used OMT in the prior week [9]. However, certain methods of OMT may be valuable in the field of dermatology [10,17]. In fact, the American Osteopathic Board of Dermatology has historically highlighted OPP with an hourlong section of the board certification exam dedicated to the "osteopathic concept of healthcare as it relates to the practice of dermatology, emphasizing a multifactorial approach to disease management" [18]. An objective grading system for the material being tested should be used to provide a standardized assessment. While evaluations of multiple choice questions are easily scalable by awarding points for correct answers, a rubric for evaluating essay type questions is not as easily quantifiable. Using the existing rubric for ACGME core competencies (Table 2), essay responses can be graded on a scale of one to five for each of the six ACGME core competencies, plus an additional domain of OPP. Thus, a perfect score would be five out of five in all seven domains, totaling 35 out of 35 possible points per essay response to indicate expert knowledge, utilization of OPP, and patient management (Table 3). This same grading system could be used on clinical evaluations upon the conclusion of clinical rotations. Many resident evaluation templates use a scale of one to five, with one representing a beginning level resident and five representing expert status in the category being evaluated. Utilizing similar grading systems across programs would allow a standardized verification of appropriate resident training progression that fulfills OR accreditation requirements.
As noted in the "Background and Intent" section of the OR requirements [16], the goal of implementing standardized assessments is to provide osteopathic faculty and residents with information that allows for comparison of resident knowledge acquisition and progression. Ideally, this information will be utilized both internally (for longitudinal evaluation during residency) and externally (across residency programs). Compiling both multiple choice and essay questions within each specialty into a central database would allow programs to easily access examinations and evaluate residents. Centralization of examinations would also enhance external comparison of residents' abilities across programs nationally. To our knowledge, this centralization process is not currently available, nor being implemented anywhere.

Limitations
Limitations to our proposed curriculum include the limited number of supporting references from the literature due to the recent implementation of OR within GME. As OR becomes more prevalent, we expect increased discussion of successful techniques to evaluate OPP. Additionally, our curricular recommendations are based on our experiences at a single institution, within one specialty. The evaluation techniques we suggest above have not been validated, nor have any outcomes been assessed. Ideally, this report will provide a basis for future assessments and innovations involving the evaluation of OPP across many specialties. Furthermore, validation procedures for multiple choice and essay questions should occur. A limiting factor to this procedure is the lack of a central agency to collect and disburse validated questions. This process could take numerous years to accomplish but would certainly bolster the integrity of osteopathic evaluations. Although some specialties lack extensive literature in OPP and OMT, osteopathic values are still relevant and can be tested objectively. The sparse research on osteopathic values in various specialties should be viewed as an opportunity to investigate more thoroughly the effects of OMT/OPP. objective Figure 2: Summarized osteopathic recognition requirements [1]. Table : Examples of multiple choice and essay questions that test OMT/OPP based on ACGME core competencies.

Assessment of patient care and medical knowledge
A  year old Hispanic male presents to the dermatology clinic with a burning sensation over his right upper back. The patient states that it began gradually over the past  weeks, fluctuates in intensity, and occasionally feels more sensitive than surrounding skin. The patient notes that he mows his lawn every weekend but does not have any new exposures he can remember. He denies recent illnesses. On examination. A faint, poorly demarcated patch of hyperpigmented skin without scale is present surrounding the inferior right scapula. What is the associated pathophysiology with this condition? A) Genetic mutation in filaggrine (FLG) gene encoding filaggrin protein B) Hypersensitivity reaction to ursodiol proteins C) Impingement of the T7 spinal nerve root [correct] D) Latent varicella zoster virus in the dorsal root ganglion E) Type 4 hypersensitivity reaction What type of osteopathic manipulative treatment (OMT) would best address the associated pathology in the above condition? A) OMT is contraindicated in this instance B) Rib articulatory technique to rib 6 C) Articulatory technique to T7 vertebra [correct] D) Spencer's technique to right shoulder E) Rib raising

Assessment of systems based practice and professionalism
A  year old female with a history of chronic bronchitis, venous stasis, insulin dependent diabetes mellitus, and obesity presents to the dermatology clinic with concern for swelling in her feet. She states that this has been a chronic problem for many years, but she noticed recently increased swelling. On examination, a cm crusted erosion is noted on the left posterior calf with dry serous fluid surrounding the wound and a pink base. As a part of the treatment plan, which billing code would best be used in order to address her swelling with appropriate lymphatic osteopathic manipulative treatment (OMT)?

Assessment of interpersonal and communication skills and practice based learning
A  year old mother presents to the dermatology clinic with her  year old son, who has had a chronic rash since age  involving his cheeks and bilateral antecubital fossa. It worsens in the winter season, itches incessantly during exacerbation, and improves mildly with Vaseline. The patient's mother brings in essential oils and states she read on the Internet they are effective for eczema. What is the most appropriate and accurate statement to make in response to the mother? A) "I'm happy to see you are inquiring about effective therapy for your son. Essential oils are a safe option for treatment for your son's condition and have variable success." B) "You should avoid essential oils. The FDA does not evaluate their content." C) "You will make your son's eczema worse with the use of essential oils." D) "While essential oils are a popular home remedy for a variety of conditions, their efficacy and adverse effects are unknown. I would discourage the use for your son as they could exacerbate his symptoms." [correct] E) "In what ways are you planning on using essential oils?"

Essay question example
A  year old White female with no significant past medical history presents to the dermatology clinic for a follow up visit after a recent excisional biopsy of a pigmented lesion on her left calf. A brief glance over the pathology report shows malignant melanoma with a Breslow depth of .mm and Clark Level III. In essay format, respond to the patient, indicating your plan for counseling, treatment, and follow up while incorporating osteopathic principles within your response. Limit  words. ACGME, Accreditation Council for Graduate Medical Education; OMT, osteopathic manipulative treatment; OPP, osteopathic principles and practice.

Conclusions
OR is key to perpetuating osteopathic values for all specialties in medicine. An increased push to obtain OR for specialties outside of primary care is needed. To adhere to ACGME requirements and evaluate the growth of residents as they progress through training, an objective evaluation of knowledge must be made; the curriculum proposed here is offered as a method to attain that goal. Evaluating the maturation of osteopathic values throughout residency not only will provide superior patient care as residents pursue their future careers, but also will create a more reputable distinction of OPP. Collaboration among programs that have already obtained OR and those attempting to obtain OR is encouraged. Best practices may be shared and will streamline the process even if programs are not in the same specialty.
Competing interests: None reported. Author contributions: All authors provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; all authors drafted the article or revised it critically for important intellectual content; all authors gave final approval of the version of the article to be published; and all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Research funding: None reported.