Student Performance on Levels 1 and 2-CE of COMLEX-USA: Do Elective Upper-Level Undergraduate Science Courses Matter?

Selection of students for admission to osteopathic medical school is a rigorous process that relies on many factors, including Medical College Admission Test (MCAT) scores, undergraduate grade point averages (GPAs), undergraduate major, the selectivity or reputation of the undergraduate institution, and an admission interview process. The validity of using MCAT scores and GPA to predict student academic achievement in osteopathic medical school and student scores on the Comprehensive Osteopathic Medical Licensing Examination-USA (COMLEX-USA) Level 1 and Level 2-Cognitive Evaluation (CE) has previously been investigated.^1-7 These studies have shown some conflicting results.

Correlations were reported between MCAT scores and COMLEX-USA Level 1 and Level 2-CE scores in several studies,^1-4 but no correlation was found between preadmission data and COMLEX-USA Level 1 scores in another study.⁵ One study² found that undergraduate GPA was a stronger predictor of overall osteopathic medical student academic achievement than MCAT scores. Some studies^1,6,7 have found a strong correlation between COMLEX-USA Level 1 achievement and didactic GPA in the first 2 years of osteopathic medical school, as well as a correlation between COMLEX-USA Level 1 and Level 2-CE achievement.

Despite rapid advancements in scientific and medical knowledge during the past several decades, the prerequisite science courses for medical school have not changed substantially. The only preadmission science courses typically required are general biology, general chemistry, organic chemistry, and physics. Most medical schools do not require upper-level science courses in biochemistry, genetics, human anatomy, human physiology, microbiology, or molecular biology—though these courses are often recommended by medical schools and are included in the first year of most medical schools.^8,9

It seems logical that if students take more upper-level science courses before matriculation, they will be better prepared academically for medical school and, thus, achieve greater success there—especially during the preclinical years. However, the relationship between the number of upper-level science courses taken before matriculation and medical school achievement has not been evaluated previously—though, according to two studies,^10,11 students with science majors did not perform better on medical licensing examinations than students with nonscience majors.

A review of published literature indicates that the effects of demographic variables, such as age, racial-ethnic background, and sex, on academic achievement in osteopathic medical school have not previously been reported. However, previous studies of student achievement in allopathic medical programs showed that overall average scores for white students were higher than those for other ethnic groups on Part 1 of the National Board of Medical Examiners (NBME) examination¹² and on both Step 1 and Step 2 of the United States Medical Licensing Examination (USMLE).¹³ In addition, average scores of African American students were found to be lower than those of other ethnic groups on all USMLE examinations.¹³

Younger students have previously been found to have higher medical school GPAs than older students,¹⁴ but no age-related differences have been found in terms of scores on medical licensing examinations.^14,15 Mixed results have been reported regarding differences in academic achievement between men and women enrolled in medical school.^12-17

The purpose of the present study is to determine the relationship between number of elective undergraduate upper-level science courses and overall academic achievement (ie, preclinical and clinical GPAs, COMLEX-USA Level 1 and Level 2-CE scores) of osteopathic medical students at Western University of Health Sciences College of Osteopathic Medicine of the Pacific (WesternU/COMP) in Pomona, California. In addition, the relative importance of MCAT scores, undergraduate GPAs, age, racial-ethnic background, and sex in predicting osteopathic medical students' academic achievement at WesternU/COMP is examined.

Methods

Academic and standardized test records of osteopathic medical students in the 2008 and 2009 graduating classes of WesternU/COMP were analyzed. Data were obtained from the institutional database maintained by the university. Data for both graduating classes were obtained midway through the 2008-2009 academic year. The 2008 cohort had already graduated from WesternU/COMP, whereas the 2009 cohort had completed the preclinical phase but not the final year of their clinical clerkships.

Records of all students in the 2008 and 2009 graduating classes were included in the analysis, except for records of any students on a leave of absence or academic suspension, which were not available. Included in the analysis were records of students who failed their first attempts on COMLEX-USA Levels 1 and 2-CE, as well as records of students in the class of 2009 who did not take the COMLEX-USA Level 2-CE. The present study was performed in full compliance with the WesternU/COMP institutional review board policy.

The four dependent variables representing overall academic achievement were preclinical GPA, clinical GPA, COMLEX-USA Level 1 score, and COMLEX-USA Level 2-CE score. The preclinical GPA and clinical GPA refer to, respectively, the GPA in basic science courses and the GPA in clinical clerkships. The independent (ie, predictor) variables were preadmission data consisting of science and undergraduate nonscience GPAs; MCAT subscores in biological sciences, physical sciences, and verbal reasoning; numbers of upper-level science courses; and the demographic variables of age, racial-ethnic background, and sex. The number of upper-level science courses taken by each student in each graduating class were tabulated for biochemistry, human anatomy, human physiology, microbiology, and molecular biology/genetics.

The age of each student at time of matriculation for each graduating class was recorded for statistical analysis. Racial-ethnic background was incorporated into the analysis with the categories of Asian/Pacific Islander, Hispanic, and other/undeclared minorities coded as 1 and all other ethnic groups as 0.¹⁸ African American students were included in “other/undeclared minorities” because of their small sample size. Sex was coded as 0 for men and 1 for women.

Pearson product moment correlation coefficients (Pearson r) were calculated to evaluate the linear associations between each dependent variable and the independent variables. Separate multivariate linear regression analyses were performed for each of the four dependent variables by entering all independent variables in a single step.^1,18

To determine the relative contribution of each independent variable to the prediction of the dependent variables, we calculated β coefficients. To assess the combined effect of the set of independent variables on predicting each dependent variable, coefficients of determination for multivariate analysis (R²) were calculated. All statistical analyses were calculated using SPSS statistical software (version 16.0; SPSS Inc, Chicago, Illinois).

Results

Records were analyzed from 358 osteopathic medical students in the 2008 and 2009 graduating classes at WesternU/COMP. Of these students, 180 (50.3%) were men, and 178 (49.7%) were women. Mean (SD) age of students was 25.5 (4.2) years. Racial-ethnic backgrounds of the students were as follows: white, 174 (48.6%); Asian/Pacific Islander, 143 (39.9%); Hispanic, 16 (4.5%); African American, 6 (1.7%); and other or undeclared minorities, 19 (5.3%). There were 181 students in the class of 2008 and 177 students in the class of 2009. At the time of data collection, 16 students in the class of 2009 had not taken the COMLEX-USA Level 2-CE examination.

Descriptive statistics on number of upper-level science courses, MCAT subscore, GPA, and COMLEX-USA score, together with correlations of preadmission academic variables with academic achievement measures, are presented in Table 1. The mean (SD) number of upper-level science courses that students took before matriculation was 3.09 (1.01), and this number was not significantly correlated with any of the achievement variables.

Biological sciences MCAT subscore and undergraduate science GPA were significantly correlated with each of the academic achievement variables (P<.01) except clinical GPA. Physical sciences MCAT subscore was significantly correlated with preclinical GPA and COMLEX-USA Level 1 score (P<.01), whereas verbal reasoning MCAT subscore was significantly correlated with only COMLEX-USA Level 2-CE (P<.01). Pearson correlation coefficients for these significant correlations were low, ranging from 0.15 to 0.30, which accounted for a small proportion of the total variance in outcome (R²=0.02-0.09).

Table 2 presents data on preadmission variables and overall academic achievement of these students based on racial-ethnic background and sex. The mean undergraduate science GPA, nonundergraduate science GPA, and MCAT score were not significantly different between white students and Asian students. However, white students tended to have significantly higher scores than Asian students on preclinical GPA and COMLEX-USA Levels 1 and 2-CE (P<.01). Overall academic scores of Hispanic students were lower than those of white students, but these differences were statistically significant only for COMLEX-USA Level 1 and MCAT scores (P<.05). Academic scores were not significantly different between white students and students of other/undeclared minority groups.

Women had higher mean undergraduate nonscience GPA and preclinical GPA, but lower mean MCAT score, than men (P<.05). There was no statistically significant difference in COMLEX-USA scores for Level 1 or Level 2-CE between men and women.

Table 3 presents data from separate multivariate regression analyses of the four achievement variables, predicted on the basis of preadmission and demographic variables. Three of the four regression models revealed statistical significance (P<.05), but the overall amount of variance in terms of R² was low, ranging from only 0.14 (for COMLEX-USA Level 2-CE score) to 0.18 (for preclinical GPA).

The β coefficients for undergraduate science GPA were significant for all achievement variables (β=0.12-0.39) and were greater than those for the undergraduate nonscience GPA and MCAT subscores—suggesting the greater predictive value of the undergraduate science GPA. Undergraduate nonscience GPA was negatively associated with preclinical GPA (β= -0.21, P<.01) and COMLEX-USA Level 1 score (β= -0.26, P<.01).

Biological sciences MCAT subscore was a modest but significant predictor of preclinical GPA (β=0.18, P<.01) and COMLEX-USA Level 1 score (β=0.22, P<.01). Physical sciences MCAT subscore significantly predicted preclinical GPA (β=0.13, P<.05), and verbal reasoning MCAT subscore significantly predicted COMLEX-USA Level 2-CE score (β=0.22, P<.01).

As shown in Table 3, the number of preadmission upper-level science courses had no predictive value on any achievement variable. Asian students did not perform as well as white students in most achievement variables (β= -0.23 to -0.13), except for clinical GPA (β= -0.05). No significant differences were observed between white students and Hispanic students or between white students and those of other/undeclared minority groups in GPA and COMLEX-USA scores, after controlling for other independent variables.

Women outperformed men in preclinical GPA (β=0.21, P<.01) and COMLEX-USA Level 2-CE score (β=0.13, P<.05). The β coefficients for increasing age were negative and statistically significant for all academic achievement variables (β= -0.24 to -0.12; P<.05 to P<.01).

Scores on COMLEX-USA Levels 1 and 2-CE were strongly correlated with both preclinical and clinical GPA (P<.01). Correlations of preclinical GPA with COMLEX-USA Level 1 and Level 2-CE scores (Pearson r=0.70 and 0.59, respectively) were greater than those of clinical GPA with COMLEX-USA Level 1 and Level 2-CE scores (Pearson r=0.27 and 0.34, respectively). In addition, student achievement on both COMLEX-USA tests were correlated (Pearson r=0.72, P<.01).

Comment

The present study is the first to our knowledge to report results of an analysis between number of preadmission upper-level science courses and achievement of students at an osteopathic medical school. No correlation was found between the number of preadmission upper-level science courses and overall academic achievement of osteopathic medical students at WesternU/COMP. This lack of correlation argues against the belief that adding upper-level science courses as prerequisites for medical school will result in greater academic success post-matriculation.

A plausible explanation for the lack of correlation is that we studied only those osteopathic medical students who had successfully completed the medical curriculum at WesternU/COMP (for the 2008 cohort) or who were in good academic standing at the time data were retrieved (for the 2009 cohort). As previously mentioned, data for those students (n=29) who failed to complete the WesternU/COMP medical program because they were on a leave of absence or academic suspension were not available. Thus, we could have inadvertently missed or misrepresented the predictive power of preadmission upper-level science courses. Future research on this matter should include medical students with academic difficulties.

In addition, it is important to note that the osteopathic medical students in our sample took an average of three upperlevel science courses, and more than 95% of these students had two or more such courses before matriculation. This low variability could have resulted in difficulties in observing statistically significant differences in outcomes.

The present study revealed correlations between preadmission academic variables, osteopathic medical school achievement, and COMLEX-USA Level 1 and Level 2-CE scores. Biological sciences MCAT subscore, physical sciences MCAT subscore, and undergraduate science GPA were correlated with most academic achievement variables. The undergraduate science GPA appeared to be the most significant predictor of overall academic achievement at WesternU/COMP, whereas the biological sciences MCAT subscore had only modest value in predicting academic achievement. This result is in agreement with the finding of Evans and Wen² that either total or undergraduate science GPA is a better predictor than MCAT subscores for overall student academic achievement in osteopathic medical school.

By contrast, several other studies have suggested that MCAT subscores may be better predictors of academic achievement than undergraduate GPA in either osteopathic^1,3 or allopathic medical school.^15,19,20 Thus, strong evidence in the literature supports the use of undergraduate GPA and MCAT score—either separately or together—to predict academic achievement in medical school.

It is interesting to note that we found undergraduate nonscience GPA to have a statistically significant and negative association with preclinical GPA and COMLEX-USA Level-1 score when controlling for undergraduate science GPA, MCAT subscores, number of upper-level science courses, age, racial-ethnic background, and sex (Table 3). This result should be interpreted in conjunction with the slightly positive—though not statistically significant—correlation between undergraduate nonscience GPA and preclinical GPA when not controlling for other factors (Table 1).

In other words, the negative β coefficient revealed in the regression model between undergraduate nonscience GPA and preclinical GPA and COMLEX-USA Level 1 score does not signify that having a higher undergraduate nonscience GPA leads to poorer achievement in medical school and board examinations. Indeed, the slightly positive bivariate correlation between undergraduate nonscience GPA and preclinical GPA suggests that the opposite may be true.

In the regression model used in the present study, each β coefficient carries the assumption that all other independent factors in consideration are equal—that is, factors such as undergraduate science GPA and MCAT subscores are the same for each student. Only in this context is the association between undergraduate nonscience GPA and medical school achievement negative.

One possible interpretation of this result is that, on average, students who prioritize information directly related to medical school and who excel in science courses—at the expense of unrelated information in nonscience courses—may be at an academic advantage. Such a learning approach may enable these students to concentrate their efforts and perform better than other students in medical school.

Although we identified the biological sciences MCAT subscore as a significant predictor of preclinical GPA and COMLEX-USA Level 1 score, verbal reasoning MCAT subscore was found to correlate with COMLEX-USA Level 2-CE score (Table 1) and to predict COMLEX-USA Level 2-CE score (Table 3). Similar findings have been reported in previous studies of students at osteopathic medical schools.^1,2 One plausible explanation of these findings is that the verbal reasoning MCAT subscore is more closely related to the clinical diagnosis and problem-solving skills measured by COMLEX-USA Level 2-CE than to the scientific understanding of disease mechanisms measured by COMLEX-USA Level 1.

In addition, verbal reasoning MCAT subscore has previously been identified as the most highly predictive factor for USMLE Step 2 achievement in allopathic medical school.²¹ Verbal reasoning MCAT subscore may also have predictive value in determining students who perform poorly in preclinical academic examinations but do well in clinical clerkships.²²

Our finding that the overall academic achievement of Asian students was lower than that of white students at WesternU/COMP—despite comparable scores in preadmission variables (Table 2)—is in agreement with previous reports of students at allopathic institutions.^12,15,23 One possible explanation for these findings is that Asian families lose some of their positive, encouraging influence over their children's academic achievement as the children mature and enter medical school.¹⁵

Another factor to consider is that Asian students are likely a more heterogeneous group than white students. Asian-born individuals represented nearly 400,000 of the 1.1 million people obtaining legal permanent residence status in the United States in 2008.²⁴ Although many Asian osteopathic medical students may have been born in the United States, many others may be immigrants with strong ties to the Asian cultures they and their families left behind. Adaptation to Western culture, language, and lifestyle for these immigrant students may occur first in medical school and may adversely affect their academic achievement.

We found in the present study that, despite low levels of representation in our sample, Hispanic students had significantly lower scores than white students on the MCAT and COMLEX-USA Level 1 examinations (Table 2). However, it is important to note that—after controlling for independent variables, such as undergraduate GPA and MCAT subscores—no significant difference was observed between Hispanic and white students in COMLEX-USA Level 1 and Level 2-CE scores (Table 3). Previous studies have shown that white students outperformed Hispanic students at allopathic medical schools in the NBME Part 1^12,25 and Part 2²⁵ examinations. These observed differences in academic achievement may be the result of differences in MCAT subscores and undergraduate academic achievement.^12,25,26

It is interesting to observe that women outperformed men in the WesternU/COMP osteopathic medical curriculum as measured by preclinical GPA and COMLEX-USA Level 2-CE score, but no difference between the sexes was observed in achievement as measured by clinical GPA and COMLEX-USA Level 1 score (Table 2). Some studies have previously shown that men had higher MCAT scores than women before matriculation,^14,16 as well as slightly higher preclinical GPA than women,¹⁷ in allopathic medical schools. Other studies^12-16 have reported that men outperformed women in the NBME Part 1 and USMLE Step 1 examinations, though women performed equally to or better than men in the NBME Part 2 and USMLE Step 2 examinations.^14-16

Hence, our data seem to differ from that of previous studies regarding the role of sex differences in overall medical school achievement. Further study is warranted to clarify this issue.

Age appears to be a statistically significant predictor of preclinical and clinical GPAs and COMLEX-USA Level 1 and Level 2-CE scores in the present study, suggesting that younger students outperform older students in overall academic achievement at WesternU/COMP. This result is consistent with a previous report¹⁴ that younger students tend to have higher GPAs than older students in allopathic medical programs.

Until recently, no age differences in scores on allopathic medical licensing examinations had been reported.^14,15 However, a report in 2007 indicated that age is a predictor of both USMLE Step 1 and Step 2 scores.¹³ Thus, conflicting results regarding the role of age differences in academic achievement at medical school suggest that further investigation of this issue is required.

Conclusion

The results of the present study suggest that the number of preadmission elective upper-level science courses has no predictive value in determining overall academic achievement of students at osteopathic medical schools. However, results also suggest that undergraduate science GPA and biological sciences MCAT subscore are significant predictors of overall academic achievement at osteopathic medical schools. Differences in some academic outcome variables may be associated with racial-ethnic background and age.