Abstract
There is nearly a century of educational research that has demonstrated that three option multiple-choice questions (MCQs) are as valid and reliable as four or five option, yet this format continues to be underutilized in educational institutions. This replication study was a quasi-experimental between groups research design conducted at three Canadian schools of nursing to examine the psychometric properties of three option MCQs when compared to the more traditional four option questions. Data analysis revealed that there were no statistically significant differences in the item discrimination, difficulty or mean examination scores when MCQs were administered with three versus four option answer choices.
References
Abozaid, H., Park, Y. S., & Tekian, A. (2017). Peer review improves psychometric characteristics of multi-choice questions. Medical Teacher, 39(S1), S50–S54. doi:http://dx.doi.org/10.1080/0142159X.2016.125473Search in Google Scholar
Anderson Hicks, N. (2011). Guidelines for identifying and revising culturally biased multiple-choice nursing examination items. Nurse Educator, 36(6), 266–270. doi:10.1097/NNE.0b013e3182333ca6Search in Google Scholar
Bailey, P. H., Mossey, S., Moroso, S., Duff Cloutier, J. D., & Love, A. (2012). Implications of multiple-choice testing in nursing education. Nurse Education Today, 32(6), e40–4. doi:10.1016/j.nedt.2011.09.011Search in Google Scholar
Boland, R. R. J., Lester, N. A., & Williams, E. (2010). Writing multiple choice questions. Academic Psychiatry, 34(4), 310–316. doi:10.1176/appi.ap.34.4.310Search in Google Scholar
Brady, A. M. (2005). Assessment of learning with multiple-choice questions. Nurse Education in Practice, 5(4), 238–242. doi:10.1016/j.nepr.2004.12.005Search in Google Scholar
Bruno, J. E., & Dirkzwager, A. (1995). Determining the optimal number of alternatives to a multiple-choice test item: An information theoretic perspective. Educational and Psychological Measurement, 55(6), 959–966. doi:https://doi.org/10.1177/0013164495055006004Search in Google Scholar
Campbell, D. E. (2011). How to write good multiple-choice questions. Journal of Paediatrics & Child Health, 47(6), 322–325. doi:10.1111/j.1440-1754.2011.02115.xSearch in Google Scholar
Cizek, G. J., & O’Day, D. M. (1994). Further investigations of nonfunctioning options in multiple-choice test items. Educational and Psychological Measurement, 53(1), 241–247. doi:10.1177/0013164494054004002Search in Google Scholar
Crehan, K. D., Haladyna, T. M., & Brewer, B. W. (1993). Use of an inclusive option and the optimal number of options for multiple-choice items. Educational and Psychological Measurement, 53(1), 241–247. doi:10.1177/0013164493053001027Search in Google Scholar
Dehnad, A., Nasser, H., & Hosseini, A. F. (2014). A comparison between three-and four-option multiple choice questions. Social and Behavioral Sciences, 98, 398–403. doi:10.1016j.sbspro.2014.03.432Search in Google Scholar
Dickinson, M. (2011). Writing multiple-choice questions for higher-level thinking. Learning Solutions Magazine. Retrieved from http://www.learningsolutionsmag.com/articles/804/writing-multiple-choice-questions-for-higher-level-thinkingSearch in Google Scholar
Downing, S. M. (2005). The effects of violating standard item-writing principles on tests and students: The consequences of using flawed test items on achievement examinations in medical education. Advances in Health Science Education, 10(2), 133–143. doi:10.1007/s10459-004-4019-5Search in Google Scholar
Edwards, B. D., Arthur, W., & Bruce, L. L. (2012). The three-option format for knowledge and ability multiple-choice tests: A case for why it should be more commonly used in personnel testing. International Journal of Selection & Assessment, 20(1), 65–81. doi:10.1111/j.1468-2389.2012.00580.xSearch in Google Scholar
Farley, J. K. (1989). The multiple-choice test: Developing the test blue-print. Nurse Educator, 14(5), 3–5. doi:10.1097/NNE.0000000000000672Search in Google Scholar
Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082–1116. doi:https://doi.org/10.3102/0034654317726529Search in Google Scholar
Gierl, M. J., Lai, H., Pugh, D., Touchie, C., Boulais, A. P., & De Champlain, A. (2016). Evaluating the psychometric characteristics of generated multiple-choice test items. Applied Measurement in Education, 29(3), 196–210. doi:10.1080/08957347.2016.1171768Search in Google Scholar
Haladyna, R. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurements in Education, 15(3), 309–334. doi:10.1207/S15324818AME1503_5Search in Google Scholar
Haladyna, T. M., & Downing, S. M. (1988). Functional distracters: Implications for test-item writing and test design. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.Search in Google Scholar
Haladyna, T. M., & Downing, S. M. (1989). A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2(1), 37–50. doi:10.1207/s15324818ame0201_3Search in Google Scholar
Haladyna, T. M., & Downing, S. M. (1993). How many options is enough for a multiple-choice item? Educational and Psychological Measurement, 53(4), 999–1010. doi:10.1177/0013164493053004013Search in Google Scholar
Hijji, B. M. (2017). Flaws of multiple choice questions in teacher-constructed nursing examination: A pilot descriptive study. Journal of Nursing Education, 56(8), 490–496. doi:10.3928/01484834-20170712-08Search in Google Scholar
Kilgour, J. M., & Tayyaba, S. (2016). An investigation into the optimal number of distractors in single-best answer exams. Advances in Health Science Education, 21, 571–585. doi:10.1007/s10459-015-9652-7Search in Google Scholar
Lee, H., & Winke, P. (2012). The differences among three-, four-, and five-option-item formats in the context of a high-stakes English-language listening test. Language Testing, 30(1), 99–123. doi:https://dx.doi.org/10.1177/0265532212451235Search in Google Scholar
Leung, S. F., Mok, E., & Wong, D. (2008). The impact of assessment methods on the learning of nursing students. Nurse Education Today, 28(6), 711–719. doi:https:doi.org/10.1016/j.nedt.2007.11.004Search in Google Scholar
Little, J., & Bjork, E. (2015). Optimizing multiple-choice tests as tools for learning. Memory & Cognition, 43(1), 14–26. doi:10.3758/s13421-014-0452-8Search in Google Scholar
March, A. L., & Robinson, C. (2015). Assessment of high-stakes testing, hopeful thinking, and goal orientation among baccalaureate nursing students. International Journal of Nursing Education Scholarship, 12(1), 1–7. doi:10.1515/ijnes-2014-0075Search in Google Scholar
National Council of State Boards of Nursing (2017). NCLEX and other exams. Retrieved from https://www.ncsbn.org/nclex.htmSearch in Google Scholar
Nwadinigwe, P. I., & Naibi, L. (2013). The number of options in a multiple-choice test item and the psychometric characteristics. Journal of Education and Practice, 4(28), 189–196. doi:https://doi.org/10.7176/JEPSearch in Google Scholar
Oermann, M. H., & Gaberson, K. B. (2014). Evaluation and testing in nursing education. New York, NY: Spring Publishing Company.Search in Google Scholar
Papenberg, M., & Musch, J. (2017). Of small beauties and large beasts: The quality of distractors on multiple-choice tests is more important than their quantity. Applied Measurement in Education, 30(4), 273–286. doi:https://doi.org/10.1080/08957347.2017.1353987Search in Google Scholar
Piasentin, K. A. (2010). Exploring the optimal number of options in multiple-choice testing. Clear Exam Review, XXI(1), 18–22.Search in Google Scholar
Raymond, M. R., Stevens, C., & Bucak, S. D. (2019). The optimal number of options for multiple-choice questions on high-stakes tests: Application of a revised index for detecting nonfunctional distractors. Advances in Health Science Education, 24(1), 141–150. doi:https://doi.org/10.1007/s10459-018-9855-9Search in Google Scholar
Redmond, S. P., Hartigan-Rogers, J. A., & Cobbett, S. (2012). High time for a change: Psychometric analysis of multiple-choice questions in nursing. International Journal of Nursing Education Scholarship, 9(1), 1–16. doi:10.1515/1548-923X.2487Search in Google Scholar
Rodriguez, M. C. (2005). Three options are optimal for multiple-choice items: A meta-analysis of 80 years of research. Educational Measurement: Issues and Practice, 24(2), 3–13.10.1111/j.1745-3992.2005.00006.xSearch in Google Scholar
Rogausch, A., Hofer, R., & Krebs, R. (2010). Rarely selected distracters in high stakes medical multiple-choice examinations and their recognition by item authors: A simulation and survey. BMC Medical Education, 10(85), 1–9. doi:10.1186/1472-6920-10-85Search in Google Scholar
Royal, K. D., & Stockdale, M. R. (2017). The impact of 3-option responses to multiple-choice questions on guessing strategies and cut score determinations. Journal of Advances in Medical Education & Professionalism, 5(2), 84–89. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5346173/Search in Google Scholar
Rudner, L. M. (2010). Implementing the graduate management admission test computerized adaptive test. In W. Van Der Linden & C. Glas (Eds.), Elements of adaptive testing (pp.151–165). New York, NY: Springer.Search in Google Scholar
Schneid, S. D., Armour, C., Park, Y. S., Yudkowsky, R., & Bordage, G. (2014). Reducing the number of options on multiple-choice questions: Response time, psychometrics and standard setting. Medical Education, 48(10), 1020–1027. doi:10.1111/medu.12525Search in Google Scholar
Seinhorst, G. (2008). Are three options better than four? (Doctoral dissertation). Retrieved from http://www.natobilc.org/documents/Projects/MChoicesreadingtestSTUDY.pdfSearch in Google Scholar
Shizuka, T., Takeuchi, O., Yashima, T., & Yoshizawa, K. (2006). A comparison of three- and four-option English tests for university entrance selection purposes in Japan. Language Testing, 23(1), 35–57. doi:10.1191/0265532206lt319oaSearch in Google Scholar
Sidick, J. T., Barret, G. V., & Doverspike, D. (1994). Three-alternative multiple choice tests: An attractive option. Personal Psychology, 47(4), 829–835. doi:10.1111/j.1744-6570.1994.tb01579.xSearch in Google Scholar
Su, W., Osisek, P. J., Montgomery, C., & Pellar, S. (2009). Designing multiple-choice test items at higher cognitive levels. Nurse Educator, 34(5), 223–227. doi:10.1097?NNE.0b013e3181b2b546Search in Google Scholar
Tarrant, M., Knierim, A., Hayes, S., & Ware, J. (2006). The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Education in Practice, 6(6), 354–363. doi:10.1016/j.nepr.2006.07.002Search in Google Scholar
Tarrant, M., & Ware, J. (2008). Impact of item-writing flaws in multiple-choice questions on student achievement in high-stakes nursing assessments. Medical Education, 42(2), 198–206. doi:10.1111/j.1365-2923.2007.02957.xSearch in Google Scholar
Tarrant, M., & Ware, J. (2010). A comparison of the psychometric properties of three-and four-option multiple-choice questions in nursing assessments. Nurse Education Today, 30(8), 539–543. doi:10.1016/j.nedt.2009.11.002Search in Google Scholar
Tarrant, M., & Ware, J. (2012). A framework for improving the quality of multiple-choice assessments. Nurse Educator, 37(3), 98–104. doi:10.1097/NNE.0b013e3182504ld0Search in Google Scholar
Thanyapa, I., & Currie, M. (2014). The number of options in multiple choice items in language test: Does it make any difference? Language Testing in Asia, 4(8), 1–21. doi:https://doi.org/10.1186/s40468-014-0008-7Search in Google Scholar
Trevisan, M. S., Sax, G., & Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51(4), 829–837. doi:10.1177/001316449105100404Search in Google Scholar
Vegada, B., Shukla, A., Khilnani, A., Charan, J., & Desai, C. (2016). Comparison between three option, four option and five option multiple choice question tests for quality parameters: A randomized study. Indian Journal of Pharmacology, 48(5), 571–575. doi:10.4103/0253-7613.190757Search in Google Scholar
Vyas, R., & Supe, A. (2008). Multiple-choice questions: A literature review on the optimal number of options. The National Medical Journal of India, 21(3), 130–133. https://www.ncbi.nlm.nih.gov/pubmed/19004145Search in Google Scholar
Ware, J., & Vik, T. (2009). Quality assurance of item writing: During the introduction of questions in medicine for high stakes examinations. Medical Teacher, 31(3), 238–243. doi:10.1080/01421590802155597Search in Google Scholar
© 2019 Walter de Gruyter GmbH, Berlin/Boston