Abstract

Multiple-choice (MC) tests are frequently used to measure language competence because they are quick, economical and straightforward to score. While degrees of correctness have been investigated for partially correct responses in combined-response MC tests, degrees of incorrectness in distractors and the role they play in determining the test-taker's final score remain comparatively unexplored. This pilot study examines degrees of incorrectness in MC test items and their potential impact on the overall reliability of scoring procedures, test micro impact, and scoring performance of individual test-takers. Investigating the above properties as quality dimensions of language testing is crucial since decisions made on the basis of such tests influence the careers and lives of test-takers. Three scoring procedures were applied to an authentic test: dichotomous, which is straightforward – correct/incorrect; polychotomous, which offers partial credit to very plausible and plausible options; and negative, which includes negative scoring when a wholly incorrect answer is selected. While these three procedures proved to be equally reliable, the polychotomous scoring method offered more sensitive and precise results, while negative scoring was particularly disadvantageous for test-takers. The polychotomous method of scoring moves away from the absolute right or wrong view and enhances sensitivity in the field of individualised language testing.