Assessing the use of multiple-choice translation items in English proficiency tests: The case of the national English proficiency test in Turkey

Betül Hazal Dinçerhttp://orcid.org/https://orcid.org/0000-0002-7730-9327 1 , Elena Antonova-Unlu 2 , and Alper Kumcu 2
  • 1 Baskent University, Baglica Campus, 06810, Ankara, Turkey
  • 2 Hacettepe University, Beytepe Campus, Ankara, Turkey
Betül Hazal DinçerORCID iD: https://orcid.org/0000-0002-7730-9327, Elena Antonova-Unlu and Alper Kumcu

Abstract

The use of translation for language teaching and assessment, by and large, has been abandoned with the adoption of audio-lingual and communicative approaches in language teaching. As a result, nowadays translation items are not commonly used for measuring language proficiency in international language proficiency tests (e. g. TOEFL, IELTS).

However, there are several countries that still use translation items in their national language proficiency tests (e. g. Turkey, Japan, China, Romania among others). The present study aims to examine whether or not multiple-choice translation items are an appropriate tool for measuring proficiency in English. To this end, the perceived level of difficulty and validity of multiple-choice translation items in the National English Proficiency Test (YDS) in Turkey were examined. The findings revealed that the participants did significantly better on the translation items than on the rest of the test items. They also perceived the translation items as the easiest among all the rest items in YDS. Moreover, while YDS as a whole indicated a strong validity based on correlation with TOEFL PBT Reading Sample Test, the translation items indicated moderate validity. Importantly, there was a significant difference between the two correlations. These findings suggest that multiple-choice translation items are likely to lower the overall validity of YDS tests, inflate the scores of test-takers and, thus, might be considered as problematic for the quality of the tests.

  • Aitken, Kenneth G. 1975. Problems in cloze testing re-examined. TESL Reporter 8(2). 9–16.

  • Bates, Douglas M, Martin Mächler, Ben Bolker & Steve Walker. 2015. Fitting linear mixed-effects models using Lme4. Journal of Statistical Software 67(1). 1–48. .

    • Crossref
    • Export Citation
  • Brown, H. Douglas. 2007. Principles of language learning and teaching. White Plains: Pearson Longman.

  • Buck, Gary. 1992. Translation as a language testing procedure: Does it work? Language Testing 9(2). 123–148. .

    • Crossref
    • Export Citation
  • Calfoglou, Christine. 2013. An optimality translation proposal for the foreign language class. In Dina Tsagari & Georgios Floros (eds.), Translation in language teaching and assessment, 93–114. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.

  • Carreres, Angeles. 2006. Strange bedfellows: Translation and language teaching: The teaching of translation into L2 in modern languages degrees; Uses and limitations. Sixth Symposium on Translation, Terminology and Interpretation in Cuba and Canada. Canadian Translators, Terminologists and Interpreters Council, 1–21.

  • Catford, John Cunnison. 1965. A linguistic theory of translation: An essay in applied linguistics. Oxford: Oxford University Press.

  • Diedenhofen, B. & J. Musch. 2015. Cocor: A comprehensive solution for the statistical comparison of correlation. PLoS ONE. .

    • Crossref
    • Export Citation
  • Hughes, Arthur. 1989. Testing for language teachers. Cambridge: Cambridge University Press.

  • Kikuchi, Keita. 2006. Revisiting English entrance examinations at Japanese universities after a decade. JALT Journal 28(1). 77–96.

    • Crossref
    • Export Citation
  • Klein-Braley, Christine. 1987. Fossil at large: Translation as a language testing procedure. In R. Grotjahn, C. Klein-Braley & D.K. Stevenson (eds.), Taking their measure: The validity and validation of language tests, 111–132. Bochum: Brockmeyer.

  • Klein-Braley, Christine & Veronica Smith. 1985. Incalculable and full of risks?’ Translation L1 to L2 as a testing procedure. In Christopher Titford & A. E. Hieke (eds.), Translation in foreign language teaching and testing, 155–168. Tübingen: Gunter Narr Verlag Tübingen.

  • Korošec, Melita Koletnik. 2013. Teaching grammar through translation. In Dina Tsagari & Georgios Floros (eds.), Translation in language teaching and assessment, 23–40. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.

  • Lado, Robert. 1961. Language testing: The construction and use of foreign language tests : A teacher’s book. Longmans: London.

  • Lee, Tzu-yi. 2013. Incorporating translation into the language classroom and its potential impacts upon L2 learners. In Dina Tsagari & Georgios Floros (eds.), Translation in language teaching and assessment, 3–22. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.

  • Munday, Jeremy. 2013. Introducing translation studies: Theories and applications, 2nd edn. Amsterdam: Routledge.

  • Nida, Eugene A. 1964. Toward a science of translating: With special reference to principles and procedures involved in Bible translating. Leiden: E. J. Brill.

  • ÖSYM. 2019. Yabancı Dil Seviye Tespit Sınavı (YDS) Kılavuzu. https://dokuman.osym.gov.tr/pdfdokuman/2019/YDS-1/kilavuz30012019.pdf (accessed 1 April 2019)

  • ÖSYM. Yabancı Dil Seviye Tespit Sınavı (YDS)İngilizce İLkbahar Dönemi 1 Nisan 2018.https://dokuman.osym.gov.tr/pdfdokuman/2018/YDSILKBAHAR/CS/ING24072018.pdf (accessed 1 November 2018).

  • Pintado-Gutiérrez, Lucía. 2018. Translation in language teaching, pedagogical translation, and code-switching: Restructuring the boundaries. Language Learning Journal 1–21. .

    • Crossref
    • Export Citation
  • R Core Team. 2019. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.r-project.org/.

  • Revelle, William. 2018. Psych: Procedures for personality and psychological research. Evanston, Illinois, USA: Northwestern University. https://cran.r-project.org/package=psych.

  • Richards, Jack C. & Theodore S. Rodgers. 2001. Approaches and methods in language teaching. Cambridge: Cambridge University Press.

  • Rivers, Wilga M. 1981. Teaching foreign-language skills. Chicago: University of Chicago.

  • Schober, Patrick & Lothar A. Schwarte. 2018. Correlation coefficients: Appropriate use and interpretation. Anesthesia and Analgesia 126(5). 1763–1768. .

    • Crossref
    • PubMed
    • Export Citation
  • Snell-Hornby, Mary. 1985. Translation as a means of integrating language teaching and linguistics. In Christopher Titford & A. E. Hieke (eds.), Translation in foreign language teaching and testing, 21–28. Tübingen: Gunter Narr Verlag Tübingen.

  • Stevenson, Douglas K. 1985. In other words: Language testers and translation tests. In Christopher Titford & A. E. Hieke (eds.), Translation in foreign language teaching and testing, 137–153. Tübingen: Gunter Narr Verlag Tübingen.

  • Sun, Youyi & Liying Cheng. 2013. Assessing second/foreign language competence using translation: The case of the college English test in China. In Dina Tsagari & Georgios Floros (eds.), Translation in language teaching and assessment, 235–252. Newcastle upon Tyne, UK: Cambridge Scholars Publishing.

  • Vinay, Jean-Paul & Jean Darbelnet. 1995. Comparative stylistics of French and English: A methodology for translation. Translated by Juan C. Sager and Marie-Josée Hamel. Amsterdam and Philadelphia: John Benjamins Publishing.

  • Weir, Cyril J. 2005. Language testing and validation. Palgrave Macmillan UK: Basingstoke.

Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

Search