Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Text & Talk

An Interdisciplinary Journal of Language, Discourse & Communication Studies

Ed. by Sarangi, Srikant


IMPACT FACTOR 2017: 0.426
5-year IMPACT FACTOR: 0.724

CiteScore 2017: 0.63

SCImago Journal Rank (SJR) 2017: 0.326
Source Normalized Impact per Paper (SNIP) 2017: 0.625

Online
ISSN
1860-7349
See all formats and pricing
More options …
Volume 34, Issue 5

Issues

The linguistic accuracy of chatbots: usability from an ESL perspective

David Coniam
  • Corresponding author
  • Department of Curriculum and Instruction, The Hong Kong Institute of Education, Tai Po, Hong Kong
  • Email
  • Other articles by this author:
  • De Gruyter OnlineGoogle Scholar
Published Online: 2014-08-29 | DOI: https://doi.org/10.1515/text-2014-0018

Abstract

This paper reports on the linguistic accuracy of five renowned “chatbots,” with an evaluator (an ESL teacher) chatting with each chatbot for about three hours. The chatting consisted of a series of set questions/statements (determined as being in the domain of an ESL learner) – aimed at assessing the accuracy and felicity of the chatbots' answers at the grammatical level. Results indicate that chatbots are generally able to provide grammatically acceptable answers, with three chatbots returning acceptability figures in the 90% range. When meaning is factored in, however, a different picture emerges, with the chatbots often providing meaningless, nonsensical answers, and the accuracy rate for the joint categories of grammar and meaning falling below 60%. The paper concludes on the note that although chatbots as “conversation practice machines” do not yet make robust chatting partners, improvements in chatbot performance bode well for future developments.

Keywords: chatbots; English language; ESL; computer analysis of language, syntax

About the article

David Coniam

David Coniam is Chair Professor in the Faculty of Education Studies at the Hong Kong Institute of Education, where he is a teacher educator, working with ESL teachers in Hong Kong secondary schools. His main publication and research interests are in language teaching methodology and language assessment.


Published Online: 2014-08-29

Published in Print: 2014-09-01


Citation Information: Text & Talk, Volume 34, Issue 5, Pages 545–567, ISSN (Online) 1860-7349, ISSN (Print) 1860-7330, DOI: https://doi.org/10.1515/text-2014-0018.

Export Citation

©2014 by Walter de Gruyter Berlin/Boston.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

[1]
Luke Fryer, Kaori Nakao, and Andrew Thompson
Computers in Human Behavior, 2018

Comments (0)

Please log in or register to comment.
Log in