The use of machine translation in the field of migrations seems to be very limited and, in view of the latest developments, it is only natural to explore its usefulness in the migratory context. In an attempt to introduce this technology into this particular area, this article reports on a qualitative study on translators’ perceptions towards machine translation and post-editing tasks. The findings of the study indicate that both are not widely developed within the migratory context and further work is required. Based on our findings, we believe that this study can contribute to opening the way for machine translation and post-editing tasks to be included into the field of migrations.
Machine translation is now a mature technology in the translation industry. It is broadly used in the translation sector with a market share forecast of $638 million in 2020 (Van der Meer & Ruopp 2014; Massardo et al. 2016). The developments in neural technology in the field of artificial intelligence have outperformed other competing developments (both rule-based and statistical engines) (Moorkens 2018) to the point that for some authors it is “bridging the gap between human and machine translation” (Wu et al. 2016). Beyond the discussion of whether machine translation has already reached human parity (Läubli, Sennrich & Volk 2018), it is sound to explore what the real perceptions of professional translators are towards using this technology. Its advantages in speeding productivity and cost reduction have already been proved (Massardo et al., 2016) but now that machine translation is increasingly being incorporated in translation processes, the voice of the translator needs to be heard. In the present study, we will address this issue from a specific perspective: that of translators working in the migratory context.
2 Related work
Research into translators’ perception towards machine translation is very recent, perhaps because translators have traditionally been left out from the process of machine translation development, with a role reduced to reviewing the final product originated by the machine (Way & Hearne 2011). It is with the
advent of statistical machine translation first, and neural systems later, both marking an improvement in quality, that we begin to hear voices concerned about the translators’ opinion in research covering large representative samples of the sector. One of the first analyses is the work carried out within the project Optimale (2011). Under this framework, a survey conducted among a group of 700 employers in Europe indicated, among other things, that translation quality is more important than speed of delivery. Quality is also the main concern mentioned in the work of Gaspari, Almaghout and Doherty (2015), who surveyed a group of 438 translators, teachers and language service providers under the QTLaunchPad project. According to the data obtained in this study, the majority of respondents (68%) used free online machine translation systems and acknowledged that these were not adapted to their needs. It is not surprising then that translators were not satisfied with the quality obtained either.
This last idea leads, in turn, to the issue of the role machine translation plays: should we think of it as a tool that improves productivity? Does its use hinder the translation process when generating translations that are not considered to be of premium quality? In the study carried out by Koskinen and Ruokonen (2017) 70% of the translators interviewed indicated that technology was essential in their work and that the main reason for using it was to increase productivity. This is also perceived by those surveyed by Cadwell, O’Brien and Teixeira (2017), a group made up of 70 translators from the European Commission’s DGT and 20 from Alpha CRC who concluded that the main reason for using machine translation is speed and increased productivity. Respondents in this survey also noted that revising and editing the output of machine translation demands greater concentration than translating from scratch. In this case, it is then concluded that machine translation should be avoided whenever this occurs. Other issues highlighted by the study refer to the concern for terminological consistency which, according to the survey, must be carefully monitored when using machine translated text. However, the use of machine translation as a source of inspiration is also mentioned in some cases, although this seems to depend on whether or not the translator trusts the source of the corpus with which the translation system has been trained. Generally speaking, it seems that confidence in the system is directly related to the data used for its training (Heinisch and Lušicky 2019). Finally, with regard to quality, it is interesting to note the distinction made in the aforementioned study by Cadwell, O’Brien and Teixeira (2017) between the coincidences offered by a translation memory and the result of the automatic system. With regard to the latter, the perception is that errors are unpredictable and incoherent, thus generating mistrust, unlike what happens with memories, whose results are considered more reliable since the differences between partial and total coincidences are clearly indicated to the translator (Sánchez-Gijón, Moorkens and Way 2019).
Looking at this panoramic review of translators’ perceptions we acknowledge that even if it aims to cover a broad range of areas –institutional, academic and commercial- there is still an important one missing: nothing is said of translators in the third sector and, more specifically, of those working in migratory contexts. A series of questions arise then: Are these translators using machine translation, and, if so, how do they use it? Are they not using this technology? Why? If, on the contrary, they do use machine translation, what do they think about it? This article aims at specifically addressing these issues. It reports on a qualitative study carried out to evaluate translators’ perceptions on machine-translated text in the field of migration.
In order to know the perceptions of translators on the usefulness of machine translation in the migratory context, we carried out a study based on a qualitative methodology of exploratory-descriptive case study (Susam-Sarajeva 2009). In terms of the method used to collect the data in this study, we adopted a methodological triangulation approach (Silverman 2013: 212). First, we conducted an exhaustive review of the literature in this field in an attempt to detect points of interest and trends that could be explored. Then, we launched a call among translators willing to participate in the study and had them fill in a preliminary questionnaire containing information on their profile.
The questionnaire was created on the Google Forms online application to facilitate circulation and participation of respondents. The decision to use an online questionnaire was based essentially on the advantages of this modality compared to other possible options (e. g. in print version), such as effective and quick access to the study population, familiarisation of the population with this type of tools, immediate contact with the subjects and direct storage of the collected data (Morón 2010: 417-418). Finally, taking into account the information elicited from the questionnaire responses, we were able to select a specific group of translators who would take part in a focus group.
3.1 Participants’ profile
With the help of the preliminary questionnaire, as mentioned above, we selected a sample of ten translators. In this respect, it is important to point out that we consider this number to be representative enough for the type of qualitative analysis we wanted to conduct. We foresaw that the quality of the answers elicited from the discussion group would shed light into the nature of the translators’ perceptions towards machine translation. In any case, reaching a statistically representative number of translation professionals would be an enormous task that obviously exceeds the aim of the present study in terms of time and resources.
The selection of participants was made according to the following aspects: working languages, translation experience (number of years), academic qualification, area of expertise, experience with translation in the third sector, and use of machine translation. The final sample (10 participants) was made up of both in-house and freelance translators who work in Spain and carry out translations related to the migratory context. All had experience in translation projects in the third sector. Also, all but three had direct experience using machine translation tools when the focus group took place. The following table lists the profiles of the participants in the focus group:
3.2 Focus group design
The celebration of the focus group took place as part of a larger event organized by the authors of the present study in relation to translation in the third sector. The simultaneous celebration of both events gave us the opportunity to frame our study in a bigger context, thus gathering valuable information on the role of the translator working in the third sector. Additionally, we had the hope to stir the debate among professionals on the particularities of translation in the field of migration.
Participants in the focus group was limited to the sample of ten translators, as previously described. The session was divided into three parts. Each part concentrated in the evaluation of a different text that had been generated by a machine translation engine. In each of the three parts, participants were asked the same following questions:
Question 1: What do you think of this Spanish translation generated by the machine translation system?
Question 2: Would you have preferred to have the source text (original English text) available to evaluate the machine translation raw text?
Question 3: Would you accept an assignment to do the post-editing of this text? If yes, under what conditions?
For question 1 participants had to choose one of the following options:
The text does not need any post-editing.
The text needs post-editing (and, if so, they had to indicate what kind of errors they had detected).
It is necessary to rewrite the text.
|Speaker||Professional activity||Years in the translation industry||Experience in third sector projects||Experience in machine translation|
|A||Freelance||> 10 years||Yes (usual projects)||No (but has experience in other tools)|
|B||In-house||> 5 years||Yes (usual projects)||Yes|
|C||In-house||> 10 years||Yes (all projects)||Yes|
|D||In-house||> 10 years||Yes (usual projects)||Yes|
|E||Freelance||> 25 years||Yes (usual projects)||No|
|F||Freelance||> 15 years||Yes (occasional||Yes|
|G||In-house||> 25 years||Yes (all projects)||Yes|
|H||Freelance||> 15 years||Yes (usual projects)||Yes|
|I||Freelance||> 10 years||Yes (occasional||Yes|
|J||In-house||> 20 years||Yes (all projects)||No (but has experience in other tools)|
Questions 2 and 3 were open-ended and participants could add any comment on the translated text or on any other issue they considered relevant. In this respect, it is worth mentioning that for the purpose of preparing the moderator’s role and improving her effectiveness in the task, we took into account the recommendations of Arksey and Knight (1999: 38-41) regarding the following aspects: knowledge about the topic, body language and affective-emotional control.
The focus group took place in Spanish, in a single face-to-face session. Participants agreed to participate and collaborate in the study and they all expressed considerable interest in the session, which led to an increase in motivation for further research in this area. The entire session was recorded using two different recording devices so as to ensure a good audio quality from anywhere in the room and as a measure to have several backups in case of technical failure. At the end of the focus group, the audio recordings were transcribed to allow thematic analysis.
3.3 Selected texts and machine translation engine
Three texts in English related to the migratory field were chosen. A fragment of approximately 100 words was extracted from each of the three texts, and was subsequently machine translated. The selected fragments belonged to the following texts:
Commission Recommendation (EU) 2017/432 of 7 March 2017 on making returns more effective when implementing the Directive 2008/115/EC of the European Parliament and of the Council.
Balancing Acts: Policy Frameworks for Migrant Return and Reintegration (Migration Policy Institute).
World Migration Report 2018 (International Organization for Migration).
The machine translation engine used to generate the Spanish translation was the EU Council Presidency Translator. This is a neural machine translation tool developed by the language technology company Tilde, in collaboration with the Institute for Bulgarian Language and the Centre for Translation Studies at University of Vienna and support from the CEF eTranslation building block.
The tables below show the resulting raw translation (EN-ES) generated by the machine translation engine:
|Member States should ensure that return decisions have unlimited duration, so that they can be enforced at any moment without the need to re-launch return procedures after a certain period of time. This should be without prejudice to the obligation to take into account any change in the individual situation of the third-country nationals concerned, including the risk of refoulement.||Los Estados miembros deben garantizar que las decisiones de retorno tengan duración ilimitada, de modo que puedan ejecutarse en cualquier momento sin necesidad de volver a poner en marcha procedimientos de retorno tras un determinado período de tiempo. Esto debe entenderse sin perjuicio de la obligación de tener en cuenta cualquier cambio que se produzca en la situación individual de los nacionales de terceros países de que se trate, incluido el riesgo de devolución.|
|(7) Member States should systematically introduce in return decisions the information that third-country nationals must leave the territory of the Member State to reach a third country, to deter and prevent unauthorised secondary movements.||(7) Los Estados miembros deben incluir sistemáticamente en las decisiones de retorno la información de que el nacional de un tercer país deberá abandonar el territorio del Estado miembro para llegar a un tercer país, al efecto de prevenir y desincentivar los movimientos secundarios no autorizados.|
|(8) Member States should make use of the derogation provided for under Article 2(2)(a) of Directive 2008/115/ EC when this can provide for more effective procedures, in particular when facing significant migratory pressure.||(8) Los Estados miembros deben recurrir a la excepción prevista en el artículo 2, apartado 2, letra a), de la Directiva 2008/115/CE si esto facilita procedimientos más eficaces, sobre todo cuando sufran una presión migratoria considerable.|
|Increasingly, return and reintegration programming is being funded out of development budgets rather than the budgets of migration authorities. As discussed above, a compelling case can be made for the alignment of return and reintegration assistance with development cooperation— but the two should not be equated. They operate in different policy frameworks that are not necessarily contradictory, but that do need to be consciously coordinated to ensure that the policy goals of one approach do not undermine the goals of the other. Development cooperation can, and in some instances does, play a part in shaping reintegration assistance around the development needs of the countries and communities to which migrants return.||Cada vez más, el programa de reintegración y reintegración está financiado fuera del desarrollo, más que los presupuestos de los autores de la migración. Como ya se ha señalado anteriormente, puede presentarse un caso convincente para la armonización de la ayuda a la vuelta y la recomprensión con la cooperación para el desarrollo, pero los dos no deben equipararse. Operan en distintos marcos políticos que no son necesariamente contradictorios, sino que deben coordinarse de manera consciente para garantizar que los objetivos políticos de un enfoque no socaven los objetivos de la otra. La cooperación al desarrollo puede, yen algunos casos, participar en la formación de la ayuda a la reintegración en torno a las necesidades de desarrollo de los países y las comunidades de servicios a los que regresan los migrantes.|
|Minority ethnic retail areas provide a significant challenge for urban planners, given the complexity of the uses to which ethnic communities may put these areas, which can go well beyond shopping for goods. Zhixi Cecilia Zhuang’s recent work on these districts in Toronto offers strong directions for planners. It is in the small aspects of daily life, such as shopping and moving about through one’s community and engaging with one’s fellow residents, that a city’s degree of inclusiveness is revealed. Planning decisions have a significant impact on the quality of these daily life experiences, which is the point of multicultural planning. High-level policy statements are meaningless if they are not implemented through urban plans.||Las minorías étnicas minoritarias suponen un reto importante para los urbanistas, dada la complejidad de los usos a los que pueden acceder las comunidades étnicas, lo que puede ir mucho más allá de las compras de bienes. El trabajo reciente de Zhixi Cecilia Zhuang de estos distritos en Toronto ofrece importantes direcciones para la planificación. Es en los aspectos más pequeños de la vida cotidiana, como las compras y el desplazamiento a través de la comunidad y la participación de los demás residentes, que el grado de inclusión de la ciudad es susceptible de inclusión. Las decisiones de planificación tienen un impacto significativo en la calidad de estas experiencias de la vida cotidiana, que es el punto de planificación multicultural. Las declaraciones políticas de alto nivel carecen de sentido si no se ejecutan a través de los planes urbanos.|
4 Findings and discussion
The qualitative data obtained from the focus group was recorded, transcribed, and then analysed to identify the main ideas in accordance with the content analysis approach (Ravitch and Mittenfelner 2015), thus ensuring the validity and reliability of the data. As already pointed out, the focus group session was structured in three different parts and three different texts were used to know translators’ perceptions towards machine translation. Due to the qualitative nature of the study, the researcher guided the session to improve the validity of the qualitative study by means of negotiating some controversial issues that arose, which could mislead the research. At a first stage, translators were required to read three different texts that had been previously translated using a machine translation system. After reading these texts, they had to answer three questions and were asked to choose among three options evaluating the quality of the translated text.
With regards to Question 1 (What do you think of this Spanish translation generated by the machine translation system?), the content analysis identifies topics closely related to an unreliable attitude both to machine translation and post-editing practices. The vast majority of the participants agreed that, although the result of machine translation was acceptable, post-editing tasks were required. However, and despite this initial lack of confidence, participants were aware that both machine translation and post-editing were gaining popularity among the translator professional market.
After showing their general perception on the translation provided by the machine translation system, Question 2 (Would you have preferred to have the source text (original English text) available to evaluate the MT raw text?) tried to elicit participants’ opinions about the information translators need in order to ensure the quality of the raw machine translated text. Participants claimed that it was essential to have the source text. Most of them pointed out that using the source text as reference material should be the first requirement in order to accept a post-editing task. In fact, participants listed numerous advantages for using the source text, including being able to identify translation errors, such as omissions, contextualizing the text or avoiding terminological, grammatical and syntactical errors. The following quotes demonstrate this point:
Speaker F: When you have the original, you are able to see the mistakes, if not, it is extremely difficult.
Speaker E: Without the original, the task is impossible.
Speaker I: I can’t tell if there’s an omission if I don’t have the original and a problem that many automatic translators have is that they omit even a complete phrase.
As for Question 3 (Would you accept an assignment to do the post-editing of this text? If yes, under what conditions?), concerning participants’ willingness to accept a post-editing task, none of them would accept it without the original version of the source text. Additionally, all of them mentioned that in order to perform adequate post-editing a set of clear post-editing guidelines would be needed. Unfortunately, availability of such guidelines is not always possible. Some actual opinions on this issue are as follows:
Speaker H: If I receive a project of this type I would ask for the original text.
Speaker E: I’ve done it. They have been texts translated using Google Translate or similar. They practically have to be translated again. They send them to you to check because they don’t agree with the content or something is wrong.
Speaker F: If you receive a postediting project like this, I think it is necessary to have the original text to see what type of engine has been used and to know the purpose of the translation (if it is for internal use, for instance).
As previously mentioned, participants had to comment on the different options provided by the machine translation system for the three different texts. All the participants agreed that all texts needed revision. All of them reported instances of terminological inconsistencies, false sense examples and lack of syntactic and stylistic coherence. However, some translators pointed out that machine translation was almost ‘perfect’ and it resembled a human translation, for instance in Text 1:
Speaker A: I must admit I’m a bit worried as I don’t see any major errors, I mean, the text is a very specific legal text, but it sounds good to me… but perhaps a lawyer reads it and there are mistakes I’m not able to identify […]
Translators noted during the focus group that machine translation engines were able to provide translations that are grammatically, syntactically and semantically correct, but they needed to be revised by translators, and domain-specific professionals in some cases, in order to obtain a ‘good translation’. They mentioned examples where anaphoric references were needed to be included instead of repetitions, or instances of omissions in the raw machine translation output.
Although the three texts were similar in content –all of them were specialized texts pertaining to the European Union institutional domain–, the participants agreed that Text 1 offered a better quality that the rest, as Text 1 reported some minimal errors. For Text 2, for instance, participants agreed that it needed a major revision as more mistakes were identified, such as grammatical and lexical inconsistencies:
Speaker I: Sometimes machine translation systems reproduce coordinated structures as they were the same word […]
Due the great amount of errors in Text 2 and Text 3, a debate started regarding the texts used to train machine translation engines. Participants stated that machine translation works better with short-sentences texts.
Speaker J: Perhaps this type of machine translation engines works better with texts containing shorter sentences and simpler structures, even if it is an administrative-legal text.
5 Concluding remarks
We are aware that the corpus of this study is very limited, and that further data would be needed in order to reach a thorough conclusion. Nevertheless, we believe the small data we provide is valuable in the sense that it accounts for a specific sector of the translation profession that usually finds difficulty in making its voice heard. Our study also provides valuable results to fuel the existing debate on machine translation and translators’ perception, and more closely translators’ perception in the migratory context.
The findings of the current study state that the participants were generally negative about machine translation and post-editing, but they were aware that both tasks are gaining popularity in the translator professional market. However, the need of the human translator cannot be avoided as, even if the machine translation engine results are similar to a human translation, they still need to be post-edited to correct terminological, grammatical and syntactical errors. Participants also highlighted the importance of having access to the original texts when performing post-editing tasks. At the same time, they required detailed guidelines or instructions if post-editing tasks have to be performed.
Although the results are limited in some way, the study reflects how participants are aware of the place machine translation and post-editing task are starting to take place within the third sector.
Cadwell, Patrick, Sharon O‘Brien, Carlos S. C. Teixeira. 2017. Resistance and accommodation: factors for the (non-) adoption of machine translation among professional translators. Perspectives. Studies in Translation Theory and Practice. DOI: http://dx.doi.org/https://doi.org/10.1080/0907676X.2017.133721010.1080/0907676X.2017.1337210Suche in Google Scholar
Gaspari, Federico, Hala Almaghout, and Stephen Doherty. 2015. „A survey of machine translation competences: Insights for translation technology educators and practitioners.“ Perspectives: Studies in Translatology 23 (3): 333-58.10.1080/0907676X.2014.979842Suche in Google Scholar
Heinisch, Barbara, and Vesna Lusicky. 2019. „User expectations towards machine translation: A case study.“ Proceedings of MT Summit XVII 2:42-48. https://docs.wixstatic.com/ugd/705d57_b3899f00d8f348299c2915186221bab2.pdf accessed 28 August 2019Suche in Google Scholar
Koponen, Maarit. 2012. Comparing human perceptions of post-editing effort with post-editing operations. In Proceedings of the 7th workshop on statistical machine translation, 7-8 June 2012, Montreal, Canada. 181-190. New York: Association for Computational Linguistics.Suche in Google Scholar
Koskinen, Kaisa, and Minna Ruokonen. 2017. „Love letter or hate mail? Translators‘ technology acceptance in the light of their emotional narratives.“ In Human Issues in Translation Technology, ed. Dorothy Kenny. Londres: Routledge.Suche in Google Scholar
Laubli, Samuel, Rico Sennrich, Martin Volk. 2018. Has Machine Translation Achieved Human Parity? A Case for Document-level EvaluationW. Computing Research Repository arXiv https://arxiv.org/abs/1808.07048v1 ( accessed 10 March 2019).Suche in Google Scholar
Lavie, Alon. 2010. Evaluating the Output of Machine Translation Systems, AMTA 2010 Tutorial, 31 October 2010, Denver, Colorado, USA. https://amta2010.amtaweb.org/AMTA/papers/6-04-LavieMTEvaluation.pdf (accessed 17 October 2017).Suche in Google Scholar
Massardo, Isabella, Jaap van der Meer, and Maxim Khalilov. 2016. TAUS Translation Technology Landscape Report. De Rijp, The Netherlands: TAUS BV.Suche in Google Scholar
Moorkens, Joss, Sharon O‘Brien, Igor A. L. da Silva, Norma B. de Lima Fonseca, and Fabio Alves. 2015. „Correlations of perceived post-editing effort with measurements of actual effort.“ Machine Translation 29 (3-4): 267-84.10.1007/s10590-015-9175-2Suche in Google Scholar
Moorkens, Joos. 2018. „What to expect from Neural Machine Translation: A practical in-class translation evaluation exercise.“ Interpreter and Translator Trainer 12 (4): 375-87.10.1080/1750399X.2018.1501639Suche in Google Scholar
Moron, M.a de los Angeles. 2010. Percepciones sobre el impacto de la movilidad en la formacion de traductores: la experiencia de los graduados en el programa LAE (Perceptions of the impact of mobility on the training of translators: the experience of graduates of the LAE programme). PhD thesis. University of Granada, Spain.Suche in Google Scholar
Optimale (2011). Optimale employer survey. http://www.ressources.univ-rennes2.fr/service-relations-internationales/optimale/attachments/article/52/WP4_Synthesis_report.pdf (accessed 23 January 2018).Suche in Google Scholar
Ravitch, S., and N. Mittenfelner. 2016. Qualitative research. Bridging the conceptual, theoretical, and methodological. Thousand Oaks: Sage.Suche in Google Scholar
Rico, C. 2018. „Traduccion y tercer sector social: ?voluntariado o profesion?“, Boletin de los traductores espanoles de las instituciones de la Union Europea. Available: http://ec.europa.eu/translation/spanish/magazine/es_magazine_es.htm Octubre/noviembre/diciembre de 2018 n.o 160Suche in Google Scholar
Sanchez-Gijon, Pilar, Joss Moorkens, and Andy Way. June 2019. „Post-editing neural machine translation versus translation memory segments.“ Machine Translation 33 (1-2): 31-59. https://doi-org.ezproxy.universidadeuropea.es/10.1007/s10590-019-09232-x10.1007/s10590-019-09232-xSuche in Google Scholar
Silverman, David. 2013. Doing qualitative research: A practical handbook. 4th ed. Thousand Oaks, CA: Sage.Suche in Google Scholar
Teixeira, Carlos S. C.2014. Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories. In O‘Brien, Sharon, Michel Simard, Lucia Specia (eds.) Proceedings of the 11th Conference of the Association for Machine Translation in the Americas: Workshop on Post-Editing Technology and Practice (WPTP3). 450-459. Vancouver: AMTA.Suche in Google Scholar
van der Meer, Jaap, and Achim Ruopp. 2014. Machine Traslation Market Report. De Rijp, The Netherlands: TAUS BV.Suche in Google Scholar
Way, Andy, and Mary Hearne. 2011. „On the Role of Translators in State-of-the-Art Statistical Machine Translation.“ Language and Linguistics Compass 5 (5): 227-48.10.1111/j.1749-818X.2011.00275.xSuche in Google Scholar
Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean. 2016. Google‘s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. Computing Research Repository arXiv:16 09.08144. https://arxiv.org/abs/1609.08144 (accessed 10 January 2018).Suche in Google Scholar
© 2020 Lorena Pérez Macías et al., published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.