EVALUATING THE EVALUATOR: TRANSLATION QUALITY IN THE INDUSTRY
DOI:
https://doi.org/10.17721/2520-6397.2025.1.01Keywords:
translation quality, quality evaluation, blind translator experiment, evaluatorAbstract
Background. A considerable number of studies have been devoted to specialized translation quality assessment, resulting in the development of various universal or specific approaches and models. However, the industry domain remains understudied, particularly the principles, approaches, and work quality of entities providing translation quality evaluation services. This study aims to identify and evaluate the relevant principles and criteria used by collective providers of translation quality evaluation services through the lens of ISO 17100 and ISO 5060 standards. Additionally, this article focuses on evaluating specialists' sensitivity to translation quality.
Methods. The comprehensively formulated methodology encompasses a blind translator experiment incorporating two collective providers of translation quality evaluation services, comparative translation analysis and related comparative-descriptive method, as well as the method of generalization.
Results. The experimental results demonstrated that Ukrainian professional service providers' approaches to translation quality evaluation conceptually do not align with the principles outlined in ISO 17100 and ISO 5060 standards, as none of the participants categorized errors or provided translation quality evaluation according to any scale. The Ukrainian translation market shows reduced sensitivity of service providers to translation quality in terms of terminology and its consistency, translation accuracy, language norms, and other categories. The evaluators' interventions were characterized by inconsistency, selectivity, and occasionally low quality.
Conclusions. The results of this study revealed conceptual flaws in approaches to specialized translation quality evaluation in the industry domain, low sensitivity of service providers to translation quality, and a lack of awareness regarding the multifaceted nature of quality. These findings indicate the need to develop a universal translation quality evaluation system and highlight the importance of training future specialists in fundamental principles of specialized translation quality evaluation services.
References
Biriukov, A. V. (2007). Machine translation quality evaluation. Extended abstract of candidate's thesis. Kyiv: KNU. Vernadsky National Library of Ukraine [in Ukrainian]. http://194.44.11.130/aref/20081124032372
Bittner, H. (2020). Evaluating the Evaluator: A Novel Perspective on Translation Quality Assessment. Routledge. https://doi.org/10.12807/ti.115202.2023.r03
Brien, S. O. (2012). Towards a dynamic quality evaluation model for translation. Journal of Specialised Translation, 17(A), 55-77.
Chernovatyi, L. M. (2013). Methodology of Teaching Translation as a Specialty. Nova Knyha [in Ukrainian].
Chesterman, A., & Wagner, E. (2002). Can Theory Help Translators?: A Dialogue Between the Ivory Tower and the Wordface. Routledge.
Daems, J. (2016). A translation robot for each translator?: a comparative study of manual translation and post-editing of machine translations: Process, quality and translator attitude. doctoral dissertation. [PhD Thesis, Ghent University]. Universiteit Gent. https://biblio.ugent.be/publication/8058017
Drugan, J. (2013). Quality in Professional Translation: Assessment and Improvement . Bloomsbury. https://www.bloomsbury.com/uk/quality-in- professional-translation-9781441149541
Hague, D., Melby, A., & Zheng, W. (2011). Surveying Translation Quality Assessment A Specification Approach. Interpreter and Translator Trainer, 5, 243-267.
Honig, H. G. (2010). Positions, Power and Practice: Functionalist Approaches and Translation Quality Assessment. Current Issues in Language and Society, 4, 6-34. https://doi.org/10.1080/13520529709615477
House, J. (2014). Translation Quality Assessment: Past and Present. Routledge.
International Organization for Standardization. (2015). Translation services - Requirements for translation services (ISO Standard № 17100:2015). https://www.iso.org/standard/59149.html
International Organization for Standardization. (2024). Translation services - Evaluation of translation output - General guidance (ISO Standard № 5060:2024). https://www.iso.org/standard/80701.html
Lauscher, S. (2000). Translation quality assessment: where can theory and practice meet? The Translator, 6(2), 149-168.
Lommel, A., Gladkoff, S., Melby, A., Wright, S., Strandvik, I., Gasova, K., Vaasa, A., Benzo, A., Marazzato Sparano, R., Foresi, M., Innis, J., Han, L., & Nenadic, G. (2024). The Multi-Range Theory of Translation Quality Measurement: MQM scoring models and Statistical Quality Control. https://doi.org/10.48550/arXiv.2405.16969
Mishchenko, A. L. (2014). Analysis of Ukrainian translations of technical documentation texts. Research Bulletin. Ser.: Philological Sciences, 126, 194-205 [in Ukrainian].
Saiko, K. O., & Saiko, M. A. (2024). Blind translator experiment: per aspera ad quality. Research Bulletin. Ser.: Philological Sciences, 210(3), 265-273 [in Ukrainian]. https://doi.org/10.32782/2522-4077-2024-210-38
Saiko, K. O., & Saiko, M. A. (2024). Component Consistency as One of the Aspects of German-Ukrainian Specialized Translation Quality Assurance. Alfred Nobel University Journal of Philology, vol. 28, issue 2, 363-379 [in German]. https://doi.org/10.32342/3041-217X-2024-2-28-22
Schaffner, C. (1998). From "Good" to "Functionally Appropriate": assessing translation quality. Translation and quality, 4, 1-5.
Williams, M. (2004). Translation quality assessment: An argumentation- centred approach. University of Ottawa Press.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Linguistic and Conceptual Worldviews
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
