READABILITY ASSESSMENT OF AI-DRIVEN INDONESIAN JOURNALISTIC TRANSLATIONS: COMPARING CHATGPT AND DEEPL PERFORMANCE
DOI:
https://doi.org/10.22216/7ngabs09Keywords:
ChatGPT, DeepL, Indonesian to English translation, Journalistic Texts, ReadabilityAbstract
This study investigates the readability of Indonesian to English translations of journalistic texts produced by ChatGPT and DeepL across the domains of law, politics, and environment using articles from Tempo and Kompas. A qualitative descriptive comparative design was employed. Six source texts were translated under controlled procedures, with standardized prompts for ChatGPT and standard input for DeepL. Two expert raters evaluated segment level readability using a three point rubric adapted from prior work. Before scoring, the raters reviewed and aligned category descriptors to support shared interpretation. Readability scores were summarized by outlet and topic, and rater comments from a focused discussion were used to contextualize patterns. The findings show small and patterned differences between systems. One rater tended to rate ChatGPT as more readable, while the other found broad parity. Readability was highest for environmental reporting and most challenging for legal texts that contain dense institutional references. Kompas items were generally read as more accessible than Tempo items. The results point to domain sensitive strengths for each system and suggest targeted post editing as a practical lever to support clarity for public facing news.
Downloads
References
AlAfnan, M. A. (2024). Large language models as computational linguistics tools: A comparative analysis of ChatGPT and Google machine translations. Journal of Artificial Intelligence and Technology, Advance online publication. doi:10.37965/jait.2024.0549
Basha, J. Y. (2024). The negative impacts of AI tools on students in academic and real-life performance. International Journal of Social Sciences and Commerce, 1(3), 1–16. doi:10.51470/IJSSC.2024.01.03.01
Castilho, S., Quinn Mallon, C., Meister, R., & Yue, S. (2023). Do online machine translation systems care for context? What about a GPT model?. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, 393–417. European Association for Machine Translation.
Çetin, Ö., & Duran, A. (2024). A comparative analysis of the performances of ChatGPT, DeepL, Google Translate, and a human translator in community-based settings. Amasya Üniversitesi Sosyal Bilimler Dergisi, 9(15), 120–173.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Thousand Oaks, CA: SAGE.
Fitriani, E., Miru, A. S., Lamada, M. S., & Fitrani, E. (2024). Analisis tentang pemahaman, persepsi dan aspek ChatGPT oleh mahasiswa [Analysis of students’ understanding, perceptions, and aspects of ChatGPT use]. Jurnal Ilmiah Multidisipliner (JIMU), 1(1), 1–12.
Gadd, S. (2024). Data and AI governance. Journal of Financial Transformation, 59, 8–19.
Gao, R., Lin, Y., Zhao, N., & Cai, Z. G. (2024). Machine translation of Chinese classical poetry: A comparison among ChatGPT, Google Translate, and DeepL. Humanities and Social Sciences Communications, 11, Article 835. doi:10.1057/s41599-024-03363-0
Handoyo, E. R., Sugiarto, J., Lolo, A., & Chai, K. (2023). Identifikasi pengaruh penggunaan ChatGPT terhadap kemampuan berpikir mahasiswa [Identifying the influence of ChatGPT use on students’ critical thinking]. KONSTELASI: Konvergensi Teknologi dan Sistem Informasi, 7(2), 45–53. doi:10.24002/konstelasi.v3i2.7241
Hendy, A., Abdurrahman, A., McCarthy, A., Tamchyna, A., & Bojar, O. (2023). How good are GPT models at machine translation? A comprehensive evaluation. doi:10.48550/arXiv.2302.09210
House, J. (2015). Translation quality assessment: Past and present (2nd ed.). Routledge.
Iqbal, M., Khan, N. U., & Imran, M. (2024). The role of artificial intelligence (AI) in transforming educational practices: Opportunities, challenges, and implications. Qlantic Journal of Social Sciences and Humanities, 5(2), 348–359. doi:10.55737/qjss.349319430
Kazu, I. Y., & Kuvvetli, M. (2023). The influence of AI-based pronunciation training on vocabulary acquisition in English learning. International Journal of Psychology and Educational Studies, 10(2), 480–493. doi:10.52380/ijpes.2023.10.2.1044
Kumar, O. P. (2023). Investigating the impact of artificial intelligence and technology in English language learning. Advances in Social Behavior Research, 3(1), 27–36. doi:10.54254/2753-7102/3/2023026
Lin, C. C., Huang, A. Y. Q., & Lu, O. H. T. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: A systematic review. Smart Learning Environments, 10, Article 41. doi:10.1186/s40561-023-00260-y
Misnawati, M. (2023). ChatGPT: Keuntungan, risiko, dan penggunaan bijak dalam era kecerdasan buatan [ChatGPT: Benefits, risks, and wise usage in the AI era]. Jurnal Teknologi Pembelajaran Indonesia, 13(2), 1–10. doi:10.55606/mateandrau.v2i1.221
Nababan, M. R., Nuraeni, A., & Sumardiono, B. (2012). Pengembangan model penilaian kualitas terjemahan [Developing a model for translation quality assessment]. Kajian Linguistik dan Sastra, 24(1), 39–57. doi:10.23917/kls.v24i1.101
O’Brien, S., & Simard, M. (2014). Introduction to special issue on post-editing. Machine Translation, 28(3–4), 159–164. https://doi.org/10.1007/s10590-014-9166-8
Omaggio Hadley, A. (2001). Teaching language in context (3rd ed.). Boston, MA: Heinle & Heinle.
Richards, J. C., & Rodgers, T. S. (2001). Approaches and methods in language teaching (2nd ed.). Cambridge, UK: Cambridge University Press. doi:10.1017/CBO9780511667305
Rizvi, M. (2023). Investigating AI-powered tutoring systems that adapt to individual student needs, providing personalized guidance and assessments. Eurasia Proceedings of Educational & Social Sciences, 31, 67–73. doi:10.55549/epess.1381518
Rusandi, E., & Rusli, T. (2021). Designing Basic/Descriptive Qualitative Research and Case Study. Al-Ubudiyah: Jurnal Pendidikan dan Studi Islam, doi:10.55623/au.v2i1.18
Saldanha, G., & O’Brien, S. (2013). Research methodologies in translation studies. Manchester, UK: St. Jerome. doi:10.1080/1750399X.2015.1016282
Son, J., & Kim, B. (2023). Translation performance from the user’s perspective of large language models and neural machine translation systems. Information, 14(10), 574. https://doi.org/10.3390/info14100574
Suharmawan, W. (2023). Pemanfaatan ChatGPT dalam dunia pendidikan [Utilization of ChatGPT in the world of education]. Education Journal: Journal Educational Research and Development, 7(2), 158–166. doi:10.31537/ej.v7i2.1248
Sun, R. (2024). Evaluating the translation accuracy of ChatGPT and DeepL through the lens of implied subjects. Arab World English Journal for Translation & Literary Studies, 8(4), 41–53. https://doi.org/10.24093/awejtls/vol8no4.5
Toury, G. (1995). Descriptive translation studies and beyond. John Benjamins.
Troqe, R., & Marchan, F. (2017). News Translation: Text analysis, fieldwork, survey. In Empirical modelling of translation and interpreting (pp. 277–310). Language Science Press. https://doi.org/10.5281/zenodo.1090974
Vieira, L. N. (2019). Post-editing of machine translation. The Routledge handbook of translation and technology (pp. 319–335). Routledge.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jurnal Kata
This work is licensed under a Creative Commons Attribution 4.0 International License.
