TITLE:
Integrated Evaluation of AI-Driven MT Systems’ Performance for Chinese-English Translation of China’s Political Documents
AUTHORS:
Daohua Hu
KEYWORDS:
AI-Driven MT Systems, Political Documents, Translation Quality Assessment, Automatic Evaluation, MQM Framework
JOURNAL NAME:
Open Journal of Social Sciences,
Vol.14 No.8,
August
20,
2026
ABSTRACT: With increasingly frequent international exchanges, the translation of China’s political documents has become a critical factor in conveying China’s voice and shaping its international image. The development of AI-driven MT systems has profoundly transformed the language service industry and its ecology. An integrated evaluation of these MT systems’ performance for Chinese-English translation of China’s political documents has been carried out. This study takes “Recommendations of the Central Committee of the Communist Party of China for Formulating the 15th Five-Year Plan for National Economic and Social Development” as the source text material, with the English version released by the State Council as the reference translation. First, the translation quality of English versions generated by DeepL, Baidu, Youdao, Doubao, ChatGPT, and Gemini 3 Flash was evaluated respectively by using three automatic evaluation metrics: BLEU, TER, and METEOR. Then, based on the MQM 2.0 core framework, the same English versions were evaluated by error analysis to assess the performance of each MT system. The results show that: 1) As for the automatic evaluation metrics, Doubao, DeepL, and Baidu ranked the top three for BLEU; the top three in ascending order were Doubao, Youdao, and DeepL for TER; Doubao, Gemini 3 Flash, and DeepL ranked the top three for METEOR. Based on the overall score of the above three metrics, the top three rankings are Doubao, DeepL, and Youdao. 2) As for MQM evaluation, all six MT systems had an error rate of 60% or above for the terminology dimension; only two MT systems had no errors in the language conventions dimension; two MT systems had no errors in the accuracy dimension; in the style dimension, one MT system had an error rate of only 4%, and others of at least above 85%. In terms of overall Chinese-English translation performance, Doubao, Youdao, and Gemini 3 Flash ranked the top three, with Doubao’s performance significantly surpassing other MT systems in the dimensions of accuracy, language conventions, and style. This study will shed light on the performance of MT systems for Chinese-English translation of China’s political documents, and further research in translation studies and post-translation editing in this digital era.