Evaluating Effect of Training Corpus Size on Performance of Statistical Translation Systems
DOI:
https://doi.org/10.29284/m2jpvk82Keywords:
Natural Language Processing, Translation Model, Statistical Machine Translation, MOSES.Abstract
While the performance of Statistical Machine Translation (SMT) systems can be influenced by numerous factors, the size of the training corpus remains a foundational determinant in shaping the learning behavior and eventual translation quality of such systems. This paper reaffirms the significance of training data volume in the context of Phrase-Based Statistical Machine Translation (PB-SMT) and investigates the impact of varying training corpus sizes on translation performance using MOSES, a well-established SMT framework. Through this empirical lens, this research work explicitly contributes to: (i) building a structured experimental framework to evaluate corpus size influence, (ii) evaluate quantitative performance trends across multiple metrics.
Outcomes presented not only validate existing beliefs about the importance of data scale but also serve as a diagnostic foundation for scaling SMT systems in under-resourced language contexts. Observations suggested a logarithmic growth in translation quality with increased training data size.
References
[1] Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1, pages 48–54. Association for Computational Linguistics.
[2] Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open-source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the ACL on interactive poster and demonstration sessions, pages 177–180. Association for Computational Linguistics.
[3] Ramesh, G., Doddapaneni, S., Bheemaraj, A., et al. Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages. (2022).
[4] Improved backing-off for m-gram language modeling. In Acoustics, Speech, and Signal Processing, 1995. ICASSP-95., 1995 International Conference on, volume 1, pages 181–184. IEEE.
[5] Kenneth Heafield. 2011. KenLM: Faster and Smaller Language Model Queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland. Association for Computational Linguistics.
[6] Franz Josef Och and Hermann Ney. 2003. "A Systematic Comparison of Various Statistical Alignment Models." Computational Linguistics, 29(1):19-52. Computational Linguistics.
[7] Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume 1, pages 160–167. Association for Computational Linguistics.
[8] David Vilar, Jia Xu, Luis Fernando d’Haro, and Hermann Ney. 2006. Error Analysis of Statistical Machine Translation Output. In Proceedings of LREC, pages 697–702.
[9] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
[10] G. Doddington, “Automatic evaluation of machine translation quality using n-gram co-occurrence statistics,” in Proceeding of the 2nd International Conference on Human Language Technology Resources, San Diego, CA, 2002, pp. 138–145.
[11] B. Satanjeev, and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” in Proceedings of the ACL 2005 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization, Ann Arbor, 2005, pp. 65–72.
[12] G. Ankush, S. Venkatapathy, and R. Sangal, “METEOR Hindi: Automatic MT evaluation metric for Hindi as a target,” in Proceedings of ICON-2010: 8th International Conference on Natural Language Processing. India: Macmillan Publishers, 2010.
[13] Matthew Snover and Bonnie Dorr, “A Study of Translation Edit Rate with Targeted Human Annotation”.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
