A Cascade Hybrid Machine Translation Architecture Using DBMT, EBMT And NMT For Indian Languages: Hindi-Punjabi
DOI:
https://doi.org/10.29284/vkdhbh57Keywords:
Machine Translation, Hybrid Translation, Hindi-Punjabi, Neural Machine Translation, Example-Based MT, Low-Resource Languages, NLP.Abstract
In natural Language Processing, Machine Translation is still challenging for low resource regional languages like Hindi and Punjabi because of morphology, script variations and Semantic Ambiguity. Previous research on Hindi- Punjabi translation has mainly focused on distinct methodologies like dictionary-based machine translation, rule-based translation, statistical machine translation and pure neural machine translation. This research proposes a cascade Hybrid MT framework for Hindi-Punjabi Translation integrating Dictionary-based direct mapping for known words, Example based machine translation (EBMT) for similar sentence patterns and Neural Machine Translation for complex linguistic sentences. The proposed system optimizes computational efficiency through knowledge-driven pipelines that prioritize low-cost translation methods before invoking expensive neural networks, therefore proposed hybrid machine translation framework is significantly more computationally efficient than pure Neural Machine Translation (NMT) models. Experimental results demonstrate that the hybrid approach achieves superior performance compared to individual methods, with BLEU scores of 42.5 for dictionary matches, 38.2 for EBMT matches, and 35.7 for NMT translations.
References
1. Hutchins, W. J., & Somers, H. L. (1992). An introduction to machine translation. Academic Press. http://www.hutchinsweb.me.uk/IntroMT-TOC.htm
2. Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2002). Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (pp. 311–318). Association for Computational Linguistics. https://doi.org/10.3115/1073083.1073135
3. Josan, G. S., & Lehal, G. S. (2008). A Punjabi to Hindi machine translation system. In Coling 2008: Companion volume: Demonstrations (pp. 157–160). Coling 2008 Organizing Committee. https://aclanthology.org/C08-3004
4. Goyal, V., & Lehal, G. S. (2009). Evaluation of Hindi to Punjabi machine translation system. International Journal of Computer Science Issues, 4(1), 36–39. https://arxiv.org/abs/0910.1868
5. Goyal, V., & Lehal, G. S. (2010). Web based Hindi to Punjabi machine translation system. Journal of Emerging Technologies in Web Intelligence, 2(2), 148–151. https://doi.org/10.4304/jetwi.2.2.148-151
6. Goyal, V., Lehal, G.S. (2011). Hindi to Punjabi Machine Translation System. In: Singh, C., Singh Lehal, G., Sengupta, J., Sharma, D.V., Goyal, V. (eds) Information Systems for Indian Languages. ICISIL 2011. Communications in Computer and Information Science, vol 139. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-19403-0_40
7. Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, & K. Q. Weinberger (Eds.), Advances in neural information processing systems 27 (pp. 3104–3112). Curran Associates. https://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks
8. Popović, M. (2015). chrF: Character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation (pp. 392–395). Association for Computational Linguistics. https://doi.org/10.18653/v1/W15-3049
9. Kaur, A., & Rani, J. (2015). A web-based Punjabi to Hindi statistical machine translation system. In 2015 2nd International Conference on Recent Advances in Engineering & Computational Sciences (RAECS) (pp. 1–6). IEEE.
10. https://doi.org/10.1109/RAECS.2015.7453298
11. Kumar, A., & Goyal, V. (2015). Statistical post editing system (SPES) applied to Hindi-Punjabi PB-SMT system. Indian Journal of Science and Technology, 8(27), 12070. https://doi.org/10.17485/ijst/2015/v8i27/12070
12. Sreelekha, S. (2017). Statistical vs rule based machine translation: A case study on Indian language perspective. arXiv. https://doi.org/10.48550/arXiv.1708.04559
13. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems 30 (pp. 5998–6008). Curran Associates. https://papers.nips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html
14. Khan, N. J., Anwar, W., & Durrani, N. (2017). Machine translation approaches and survey for Indian languages. arXiv. https://doi.org/10.48550/arXiv.1701.04290
15. Kunchukuttan, A., Mehta, P., & Bhattacharyya, P. (2018). The IIT Bombay English-Hindi parallel corpus. In N. Calzolari, K. Choukri, C. Cieri, M. Declerck, S. Goggi, K. Hasida, N. Ide, & A. Mariani (Eds.), Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018) (pp. 2893–2900). European Language Resources Association (ELRA). https://aclanthology.org/L18-1548
16. Choudhary, H., Rao, S., & Rohilla, R. (2020). Neural machine translation for low-resourced Indian languages. arXiv. https://doi.org/10.48550/arXiv.2004.13819
17. Kakwani, D., Kunchukuttan, A., Golla, S., N.C., G., Bhattacharyya, A., Khapra, M. M., & Kumar, P. (2020). IndicNLPSuite: Monolingual corpora, evaluation benchmarks and pre-trained multilingual language models for Indian languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 (pp. 4948–4961). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.findings-emnlp.445
18. NLLB Team, Costa-jussà, M. R., Cross, J., Çelebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., Licht, D., Maillard, J., Sun, A., Wang, S., Wenzek, G., Youngblood, A., Akula, B., Barrault, L., Gonzalez, G. M., Hansanti, P., ... Wang, J. (2024). Scaling neural machine translation to 200 languages. Nature, 630, 841–851. https://doi.org/10.1038/s41586-024-07335-x
19. Gala, J., Chitale, P. A., AK, R., Gumma, V., Doddapaneni, S., Kumar, A., Nawale, J., Sujatha, A., Puduppully, R., Raghavan, V., Kumar, P., Khapra, M. M., Dabre, R., & Kunchukuttan, A. (2023). IndicTrans2: Towards high-quality and accessible machine translation models for all 22 scheduled Indian languages. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 115–131). Association for Computational Linguistics.
20. https://aclanthology.org/2023.emnlp-main.895
21. Roy, M. K., Arora, K. K., & Arora, S. (2023). Bidirectional Hindi–Punjabi machine translation. In P. Majumder, M. Mitra, S. Gangopadhyay, & P. Mehta (Eds.), Working notes of FIRE 2023: Forum for Information Retrieval Evaluation (pp. 796–801). CEUR Workshop Proceedings. https://ceur-ws.org/Vol-3681/T9-4.pdf
22. Dabre, R., & Kunchukuttan, A. (2024). Findings of WMT 2024’s MultiIndicMT shared task for machine translation of 22 Indian languages. In Proceedings of the Ninth Conference on Machine Translation (WMT 2024) (pp. 669–676). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.wmt-1.55
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
