Enhancement In Telugu Speech-To-Text Transformation With Generative AI And Signal Processing Techniques
DOI:
https://doi.org/10.29284/64qy1616Keywords:
Telugu Speech Recognition, Automatic Speech Recognition (ASR), Generative AI, Signal Processing, Large Language Models (LLM), Speech-to-Text Transformation.Abstract
The Telugu Speech-to-Text (STT) systems are to be important in digital governance, education, and accessibility; still, they are not very productive because of noise, dialect variations, code-mixed language, and small labelled data sets. Conventional ways of Automatic Speech Recognition (ASR) have been known to have transcription errors, thus it has low linguistic accuracy. To overcome this issue, this paper will suggest a hybrid Telugu STT pipeline that would integrate both signal processing algorithm and refining based on Generative AI. The proposed methodology will incorporate the audio preprocessing (silence and normalization, spectral noise reduction), the extraction of MFCC features, Transformer-based acoustic modeling with CTC decoding, and the use of a Large Language Model (LLM) to correct linguistic information. It is evidenced by the results of the experiment that speech quality and accuracy in transcription significantly improved. The preprocessing phase decreased the audio duration to 3.5 seconds, and reduced audio samples to 54,000 getting around 3233 percent decreasing redundant information. The character error rate (CER) and Word Error rate (WER) of the baseline ASR system were 2.96 and 2.98 respectively. Following the implementation of post-processing based on the LLM, WER had fallen to 1.61 and CER to 1.75, which corresponded to Relative Error Reductions (RER) of 46% and 41 respectively. This evidence indicates that signal amplification with Generative AI-based correction could perform much better in terms of transcription accuracy, linguistic consistency, and readability and the framework can be used as an effective tool to achieve robust Telugu speech recognition in practical scenarios.
References
1. Sravan, Medapati Venkata Manga Naga, and K. Venkata Rao. "A Hybrid Deep Learning Framework for Speech-to-Text Conversion as Part of Telemedicine System Integrated With 5G." International journal of electrical and computer engineering systems 16, no. 4 (2025): 333-346.
2. Huang, Kaiyu, Fengran Mo, Xinyu Zhang, Hongliang Li, You Li, Yuanchi Zhang, Weijian Yi et al. "A survey on large language models with multilingualism: Recent advances and new frontiers." arXiv preprint arXiv:2405.10936 (2024).
3. Shah, Meerabahen M., and Hiren R. Kavathiya. "Development Of A Model To Analyze & Interpret Vernacular Voice Recognition Of Gujarati Dialects." PhD diss., Department of Computer Science, Faculty of Science Atmiya University., 2024.
4. Mishra, Abhishek, Ritesh Sur Chowdhury, Vartul Bahuguna, Isha Pandey, and Ganesh Ramakrishnan. "Language translation, and change of accent for speech-to-speech task using diffusion model." arXiv preprint arXiv:2505.04639 (2025).
5. Kumar, Gokul Karthik, S. V. Praveen, Pratyush Kumar, Mitesh M. Khapra, and Karthik Nandakumar. "Towards building text-to-speech systems for the next billion users." In Icassp 2023-2023 ieee international conference on acoustics, speech and signal processing (icassp), pp. 1-5. IEEE, 2023.
6. Rajakumari, R. Geetha, D. Karthika Renuka, and L. Ashok Kumar. "ENHANCING ASR ACCURACY AND COHERENCE ACROSS INDIAN LANGUAGES WITH WAV2VEC2 AND GPT-2."
7. Priya, M. Yagnasri, Sanjivani P. Shendre, Samrudhi S. Kamble, Swetha Sridhar, and Suja Palaniswamy. "Emotion-Aware Text-to-Speech Synthesis for Enhanced Accessibility: Synthetic Data Generation and Automated TTS for the Visually Impaired." In 2025 International Conference on Emerging Technologies in Computing and Communication (ETCC), pp. 1-6. IEEE, 2025.
8. Shanthamallappa, Mahadevaswamy, Kiran Puttegowda, Naveen Kumar Hullahalli Nannappa, and Sudheesh Kannur Vasudeva Rao. "Robust automatic speech recognition using wavelet-based adaptive wavelet thresholding: A review." SN Computer Science 5, no. 2 (2024): 248.
9. Mahaganapathy, Ahrane, and Kengatharaiyer Sarveswaran. "Natural Language Processing Journal."
10. Singh, Amitoj, Virender Kadyan, Munish Kumar, and Nancy Bassan. "ASRoIL: a comprehensive survey for automatic speech recognition of Indian languages." Artificial Intelligence Review 53, no. 5 (2020): 3673-3704.
11. Rajendran, S., L. Ramamoorthy, and L. Sobha. "WHITE PAPER ON TECHNOLOGICAL DEVELOPMENT OF TAMIL."
12. Acharya, Aayush. "AUTOMATIC SPEECH RECOGNITION AND CLASSIFICATION OF NEPALI SPEECH." PhD diss., TRIBHUVAN UNIVERSITY, 2022.
13. Singh, Amitoj, Navkiran Kaur, Vinay Kukreja, Virender Kadyan, and Munish Kumar. "Computational intelligence in processing of speech acoustics: a survey." Complex & Intelligent Systems 8, no. 3 (2022): 2623-2661.
14. Wang, Desheng, Yangjie Wei, Ke Zhang, Dong Ji, and Yi Wang. "Automatic speech recognition performance improvement for Mandarin based on optimizing gain control strategy." Sensors 22, no. 8 (2022): 3027.
15. Tissera, Muditha, and Hassaan Saadhiq. "Natural Language Processing Resources for Tamil Language: A Systematic Review." Journal of information and communication convergence engineering 23, no. 4 (2025): 236-258.
16. Lokhande, Mukul, Tanushree Dewangan, Mohd Sharik Mansoori, Tejas Chaudhari, Damayanti Lokhande, Adam Teman, and Santosh Kumar Vishvakarma. "Bhasha-Rupantarika: Algorithm-Hardware Co-design approach for Multilingual Neural Machine Translation." arXiv preprint arXiv:2510.10676 (2025).
17. Nanduri, Dinesh Kumar. "Exploring the Role of Generative Artificial Intelligence in Culturally Relevant Storytelling for Native Language Learning Among Children." Master's thesis, University of Maryland, College Park, 2024.
18. Muthusamy, Dharmalingam, Shunmuga Priya Muthusamy Chinnan, and Geetha Muthuswamy Prakashpathy. "From Waves to Words: The Impact of Large Language Models on Automatic Speech Recognition Systems: An Overview." Geetha, From Waves to Words: The Impact of Large Language Models on Automatic Speech Recognition Systems: An Overview.
19. Soundarraj, Rajkumaran, Shenbagarajan Anantharajan, and Saranraj Loganathan. "PhonoMetric: a dual-metric engine for real-time English language accent evaluation and personalized speech training for Indian learners." Frontiers in Communication 10 (2025): 1704484.
20. Chondhekar, Sujal, Vasanth Murukuri, Rushabh Vasani, Sanika Goyal, Rajshree Badami, Anushree Rana, Sanjana SN et al. "When De-noising Hurts: A Systematic Study of Speech Enhancement Effects on Modern Medical ASR Systems." arXiv preprint arXiv:2512.17562 (2025).
21. Darvin, Ron. "The need for critical digital literacies in generative AI-mediated L2 writing." Journal of Second Language Writing 67 (2025): 101186.
22. Inuwa-Dutse, Isa. "Naijanlp: A survey of nigerian low-resource languages." arXiv preprint arXiv:2502.19784 (2025).
23. Omnilingual, A. S. R., Gil Keren, Artyom Kozhevnikov, Yen Meng, Christophe Ropers, Matthew Setzler, Skyler Wang et al. "Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages." arXiv preprint arXiv:2511.09690 (2025).
24. Pratap, Vineel, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky et al. "Scaling speech technology to 1,000+ languages." Journal of Machine Learning Research 25, no. 97 (2024): 1-52.
25. Jain, Dhruv, Harshit Shukla, Gautam Rajeev, Ashish Kulkarni, Chandra Khatri, and Shubham Agarwal. "VoiceAgentBench: Are Voice Assistants ready for agentic tasks?." arXiv preprint arXiv:2510.07978 (2025).
26. Madan, Chetan, Harshita Diddee, Deepika Kumar, and Mamta Mittal. "CodeFed: Federated speech recognition for low-resource code-switching detection." ACM Transactions on Asian and Low-Resource Language Information Processing 23, no. 1 (2024): 1-14.
27. Ramjee, Pragnya, Bhuvan Sachdeva, Satvik Golechha, Shreyas Kulkarni, Geeta Fulari, Kaushik Murali, and Mohit Jain. "CataractBot: an LLM-powered expert-in-the-loop chatbot for cataract patients." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 9, no. 2 (2025): 1-31.
28. Behravan, Majid, Elham Mohammadrezaei, Mohamed Azab, and Denis Gračanin. "Multilingual standalone trustworthy voice-based social network for disaster situations." In 2024 IEEE 15th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), pp. 264-270. IEEE, 2024.
29. Choi, Wan Chong, Chi In Chang, Iek Chong Choi, and Lai Chu Lam. "Country Landscape of Large Language Models Development: A Review." (2025).
30. https://www.kaggle.com/datasets/ziya07/indian-languages-audio-dataset
31. Arora, Siddhant, Hayato Futami, Jee-weon Jung, Yifan Peng, Roshan Sharma, Yosuke Kashiwagi, Emiru Tsunoo, Karen Livescu, and Shinji Watanabe. "Universlu: Universal spoken language understanding for diverse tasks with natural language instructions." In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 2754-2774. 2024.
32. Pashangpour, Souren, and Goldie Nejat. "The future of intelligent healthcare: A systematic analysis and discussion on the integration and impact of robots using large language models for healthcare." Robotics 13, no. 8 (2024): 112.
33. Tuli, Esmot Ara, Jae-Min Lee, and Dong-Seong Kim. "Integration of quantum technologies into metaverse: Applications, potentials, and challenges." IEEE Access 12 (2024): 29995-30019.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
