Automated Data Classification And Pre-Processing Of It Service Tickets For Future Machine Learning Predictions
DOI:
https://doi.org/10.29284/3qhf5r63Keywords:
IT Service Management (ITSM), Ticket Classification, Text Pre- processing, Data Cleansing, ManageEngine ServiceDesk Plus (MSP).Abstract
Nowadays, many IT help desks depend on ticket systems when dealing with issues, inquiries, or system failures. Working through these manually often drags the pace, consumes extra work hours, and puts schedules under pressure. Although operational platforms gather vast amounts of textual records, those rarely fit algorithmic needs without adjustment. A lot comes cluttered - filled with irregularities that block smooth analysis. Consider missing pieces of information, duplicated entries, inconsistent formats, embedded snippets of code, misplaced tags, multilingual phrases, or extended freeform descriptions tough to interpret by machines. One way to study raw IT support tickets is by tracking their path across an entire system, beginning at ManageEngine ServiceDesk Plus, where they are made. From there, they travel toward processing inside a private SQL Server environment. Step follows step using R-based utilities that tidy and reshape textual data ahead of further analysis. Early on, detection shifts focus to non-English texts, and these get set aside quickly. Following that stage, duplicates are removed, leaving behind only distinct words ready for a closer look. Once filtered, words without clear value disappear. Then comes reducing terms to basic versions, so “running” becomes “run”, evening out small mismatches. Strange marks or uneven layout get removed. Every line shift into a consistent format, and later on, the key terms become clearer once the focus shifts towards the most repeated words. Visual maps of frequent word pairings uncover patterns not obvious at first glance. Meanwhile, broader examinations help keep rare outliers from skewing interpretation. From a total of 33,059 service tickets, every data entry contains structured fields alongside text columns such as Subject and Description. The results part shows that rigorous data cleansing improves the data consistency, produces a high-quality dataset, and reduces noise, which is suitable for training datasets using a machine learning algorithm. For future work, the focus will be on integrating the cleaned dataset with Azure Machine Learning Studio for automated ticket dispatching. Also, aiming at the optimization of the proposed model through hyperparameter tuning, continuous retraining with live datasets, and real-time deployment to enhance the service efficiency, SLA compliance, and overall satisfaction of the users.
References
[1] A. Gandomi and M. Haider, “Beyond the hype: Big data concepts, methods, and analytics,” International Journal of Information Management, vol. 35, no. 2, pp. 137–144, 2015.
[2] D. Tikk and Z. T. Kardkova´cs, “Text mining in information systems research: A systematic literature review,” CAIS, vol. 41, p. Article 12, 2017.
[3] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “From data mining to knowledge discovery in databases,” AI magazine, vol. 17, no. 3, pp. 37–54, 1996.
[4] C. D. Manning, P. Raghavan, and H. Schu¨tze, Introduction to information retrieval. Cambridge University Press, 2008.
[5] R. Campos, G. Dias, A. M. Jorge, and A. Jatowt, “A framework for term weighting approaches in text mining applications,” Expert Systems with Applications, vol. 40, no. 11, pp. 4243–4251, 2013.
[6] G. Kaur, I. Saini, and S. S. Chahal, “Text preprocessing for classification using machine learning in information retrieval,” in 2019 2nd International Conference on Intelligent Computing, Instrumentation and Control Technologies (ICICICT), vol. 1, pp. 738–742, IEEE, 2019.
[7] M. Rahman and M. Tasnim, “The impact of data preprocessing on the performance of classification algorithms,” Procedia Computer Science, vol. 194, pp. 35–42, 2021.
[8] L. Zhang, S. Wang, and X. Liu, “Understanding bag-of-words model: a statistical framework,” in International Conference on Image and Graphics, pp. 638–647, Springer, 2010.
[9] M. F. Porter, “An algorithm for suffix stripping,” Program, vol. 14, no. 3, pp. 130– 137, 1980.
[10] F. Al-Hawari, A. Al-Badarneh, S. Khasawneh, and H. Barham, “A machine learning-based help desk system for IT service management,” Journal of King Saud University - Computer and Information Sciences, vol. 33, no. 6, pp. 701–712, 2021.
[11] S. Ghosh and S. Choudhury, “Impact of data preprocessing on the performance of text classification: An empirical study,” International Journal of Computer Applications, vol. 56, no. 12, pp. 1–6, 2012.
[12] J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques. Elsevier, 2011.
[13] C. D. Manning and H. Schutze, Foundations of Statistical Natural Language Processing. MIT Press, 1999.
[14] A. Parameswaran, K. Patel, and R. Mishra, “Custom stopword list generation for domain-specific text classification,” Procedia Computer Science, vol. 185, pp. 471–478, 2021.
[15] M. M. Rahman, M. R. Islam, and N. Akter, “Ticket classification using machine learning for IT service management,” International Journal of Advanced Computer Science and Applications, vol. 9, no. 4, 2018.
[16] A. Joshi, R. Sinha, and P. Sharma, “Automated IT ticket categorization using deep learning,” IEEE Access, vol. 8, pp. 56045–56053, 2020.
[17] N. Kumar and R. Singh, “Word cloud-based exploratory analysis and keyword filtering in NLP pipelines,” Procedia Computer Science, vol. 152, pp. 135–142, 2019.
[18] A. Garc´ıa and S. Garc´ıa, “Preprocessing in text mining: A survey,” Journal of Intelligent & Fuzzy Systems, vol. 31, no. 1, pp. 1–11, 2016.
[19] A. Kannan and V. Gurusamy, “A review on text preprocessing techniques for text mining applications,” International Journal of Computer Science & Engineering Technology, vol. 5, no. 1, pp. 1–5, 2014.
[20] S. Salloum, M. Al-Emran, and K. Shaalan, “Big data analytics on unstructured data: A review,” International Journal of Data Science and Analytics, vol. 4, no. 2, pp. 85–104, 2017.
[21] A. Kumar and J. Singh, “Automated classification of IT service tickets using machine learning,” in Procedia Computer Science, vol. 152, pp. 218–225, Elsevier, 2019.
[22] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” NAACL-HLT, 2019.
[23] K. Saurabh, M. M. Tripathi, and S. Mahapatra, “Efficient utilization of energy in IoT devices using machine learning algorithms,” International Journal of Experimental Research and Review, vol. 47, pp. 133–145, 2025.
[24] J. Ahmad and S. Kumar, “Emotion classification from COVID-19 pandemic tweets using Roberta,” Journal of Information Systems Engineering and Management, vol. 10, no. 32s, pp. 546, 554, 2025.
[25] J. Ahmad and S. Kumar, “Sentiment analysis of COVID-19 tweets using BERT,” Educational Administration: Theory and Practice, vol. 30, no. 5, pp. 5052–5059, 2024.
[26] K. Saurabh, M. M. Tripathi, and S. Mahapatra, “A robust machine learning approach using smote and hyperparameter optimization for diabetes prediction using wearable IoT device data,” Journal of Dynamics and Control, vol. 9, no. 2, pp. 44– 56, 2025.
[27] M. M. Tripathi and P. K. Patahk, “A systematic review: Forecasting post-pandemic health trends with machine learning methods,” International Journal of Intelligent Systems and Application in Engineering, vol. 166, 2024.
[28] A. Fawaz et al., “An automated help desk system for IT service management using machine learning,” Journal of King Saud University - Computer and Information Sciences, vol. 33, no. 7, pp. 812–823, 2021.
[29] GreyCampus, “Incident management.” https://www.greycampus.com/opencampus/itil-foundation/incident-management.
[30] Sqlservertutorial, “SQL Server data types,” 2020. https://www.sqlservertutorial.
net/sql-server-basics/sql-server-data-types
[31] W. Li and R. Zhang, “Language detection and filtering for multilingual IT support data,” in 2020 IEEE International Conference on Big Data (Big Data), pp. 1234– 1242, IEEE, 2020.
[32] S. H. Kim and J. Lee, “Text normalization and preprocessing for machine learning applications in IT service datasets,” Expert Systems with Applications, vol. 166, p. 114050, 2021.
[33] B. Almeida and C. Silva, “Keyword extraction techniques for IT service tickets: A comparative study,” Information Processing & Management, vol. 53, no. 6, pp. 1369–1381, 2017.
[34] A. Singh and P. Sharma, “Data preprocessing techniques for improving text classification in IT service management,” Journal of Information Systems and Technology Management, vol. 19, no. 1, pp. 1–15, 2022.
[35] R. Studio and A. AWS, “Text mining using R,” March 2020. https://rstudio-pubs- static.s3.amazonaws.com.
[36] E. Guides, “Text mining and word cloud fundamentals in R: 5 simple steps you should know,” November 2017. https://www.r-bloggers.com/2017/02/text-mining- and-word-cloud-fundamentals.
[37] STHDA, “Text mining and word cloud fundamentals in R: 5 simple steps you should know,” 2020. http://www.sthda.com/english/wiki/text-mining-and-word- cloud-fundamentals.
[38] S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer, “Wrangler: Interactive visual specification of data transformation scripts,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3363–3372, 2011.
[39] L. Zhang, X. Wang, and R. Liu, “IT service ticket classification and routing using deep learning,” IEEE Transactions on Services Computing, vol. 13, no. 3, pp. 509– 521, 2020.
[40] M. Alam and M. Hossain, “Automated IT ticket classification using machine learning techniques,” Journal of Network and Computer Applications, vol. 181, pp. 103000, 2021.
[41] Y. Liu and M. Chen, “Text preprocessing and feature engineering for ticket classification in IT service systems,” in 2019 IEEE International Conference on Big Data, pp. 4218–4225, IEEE, 2019.
[42] J. Tang and P. Li, “Natural language processing techniques for IT service management: A survey,” ACM Computing Surveys, vol. 55, no. 4, pp. 1–36, 2022.
[43] W. Huang and L. Zhao, “Impact of preprocessing and feature selection on machine learning models for ticket categorization,” Expert Systems with Applications, vol. 166, p. 114084, 2021.
[44] H. Sun and Q. Li, “Enhanced ticket routing in IT service management using NLP and classification algorithms,” Information Sciences, vol. 512, pp. 1292–1308, 2020.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
