A Hybrid LSTM Autoencoder and Isolation Forest Framework for Anomaly-Based Ransomware Detection in Network Traffic
DOI:
https://doi.org/10.29284/zv27wr86Keywords:
Ransomware Detection, Anomaly Detection, LSTM Autoencoder, Isolation Forest, Network Traffic, Hybrid Learning .Abstract
In this study, a novel anomaly detection-based hybrid model is realized for the prediction of ransomware attacks by monitoring network traffic. The signature or rule-based traditional approach to combating polymorphic ransomware and zero-day attack is under severe strain, especially in the presence of encrypted network traffic, that requires using more adaptive and flexible approaches that analyse behaviours. The proposed framework exploits temporal patterns of traffic by LSTM autoencoder and identifies statistically anomalous values with Isolation Forest. The outputs from the two models are then combined by a weighted merge mechanism to a final anomaly score to determine if traffic is benign or malicious. The proposed model was tested on an enhanced version of UGRansome_Dataset_2024 dataset having 111,904 records in unbalanced form, and balanced by SMOTE-ENN technique up to 149,197 records containing benign and malicious traffic. The simulation results demonstrate the superiority of the proposed model with 99.66% accuracy, 99.78% precision, 99.85% recall, 99.81% F1-score. These results demonstrate that the combined approach can effectively reduce false alarms, while the ability to detect attacks is significantly improved, and therefore can be used in practice networked environments.
References
[1] B. Riskhan, A. M. Raufi, and M. H. Usmani, “Physical Security to Cybersecurity (Challenges and Implications in the Modern Digital Landscape),” 2024.
[2] A. Greubel, D. Andres, and M. Hennecke, “Analyzing Reporting on Ransomware Incidents: A Case Study,” Soc. Sci., vol. 12, no. 5, May 2023, doi: 10.3390/socsci12050265.
[3] J. Ferdous, R. Islam, A. Mahboubi, and M. Z. Islam, “A Review of State-of-the-Art Malware Attack Trends and Defense Mechanisms,” IEEE Access, vol. 11, pp. 121118–121141, 2023, doi: 10.1109/ACCESS.2023.3328351.
[4] C. Beaman, A. Barkworth, T. D. Akande, S. Hakak, and M. K. Khan, “Ransomware: Recent advances, analysis, challenges and future research directions,” Comput. Secur., vol. 111, Dec. 2021, doi: 10.1016/j.cose.2021.102490.
[5] S. H. Kok, A. Abdullah, and N. Z. Jhanjhi, “Early detection of crypto-ransomware using pre-encryption detection algorithm,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 5, pp. 1984–1999, May 2022, doi: 10.1016/j.jksuci.2020.06.012.
[6] F. Aldauiji, O. Batarfi, and M. Bayousef, “Utilizing Cyber Threat Hunting Techniques to Find Ransomware Attacks: A Survey of the State of the Art,” IEEE Access, vol. 10, pp. 61695–61706, 2022, doi: 10.1109/ACCESS.2022.3181278.
[7] T. M. Hassin Mohamed, B. A. Saleh Al-Rimy, and S. A. Almalki, “A Ransomware Early Detection Model based on an Enhanced Joint Mutual Information Feature Selection Method,” Engineering, Technology and Applied Science Research, vol. 14, no. 4, pp. 15400–15407, Aug. 2024, doi: 10.48084/etasr.7092.
[8] A. Alexei and A. Alexei, “THE DIFFERENCE BETWEEN CYBER SECURITY VS INFORMATION SECURITY,” Journal of Engineering Science, vol. 29, no. 4, pp. 72–83, Jan. 2023, doi: 10.52326/jes.utm.2022.29(4).08.
[9] G. Nagar, “The Evolution of Ransomware: Tactics, Techniques, and Mitigation Strategies,” International Journal of Scientific Research and Management (IJSRM), vol. 12, no. 06, pp. 1282–1298, Jun. 2024, doi: 10.18535/ijsrm/v12i06.ec09.
[10] M. Güven, “Dynamic Malware Analysis Using a Sandbox Environment, Network Traffic Logs, and Artificial Intelligence,” International Journal of Computational and Experimental Science and Engineering, vol. 10, no. 3, pp. 480–490, Jul. 2024, doi: 10.22399/ijcesen.460.
[11] K. K. Verma, B. M. Singh, and A. Dixit, “A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system,” International Journal of Information Technology (Singapore), vol. 14, no. 1, pp. 397–410, Feb. 2022, doi: 10.1007/s41870-019-00364-0.
[12] N. Christakis and D. Drikakis, “Reducing Uncertainty and Increasing Confidence in Unsupervised Learning,” Mathematics, vol. 11, no. 14, Jul. 2023, doi: 10.3390/math11143063.
[13] A. Redhu, P. Choudhary, K. Srinivasan, and T. K. Das, “Deep learning-powered malware detection in cyberspace: a contemporary review,” 2024, Frontiers Media SA. doi: 10.3389/fphy.2024.1349463.
[14] A. Lumazine et al., “Ransomware Detection in Network Traffic Using a Hybrid CNN and Isolation Forest Approach,” Oct. 15, 2024. doi: 10.22541/au.172901014.44599790/v1.
[15] E. Berrueta, D. Morato, E. Magaña, and M. Izal, “Crypto-ransomware detection using machine learning models in file-sharing network scenarios with encrypted traffic,” Expert Syst. Appl., vol. 209, Dec. 2022, doi: 10.1016/j.eswa.2022.118299.
[16] M. Venturini, F. Freda, E. Miotto, M. Conti, and A. Giaretta, “Differential Area Analysis for Ransomware: Attacks, Countermeasures, and Limitations,” IEEE Trans. Dependable Secure Comput., vol. 22, no. 4, pp. 3449–3464, 2025, doi: 10.1109/TDSC.2025.3532324.
[17] S. Chadler, S. Davis, T. Delacroix, and W. Carrington, “Ransomware Detection Using Multi-Vector Anomaly Profiling for Maximum Security,” Nov. 15, 2024. doi: 10.21203/rs.3.rs-5452210/v1.
[18] A. Hussain, A. Saadia, M. Alhussein, A. Gul, and K. Aurangzeb, “Enhancing ransomware defense: deep learning-based detection and family-wise classification of evolving threats,” PeerJ Comput. Sci., vol. 10, pp. 1–44, 2024, doi: 10.7717/peerj-cs.2546.
[19] M. Davidian, M. Kiperberg, and N. Vanetik, “Early Ransomware Detection with Deep Learning Models,” Future Internet, vol. 16, no. 8, Aug. 2024, doi: 10.3390/fi16080291.
[20] E. Li, H. He, C. Huang, J. Zhang, H. Cheng, and Y. Ge, “Anomaly-Driven Crypto-Locking Behavior Analysis for Ransomware Detection through Semantic Flow Mapping,” Dec. 12, 2024. doi: 10.36227/techrxiv.173398043.39428308/v1.
[21] S. Muhammad, “AI-Driven Ransomware Defense Framework for Digital Banking International Journal of Innovative Research in Science Engineering and Technology (IJIRSET) AI-Driven Ransomware Defense Framework for Digital Banking,” Article in International Journal of Innovative Research in Science Engineering and Technology, 2025, doi: 10.15680/IJIRSET.2025.1406012.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
