A Hybrid Deep Learning Framework for Multi-Camera Object Tracking and Anomaly Detection in Surveillance
DOI:
https://doi.org/10.29284/ijasis.12.1.2026.83-103Keywords:
Multi-camera object tracking; anomaly detection; deep learning; convolutional neural network (CNN); recurrent neural network (RNN); attention mechanism; spatio-temporal fusion; autoencoder; surveillance systems; smart city security.Abstract
Modern surveillance systems face increasing challenges in real-time object tracking, behavioral analysis, and anomaly detection across distributed camera networks. Traditional single-camera solutions fail to address spatial discontinuity and occlusion issues in multi-camera environments. To overcome these challenges, this paper proposes a hybrid deep learning framework integrating Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with attention-based data fusion for multi-camera object tracking and anomaly detection in large-scale surveillance networks. The proposed framework employs a CNN-based feature extractor to identify and encode object characteristics across multiple camera streams, followed by a spatio-temporal attention module that aligns overlapping regions and correlates object trajectories between cameras. The RNN-LSTM network models temporal dependencies to track object motion patterns, while an autoencoder-based anomaly detector identifies abnormal behaviors such as unattended baggage, unauthorized entry, or erratic movement. A multi-view consistency algorithm ensures smooth cross-camera transition tracking. Experimental evaluation on PETS 2009, TownCentre, and Avenue datasets demonstrates the framework’s superiority, achieving a tracking accuracy (MOTA) of 94.7%, an IDF1 score of 92.5%, and an anomaly detection accuracy of 96.2%, outperforming existing CNN-LSTM and transformer-based baselines. The proposed hybrid framework enhances situational awareness, robustness, and scalability, making it suitable for deployment in smart cities, public safety monitoring, and industrial surveillance.
References
1. Veesam, S.B., Rao, B.T., Begum, Z., Patibandla, R.L., Dcosta, A.A., Bansal, S., Prakash, K., Faruque, M.R.I. and Al-Mugren, K.S., 2025. Multi-camera spatiotemporal deep learning framework for real-time abnormal behavior detection in dense urban environments. Scientific Reports, 15(1), p.26813.
2. Peri, N., Khorramshahi, P., Rambhatla, S.S., Shenoy, V., Rawat, S., Chen, J.C. and Chellappa, R., 2020. Towards real-time systems for vehicle re-identification, multi-camera tracking, and anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 622-623).
3. Fei, L. and Han, B., 2023. Multi-object multi-camera tracking based on deep learning for intelligent transportation: A review. Sensors, 23(8), p.3852.
4. Pereira, S.S. and Maia, J.E.B., 2024. MC-MIL: video surveillance anomaly detection with multi-instance learning and multiple overlapped cameras. Neural Computing and Applications, 36(18), pp.10527-10543.
5. Shaban, M., Elpeltagy, M., Khedr, A.Y. and Al-Marakeby, A., 2025. Multi-Camera Based Intelligent Video Surveillance System Using SSD_Mobilenet_V3. Journal of Al-Azhar University Engineering Sector, 20(76), pp.974-996.
6. Dharan, A.M. and Mukhopadhyay, D., 2025. A comprehensive survey on machine learning techniques to mobilize multi-camera network for smart surveillance. Innovations in Systems and Software Engineering, 21(1), pp.313-332.
7. Hedde, O.V. and Kumar, M.A., 2023, August. Multi-Camera Multi-Person Tracking in Surveillance System. In 2023 12th International Conference on Advanced Computing (ICoAC) (pp. 1-7). IEEE.
8. Dhivya, R., Ajay, V., Dheena, P., Madhavan, P.A. and Meiarul, S., 2025, March. Hybrid Motion Vector and Deep Learning Approach for Crime Event Detection and Face Identification in Surveillance Footage. In 2025 5th International Conference on Expert Clouds and Applications (ICOECA) (pp. 902-909). IEEE.
9. Zaman, M.I., Bajwa, U.I., Saleem, G. and Raza, R.H., 2024. A robust deep networks based multi-object multi-camera tracking system for city scale traffic. Multimedia Tools and Applications, 83(6), pp.17163-17181.
10. Pathirannahalage, I., Jayasooriya, V., Samarabandu, J. and Subasinghe, A., 2025. A comprehensive analysis of real-time video anomaly detection methods for human and vehicular movement. Multimedia Tools and Applications, 84(10), pp.7519-7564.
11. Khan, A.A. and Sabeenian, R.S., 2025. Missing Object Detection and Tracking from Video Surveillance Camera Footage Based on Deep Learning Through Convolutional Gated Recurrent Neural Networks (CGRNN). Journal of Electrical Engineering & Technology, pp.1-9.
12. Shathik, J.A., 2025. Smart Vision Systems For Public Safety: Real-Time Crowd Monitoring And Anomaly Detection In Urban Spaces Using Deep Learning And Edge Computing. International Journal of Applied Mathematics, 38(6s), pp.720-743.
13. K.Kalpana., Dr.B.Paulchamy.,2022, "A novel design of nano router with high-speed crossbar scheduler for digital systems in QCA paradigm", Circuit World, Vol. 48 No. 4, pp. 464-478.
14. Benito-Picazo, J., Dominguez, E., Palomo, E.J. and López-Rubio, E., 2020. Deep learning-based video surveillance system managed by low cost hardware and panoramic cameras. Integrated Computer-Aided Engineering, 27(4), pp.373-387.
15. Dhuri, K.G. and Patil, S., 2025. Design of an integrated novel method for real-time anomaly detection against women using multimodal spatio-temporal, graph-based, and federated learning architectures. International Journal of Information Technology, pp.1-14.
16. Baala, A., Mostafa, H. and Mohssine, B., 2025, May. A Comprehensive Systematic Review of Deep Learning Techniques for Anomaly Detection in Urban Video Surveillance. In 2025 5th International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) (pp. 1-7). IEEE.
17. Saurav, S., Saini, R. and Singh, S., 2022. A dual-stream fused neural network for fall detection in multi-camera and 360∘ videos. Neural computing and applications, 34(2), pp.1455-1482.
18. Uddin, S., Hussain, B., Ahmad, N., Hussain, A. and Fareed, S., 2025. A hybrid framework for temporal object behavior analysis using lstm and real-time detection. International Journal of Ethical AI Application, 1(4), pp.20-30.
19. Dhanasekar J, Sudha VK. (2024) Relative Feature Classification Method for Diabetic Retinopathy Detection from Scanned Eye Images. Indian Journal of Science and Technology. 17(31): 3244-3252. https://doi.org/10.17485/IJST/v17i31.1133
20. K. Sivakami, S. Arunkumar, M. P. Usharani, J. Dhanasekar, S. Kokilamani and N. Revathi, ""Data Clustering and Routing using Improved Water Wave Optimization in Wireless Sensor Network,"" 2024 5th International Conference on Communication, Computing & Industry 6.0 (C2I6), Bengaluru, India, 2024, pp. 1-6, doi: 10.1109/C2I663243.2024.10894977.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
