Real-Time High-Speed Autonomous Driving Via Lidar–Event Camera Fusion Using Attention-Enriched Spatio-Temporal Graph Neural Networks

Dr. Amol Raoji Madane; Dr. Ho Chin Kuan

doi:10.29284/3cs25d19

Authors

Dr. Amol Raoji Madane Post-Doc fellow, Asia Pacific University of Technology, Jalan Technologi 5, Taman Technologi, Malaysia, 57000
Dr. Ho Chin Kuan Vice Chancellor and Post-Doc Research Guide, Asia Pacific University of Technology, Jalan Technologi 5, Taman Technologi Malaysia

DOI:

https://doi.org/10.29284/3cs25d19

Keywords:

Autonomous Vehicles, Bacterial Colony-Driven Attention-enriched Spatio-Temporal Graph Neural Network (BC-AST-GNN), Event-Based Camera, High-Speed Maneuvers, Object Detection and Tracking.

Abstract

In autonomous vehicle driving, the use of diverse sensors to recognize surrounding vehicles at medium and long distances is essential for enhancing perception and ensuring safe high-speed maneuvers. Frame-based cameras suffer from motion blur and latency during rapid movements, while LiDAR, though highly accurate in 3D spatial mapping, has limited temporal resolution and struggles to detect small or low-reflectivity objects. To address these constraints, this research proposes a fusion approach that integrates LiDAR with event-based camera data to improve detection and execution of high-speed autonomous maneuvers. Event-based cameras asynchronously capture pixel-level brightness changes with microsecond latency, generating motion-rich, blur-free information that is ideally suited for dynamic environments. The data is preprocessed using noise reduction and normalization techniques to isolate relevant objects and filter out background noise. The proposed framework projects LiDAR point clouds into image space, aligns them with event data, and converts the fused information into bird’s-eye view (BEV) coordinates for robust object detection and tracking using a Bacterial Colony-Driven Attention-Enriched Spatio-Temporal Graph Neural Network (BC-AST-GNN) algorithm, which is applied to model spatio-temporal relationships between objects, combining event-driven motion cues with LiDAR’s 3D localization. The model further incorporates a state-estimation mechanism to refine object trajectories between LiDAR scans, ensuring stable tracking and predictive accuracy in high-speed environments, with the resulting perception outputs directly supporting a decision-making module that anticipates object interactions and selects safe and efficient maneuvers such as obstacle avoidance, lane changes, and emergency braking. Experimental results demonstrate that the BC-AST-GNN-based fusion approach significantly enhances detection accuracy, achieving 97.12% for Cars, 91.74% for Vans, 90.05% for Trucks, 92.63% for Pedestrians, and 94.02% for Cyclists, outperforming the baseline method in all categories. These advancements contribute directly to safer, more agile, and more reliable autonomous vehicle operation.

References

[1]. Guo, A., Huang, J., Lv, C., Chen, L., and Wang, F.Y., 2024. Advances in autonomous vehicle testing: The state of the art and future outlook on driving datasets, simulators and proving grounds. Authorea Preprints. 10.22541/au.172446843.37817639/v1

[2]. Cornwall, C., 2022. Event-Based Obstacle Detection with Commercial Lidar (Master's thesis, Utah State University).

[3]. Xiao, C., Cao, Q., Zhong, Y., Lan, L., Zhang, X., Luo, Z. and Tao, D., 2024. Motiontrack: Learning motion predictor for multiple object tracking. Neural Networks, 179, p.106539. https://doi.org/10.1016/j.neunet.2024.106539

[4]. Luo, G., Shao, C., Cheng, N., Zhou, H., Zhang, H., Yuan, Q. and Li, J., 2023. EdgeCooper: Network-aware cooperative LiDAR perception for enhanced vehicular awareness. IEEE Journal on Selected Areas in Communications, 42(1), pp.207-222. 10.1109/JSAC.2023.3322764

[5]. Tejero-Ruiz, D., Solís-Martín, D., Pérez-Grau, F.J. and Galan-Paez, J., 2024. Unmanned Aerial Vehicle Navigation with Event Cameras. Available at SSRN 4861406. http://dx.doi.org/10.2139/ssrn.4861406.

[6]. Su, H., Gao, L., Liu, T. and Kneip, L., 2024. Motion-Aware Optical Camera Communication with Event Cameras. IEEE Robotics and Automation Letters. 10.1109/LRA.2024.3517292

[7]. Xing, J., Wei, D., Zhou, S., Wang, T., Huang, Y. and Chen, H., 2024. A comprehensive study on self-learning methods and implications to autonomous driving. IEEE Transactions on Neural Networks and Learning Systems. 10.1109/TNNLS.2024.3440498

[8]. Siyu, L., 2024. LiDAR-Driven Drivable Space Detection with Deep Neural Network.

[9]. Shalumov, A., Halaly, R. and Tsur, E.E., 2021. Lidar-driven spiking neural network for collision avoidance in autonomous driving. Bioinspiration & Biomimetics, 16(6), p.066016. 10.1088/1748-3190/ac290c

[10]. El Shair, Z. and Rawashdeh, S., 2023. High-temporal-resolution event-based vehicle detection and tracking. Optical Engineering, 62(3), pp.031209-031209. https://doi.org/10.1117/1.OE.62.3.031209

[11]. Feng, J., Zhao, P., Zheng, H., Konpang, J., Sirikham, A. and Kalnaowakul, P., 2025. Enhancing Autonomous Driving Perception: A Practical Approach to Event-Based Object Detection in CARLA and ROS. Vehicles, 7(2), p.53. https://doi.org/10.3390/vehicles7020053

[12]. Kambale, A.W., 2021. Event-based camera for object tracking.

[13]. Sakhai, M., Sithu, K., Oke, M.K.S. and Wielgosz, M., 2025. Cyberattack Resilience of Autonomous Vehicle Sensor Systems: Evaluating RGB vs. Dynamic Vision Sensors in CARLA. Applied Sciences, 15(13), p.7493. https://doi.org/10.3390/app15137493

[14]. Queen, K.J., 2023. Event-Based Perception for Ground Vehicle Control (Doctoral dissertation, University of Pennsylvania).

[15]. Jiao, J., Chen, F., Wei, H., Wu, J. and Liu, M., 2023. LCE-Calib: Automatic LiDAR-frame/event camera extrinsic calibration with a globally optimal solution. IEEE/ASME Transactions on Mechatronics, 28(5), pp.2988-2999. 10.1109/TMECH.2023.3259444

[16]. Iqbal, H., Campo, D., Marin-Plaza, P., Marcenaro, L., Gómez, D.M. and Regazzoni, C., 2021. Modeling perception in autonomous vehicles via 3D convolutional representations on LiDAR. IEEE Transactions on Intelligent Transportation Systems, 23(9), pp.14608-14619. 10.1109/TITS.2021.3130974

[17]. Wu, Q., Li, X., Wang, K. and Bilal, H., 2023. Regional feature fusion for on-road detection of objects using camera and 3D-LiDAR in high-speed autonomous vehicles. Soft Computing, 27(23), pp.18195-18213. https://doi.org/10.1007/s00500-023-09278-3

[18]. Astha Modak, Samruddhi Paradkar, Shruti Manwatkar, Amol R. Madane, and Ashwini M. Deshpande. "Human head pose and eye state-based driver distraction monitoring system." In Proceedings of 3rd International Conference on Computer Vision and Image Processing: CVIP 2018, Volume 1, pp. 403-421. Singapore: Springer Singapore, 2019.

[19]. Dinesh Kadam, Amol R. Madane, Krishnan Kutty, S. V. Bonde, “Rain Streaks Elimination using Image Processing Algorithms”, Signal & Image Processing: An International Journal (SIPIJ), Vol. 10, No. 3, pp. 21-32, Jun-2019.

[20]. Apeksha Kamble, Akanksha Singh, Damini Thusoo, Amol R. Madane, "Design and Implementation of Surround View Monitoring System in View of Autonomous Vehicle", 6th International Conference on Data Management, Analytics and Innovation 2022 (ICDMAI 2022), India, pp. 509-519, 14-16 Jan 2022.

[21]. Ashish Kathe, Pranav Teli, Amol R. Madane, “COVID-19 Regulations Check: Social Distancing, People Counting and Mask Wear Check”, 6th International Conference on Data Management, Analytics and Innovation 2022 (ICDMAI 2022), India, pp. 467-484, 14-16 Jan 2022.

[22]. Priyojit Chakraborty, Amol R. Madane, “Vehicle Queue Management System at Fuel Station”, NIELIT's International Conference on Communication, Electronics and Digital Technologies (NICE-DT'23), New Delhi, India, 10-11 Feb 2023.

[23]. Amol Raoji Madane, Dr. Ho Chin Kuan, "Convxpf-Net: An Explainable AI Approach for Recognizing Pedestrian Behaviour Using Convolutional Xgboost Pyramid Fractional Network", International Journal of Advances in Signal and image Sciences, Nov 2025.

[24]. Amol Raoji Madane, Dr. Ho Chin Kuan, "Adaptive Multi-Spectral Vision Fusion for Reliable Hazard Identification in Challenging Visibility Conditions", International Journal of Applied Mathematics, Vol 38, No. 12s, pp. 4241-4257, Dec 2025. (ISSN: 1311-1728 (printed version), ISSN: 1314-8060 (on-line version)).

[25]. Amol R. Madane, Dr. Uttam D. Kolekar, “High Frequency Component based Addition Algorithm of Digital Image Watermarking”, International Journal of Computer Applications, Vol. 50, No. 10, pp. 30-34, Jul 2012 (ISBN: 973-93-80869-59-3) (Impact Factor- 0.821).