A Multistage Adaptive 3D-CNN Framework for Driver Drowsiness Detection Using Multi-Condition Facial Behavior Analysis

Nrusimhadri Naveen; Dr. Aarti

doi:10.29284/4nzzqz12

Authors

Nrusimhadri Naveen PhD scholar, Department of Computer Science and Engineering, Shri Venkateshwara university, Gajraula,UP.
Dr. Aarti Assistant Professor, Department of Computer Science and Engineering, Bharati Vidyapeeth college of Engineering,New Delhi.

DOI:

https://doi.org/10.29284/4nzzqz12

Keywords:

Drowsiness; Deep Learning; Convolutional Neural Network; Accuracy; Detection.

Abstract

The need for improved detection technology to address this worldwide problem is highlighted by an increase in driver fatigue-related incidents. Many machine learning and deep learning techniques, such as CNN variations, have been tested by researchers; nevertheless, real-time deployment is challenging due to computational complexity, limited evaluation accuracy, and questionable reliability. Driver drowsiness detection using a multistage adaptive 3D-CNN model that takes system performance and complexity into account. Driver fatigue symptoms are categorised using our models. Several 3D CNN architectures are used to gather face attributes for sub-models. The head condition model detects sideways glancing, eyebrow elevation, and head snoozing. Eye closures, pain, and partially closed eyes are all detected by the eye alignment prototype. Open and covered yawning are recognised using the mouth condition model. States are categorised as day, sunglasses, night, and normal by the specs and typical conditions model. Each model, which uses specialised algorithms to identify different driver sleepiness variables, is crucial to the system. thoroughly investigated utilising the KEC-DDD and NTHU-DDD datasets, which demonstrate their resilience and flexibility. The sub-models' ability to effectively capture driver behaviours in a variety of settings is confirmed by dataset validation results.

References

[1] Hu, Y, Lu, M, Xie, C & Lu, X 2020, 'Driver Drowsiness Recognition via 3D Conditional GAN and Two-Level Attention Bi-LSTM', IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 12, pp. 4755-4768.

[2] Memisevic, R 2013, 'Learning to relate images', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1829-1846.

[3] Krizhevsky, A, Sutskever, I & Hinton, GE 2017, 'Imagenet classification with deep convolutional neural networks', Communications of the ACM, vol. 60, no. 6, pp. 84-90.

[4] Lecun, Y, Bottou, L, Bengio, Y & Haffner, P 1998, 'Gradient-based learning applied to document recognition', In Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324.

[5] Lee, K & Park, M 2021, 'Real-Time Drowsiness Detection Using Eye Closure and Head Pose Estimation', Sensors, vol. 8, no. 2, pp. 4071-4082.

[6] Husain, SS, Mir, J, Anwar, SM, Rafique, W & Ullah, MO 2022, 'Development and validation of a deep learning-based algorithm for drowsiness detection in facial photographs', Multimedia Tools and Applications, vol. 81, no. 1, pp. 20425-20441.

[7] Hwang, J & Kim, N 2021, 'An ensemble learning model for driver drowsiness detection and accident prevention using the behavioral features analysis', International Journal of Intelligent Computing and Cybernetics, vol. 15, no. 2, pp. 224-244.

[8] Ji, S, Xu, W, Yang, M & Yu, K 2013, '3D convolutional neural networks for human action recognition', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 221-231.

[9] Ji, Z, Cao, Y, & Li, J 2020, 'Self-Supervised Learning for Face Feature Extraction', In Proceedings of IEEE International Conference on Multimedia and Expo ICME, pp. 1-6. 28.

[10] Johnson, M & Davis, E 2022, 'Real-Time Driver Drowsiness Detection Using Convolutional Neural Networks', In Proceedings of 6th International Conference on Trends in Electronics and Informatics (ICOEI), Tirunelveli, India, pp. 456-467. 112

[11] Hu, Z, Duan, Q, Zhang, P & Tao, H, 'An attention-erasing stripe pyramid network for face forgery detection', Signal Image and Video Processing, vol. 17, no. 8, pp. 4123-4144.

[12] Hugar, JG, Naseer, MM, Waris, A & Khan, MA 2020, 'Road Traffic Accident Research in India: A Scientometric Study from 1977 to 2020', Library Philosophy and Practice, vol. 14, no. 4325, pp. 1-19. 24.

[13] Ranjan, R, Patel, VM & Chellappa, R 2016, 'HyperFace: A Deep Multi Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 1, pp. 121-135.

[14] Kim, S & Lee, H 2020, 'AUTOMOTIVE: A Case Study on Automatic multimodal Drowsiness detection for smart vehicles', IEEE Access, vol. 9, no. 1, pp. 153678-153700.

[15] King, DE 2009, 'Dlib-ml: A Machine Learning Toolkit. Journal of Machine Learning Research', vol. 10, no. 3, pp. 1755-1758.

[16] Liu, W & Chen, X 2020, 'Deep Belief Networks for Driver Drowsiness Detection', IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 11, pp. 4505-4514.

[17] Liu, W, Wen, Y, Yu, Z, Li, M, Raj, B & Song, L 2017, 'SphereFace: Deep Hypersphere Embedding for Face Recognition', In the proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 6738-6746. 113

[18] Magan, E, Sesmero, MP, Alonso-Weber, JM & Sanchis, A 2022, 'Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images', Applied Sciences, vol. 12, no. 2, pp. 1145-1163.

[19] Nguyen, T & Bui, H 2020, 'Driver Drowsiness Detection Using Deep Learning and Infrared Sensors', Applied Sciences, vol. 10, no. 9, pp. 901-922.

[20] Parkhi, OM, Vedaldi, A & Zisserman, A 2015, 'Deep Face Recognition', In the proceedings of British Machine Vision Conference (BMVC), Swansea, UK, pp. 786-803.

[21] Patel, A & Gupta, R 2022, 'Driver Drowsiness Detection Based on Eyelid Movements', Pattern Recognition Letters, vol. 14, no. 5, pp. 110-119.

[22] Li, H, Lin, Z, Shen, X, Brandt, J & Hua, G 2015, 'Face Detection and Alignment in Unconstrained Environments Using Deep Learning', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 382, no. 3, pp. 242-255.

[23] Li, H, Wang, J, & Fu, Y 2018, 'FaceFeatureNet: A Lightweight and Efficient Face Feature Extraction Model', In the proceedings of 25th IEEE International Conference on Image Processing, Athens, Greece, pp. 2495-2499.

[24] Li, P, Wang, Z, Wang, B & Yang, Y 2020, 'Adaptive Feature Pyramid Network for Face Detection', In the proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, pp. 11942-11951.

[25] Schroff, F, Kalenichenko, D, & Philbin, J 2015, 'FaceNet: A Unified Embedding for Face Recognition and Clustering', In the proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, pp. 815-823.

[26] Shen, W, Wang, Y & Bai, X 2017, 'Context-Aware Face Detection and Landmark Localization', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 10, pp. 2519-2528.

[27] Shepley, A, Falzon, G & Kwan, P 2021, 'Confluence: A robust non-IoU alternative to non-maxima suppression in object detection', IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 10, pp. 1-15.

[28] Singh, A & Chawla, N 2020, 'Driver Drowsiness Detection Using a Hybrid Approach of CNN and LSTM', Pattern Analysis and Applications, vol. 23, no. 4, pp. 1601-1613.