LPRHNet: Depth-Wise Convolution Enabled Feature Learning for License Plate Recognition through Hybrid Network Scheme

Deepesh  Dewangan; Dr. Ranu  Pandey

doi:10.29284/n0v3dw58

Authors

Deepesh Dewangan Shri Rawatpura Sarkar University, Raipur, Chhattisgarh, India
Dr. Ranu Pandey Shri Rawatpura Sarkar University, Raipur, Chhattisgarh, India

DOI:

https://doi.org/10.29284/n0v3dw58

Keywords:

Convolutional Neural Networks, Feature Extraction and Learning, Hybrid Architecture, License Plate Recognition, Smart Traffic Systems

Abstract

License plate recognition is challenging due to varied lighting, diverse plate designs, and occlusions. It is crucial for smart traffic management, enabling efficient monitoring, law enforcement, and traffic flow optimization, contributing to enhanced safety and streamlined urban mobility. In this direction, we introduce a hybrid Convolutional Neural Network (CNN) model for license plate detection, merging the strengths of DenseNet, InceptionNet, and XceptionNet through depth-wise convolution. Our model attains unprecedented accuracy, boasting remarkable results on diverse public datasets: 99.1% for AOLP, 96.8% for Chinese LP, 59.8% for RODOSOL-ALPR, and 78.3% for UFPR-ALPR. The model’s collective prowess manifests in an impressive average accuracy of 83.5%, positioning it at the forefront of license plate detection methodologies. Notably, our approach surpasses several state-of-the-art methods and is validated through standard evaluation measures. This work signifies a substantial stride toward more accurate and robust systems crucial for real-world applications, such as smart traffic management and law enforcement.

References

1. Dewangan, D.K., Sahu, S.P. Towards the design of vision-based intelligent vehicle system: methodologies and challenges. Evol. Intel. 16, 759–800 (2023). https://doi.org/10.1007/s12065-022-00713-2.

2. “Road traffic deaths, Global Health Observatory data repository byWorld Health Organization,” 2019. http://apps.who.int/gho/data/node.main.A997.

3. D. K. Dewangan and S. P. Sahu, “Real time object tracking for intelligent vehicle,” 2020 1st Int. Conf. Power, Control Comput. Technol. ICPC2T 2020, pp. 134–138, 2020, doi: 10.1109/ICPC2T48082.2020.9071478.

4. A. A. H. Farhat et al., “OCR-based hardware implementation for qatari number plate on the Zynq SoC,” 2017 9th IEEE-GCC Conf. Exhib. GCCCE 2017, 2018, doi: 10.1109/IEEEGCC.2017.8448145.

5. J. Yepez and S. B. Ko, “Improved license plate localisation algorithm based on morphological operations,” IET Intell. Transp. Syst., ol. 12, no. 6, pp. 542–549, 2018, doi: 10.1049/iet-its.2017.0224.

6. F.Wang, L. Man, B.Wang, Y. Xiao, W. Pa, and X. Lu, “Fuzzy-based algorithm for color recognition of license plates,” Pattern Recognit. Lett., vol. 29, no. 7, pp. 1007–1020, 2008, doi: 10.1016/j.patrec.2008.01.026.

7. J. fang Song, “Vehicle Detection Using Spatial Relationship GMM for ComplexUrban Surveillance in Daytime and Nighttime,” Int. J. Parallel Program., vol. 46, no. 5, pp. 859–872, 2018, doi: 10.1007/s10766-017-0543-9.

8. J. M. Guo, Y. F. Liu, and J. Der Lee, “License plate localization and character segmentation with feedback self-learning and hybrid-binarization techniques,” IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 57, no. 3, pp. 1417–1424, 2007, doi: 10.1109/TENCON.2007.4429069.

9. C. Y. Lei and J. H. Liu, “Vehicle license plate character segmentation method based on Watershed algorithm,” 2010 Int. Conf. Mach. Vis. Human-Machine Interface, MVHI 2010, pp. 447–452, 2010, doi: 10.1109/MVHI.2010.161.

10. J. F. Xu, S. F. Li, and M. S. Yu, “Car license plate extraction using color and edge information,” Proc. 2004 Int. Conf. Mach. Learn. Cybern., vol. 6, no. August, pp. 3904–3907, 2004, doi: 10.1109/icmlc.2004.1380528.

11. W. Weihong and T. Jiaoyang, “Research on License Plate Recognition Algorithms Based on Deep Learning in Complex Environment,” IEEE Access, vol. 8, pp. 91661–91675, 2020, doi: 10.1109/ACCESS.2020.2994287.

12. D. K. Dewangan and S. P. Sahu, “Lane detection in intelligent vehicle system using optimal 2- tier deep convolutional neural network,” Multimed. Tools Appl., vol. 82, no. 5, pp. 7293–7317, 2023, doi: 10.1007/s11042-022-13425-7.

13. Gupta, A. Anpalagan, L. Guan, and A. S. Khwaja, “Deep learning for object detection and scene perception in self-driving cars: Survey, challenges, and open issues,” Array, vol. 10, no. January, p. 100057, 2021, doi: 10.1016/j.array.2021.100057.

14. X.Wu, W. Li, D. Hong, R. Tao, and Q. Du, “Deep Learning for Unmanned Aerial Vehicle-Based Object Detection and Tracking: A survey,” IEEE Geosci. Remote Sens. Mag., vol. 10, no. 1, pp. 91–124, 2022, doi: 10.1109/MGRS.2021.3115137.20

15. Mhalla, T. Chateau, and N. E. Ben Amara, “Spatio-temporal object detection by deep learning: Video-interlacing to improve multi-object tracking,” Image Vis. Comput., vol. 88, pp. 120–131, 2019, doi: 10.1016/j.imavis.2019.03.002.

16. D. K. Dewangan and S. P. Sahu, “Optimized Convolutional Neural Network for Road Detection with Structured Contour and Spatial Information for Intelligent Vehicle System,” Int. J. Pattern Recognit. Artif. Intell., vol. 36, no. 6, 2022, doi: 10.1142/S0218001422520024.

17. D. Zheng, Y. Zhao, and J. Wang, “An efficient method of license plate location,” Pattern Recognit. Lett., vol. 26, no. 15, pp. 2431–2438, 2005, doi: 10.1016/j.patrec.2005.04.014.

18. F. Faradji, A. H. Rezaie, and M. Ziaratban, “A morphological-based license plate location,” Proc. - Int. Conf. Image Process. ICIP, vol. 1, pp. I-57-I–60, 2006, doi: 10.1109/ICIP.2007.4378890.

19. S. Rasheed, A. Naeem, and O. Ishaq, “Automated number plate recognition using hough lines and template matching,” Lect. Notes Eng. Comput. Sci., vol. 1, pp. 199–203, 2012.

20. S. K. Kim, D. W. Kim, and H. J. Kim, “Recognition of vehicle license plate using a genetic algorithm based segmentation,” IEEE Int. Conf. Image Process., vol. 2, pp. 661–664, 1996, doi: 10.1109/icip.1996.560964.

21. W. Jia, H. Zhang, and X. He, “Region-based license plate detection,” J. Netw. Comput. Appl., vol. 30, no. 4, pp. 1324–1333, 2007, doi: 10.1016/j.jnca.2006.09.010.

22. K. Deb, H. U. Chae, and K. H. Jo, “Vehicle license plate detection method based on sliding concentric windows and histogram,” J. Comput., vol. 4, no. 8, pp. 771–777, 2009, doi: 10.4304/jcp.4.8.771-777.

23. H. Zhang, W. Jia, X. He, and Q. Wu, “Learning-based license plate detection using global and local features,” Proc. - Int. Conf. Pattern Recognit., vol. 2, pp. 1102–1105, 2006, doi: 10.1109/ICPR.2006.758.

24. J. Matas and K. Zimmermann, “Unconstrained licence plate and text localization and recognition,” IEEE Conf. Intell. Transp. Syst. Proceedings, ITSC, vol. 2005, pp. 572–577, 2005, doi: 10.1109/ITSC.2005.1520111.

25. H. Li and C. Shen, “Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs,” 2016, [Online]. Available: http://arxiv.org/abs/1601.05610.

26. K. H. Lin, H. Tang, and T. S. Huang, “Robust license plate detection using image saliency,” Proc. - Int. Conf. Image Process. ICIP, vol. 2, no. 1, pp. 3945–3948, 21, 2010, doi: 10.1109/ICIP.2010.5649878.

27. F. Porikli and T. Kocak, “Robust license plate detection using covariance descriptor in a neural network framework,” Proc. - IEEE Int. Conf. Video Signal Based Surveill. 2006, AVSS 2006, no. November 2006, pp. 107–112, 2006, doi: 10.1109/AVSS.2006.100.

28. K. K. Kim, K. I. Kim, J. B. Kim, and H. J. Kim, “Learning-based approach for license plate recognition,” Neural Networks Signal Process. - Proc. IEEE Work., vol. 2, no. C, pp. 614–623, 2000, doi: 10.1109/nnsp.2000.890140.

29. T. Nukano, M. Fukumi, and M. Khalid, “Vehicle license plate character recognition by neural networks,” Proc. 2004 Int. Symp. Intell. Signal Process. Commun. Syst. ISPACS 2004, pp. 771–775, 2004, doi: 10.1109/ispacs.2004.1439164.

30. J. Jiao, Q. Ye, and Q. Huang, “A configurable method for multi-style license plate recognition,” Pattern Recognit., vol. 42, no. 3, pp. 358–369, 2009, doi: 10.1016/j.patcog.2008.08.016.

31. S. M. Silva and C. R. Jung, “Real-time license plate detection and recognition using deep convolutional neural networks,” J. Vis. Commun. Image Represent., vol. 71, p. 102773, 2020, doi: 10.1016/j.jvcir.2020.102773.

32. B. Shi, X. Bai, and C. Yao, “An End-to-End Trainable Neural Network for Image- Based Sequence Recognition and Its Application to Scene Text Recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 11, pp. 2298–2304, 2017, doi: 10.1109/nsyss.2017.7885812.

33. R. Laroca, A. B. Araujo, L. A. Zanlorensi, E. C. De Almeida, and D. Menotti, “Towards Image-Based Automatic Meter Reading in Unconstrained Scenarios: A Robust and Efficient Approach,” IEEE Access, vol. 9, pp. 67569–67584, 2021, doi: 10.1109/ACCESS.2021.3077415.

34. J. Wang and X. Hu, “Gated recurrent convolution neural network for OCR,” Adv. Neural Inf. Process. Syst., vol. 2017-December, no. Nips, pp. 335–344, 2017.

35. G. R. Gon，calves, M. A. Diniz, R. Laroca, D. Menotti, and W. R. Schwartz, “Multi-task Learning for Low-Resolution License Plate Recognition,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11896 LNCS, no. i, pp. 251–261, 2019, doi: 10.1007/978-3-030-33904-3 23.

36. J. Spanhel, J. Sochor, R. Juranek, A. Herout, L. Marsik, and P. Zemcik, “Holistic recognition of low quality license plates by CNN using track annotated data,” 2017 14th IEEE Int. Conf. Adv. Video Signal Based Surveillance, AVSS 2017, no. August, 2017, doi: 10.1109/AVSS.2017.8078501.22

37. R. Atienza, “Vision Transformer for Fast and Efficient Scene Text Recognition,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 12821 LNCS, pp. 319–334, 2021, doi: 10.1007/978-3-030-86549-8 21.

38. J. Baek et al., “What is wrong with scene text recognition model comparisons? dataset and model analysis,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2019- October, pp. 4714–4722, 2019, doi: 10.1109/ICCV.2019.00481.

39. W. Liu, C. Chen, K. Y. K. Wong, Z. Su, and J. Han, “STAR-Net: A spatial attention residue network for scene text recognition,” Br. Mach. Vis. Conf. 2016, BMVC 2016, vol. 2016-September, no. September, pp. 43.1-43.13, 2016, doi: 10.5244/C.30.43.

40. F. Borisyuk, A. Gordo, and V. Sivakumar, “Rosetta: Large scale system for text detection and recognition in images,” Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., pp. 71–79, 2018, doi: 10.1145/3219819.3219861.

41. B. Shi, X. Wang, P. Lyu, C. Yao, and X. Bai, “Robust Scene Text Recognition with Automatic Rectification,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, pp. 4168–4176, 2016, doi: 10.1109/CVPR.2016.452.

42. C. Y. Lee and S. Osindero, “Recursive Recurrent Nets with Attention Modeling for OCR in the Wild,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-December, no. 3, pp. 2231–2239, 2016, doi: 10.1109/CVPR.2016.245.

43. R. Laroca, E. Cardoso, D. Lucio, V. Estevam, and D. Menotti, “On the Crossdataset Generalization in License Plate Recognition,” Int. Conf. Comput. Vis. Theory Appl., pp. 166–178, 2022, doi: 10.5220/0010846800003124.

44. G. Hsu, J. Chen, and Y. Chung, “Application-Oriented License Plate Recognition,” IEEE Trans. Veh. Technol., vol. 62, no. 2, pp. 552–561, 2013, doi: 10.1109/TVT.2012.2226218.

45. R. Laroca et al., “A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector,” Proc. Int. Jt. Conf. Neural Networks, vol. 2018- July, 2018, doi: 10.1109/IJCNN.2018.8489629.

46. W. Zhou, H. Li, Y. Lu, and Q. Tian, “Principal visual word discovery for automatic license plate detection,” IEEE Trans. Image Process., vol. 21, no. 9, pp. 4269–4279, 2012, doi: 10.1109/TIP.2012.2199506.

47. M. L¨angkvist, L. Karlsson, and A. Loutfi, “Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” Pattern Recognit. Lett., vol. 42, no. 1, pp. 11–24, 2014, [Online]. Available: http://arxiv.org/abs/1512.0056723

48. F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” in In Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258. doi: 10.4271/2014-01-0975.

49. G. Huang, Z. Liu, L. van der Maaten, and K. Q.Weinberger, “Densely Connected Convolutional Networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708.

50. Dewangan, D.K., Sahu, S.P. (2021). Lane Detection for Intelligent Vehicle System Using Image Processing Techniques. In: Verma, G.K., Soni, B., Bourennane, S., Ramos, A.C.B. (eds) Data Science. Transactions on Computer Systems and Networks. Springer, Singapore. https://doi.org/10.1007/978-981-16-1681-5 21.

51. Sahu, S., Sahu, S.P., Dewangan, D.K. (2023). Pedestrian Detection Using MobileNetV2 Based Mask R-CNN. In: Joby, P.P., Balas, V.E., Palanisamy, R. (eds) IoT Based Control Networks and Intelligent Systems. Lecture Notes in Networks and Systems, vol 528. Springer, Singapore. https://doi.org/10.1007/978-981-19-5845-8 22 .