Efficient Signature Localization and Detection from Authenticated ID Cards

Thejashwini B L; H S Nagendraswamy

doi:10.29284/sza6kh02

Authors

Thejashwini B L Department of Studies in Computer Science, University of Mysore, Mysuru, India, 0009-0005-7273-8754
H S Nagendraswamy Department of Studies in Computer Science, University of Mysore, Mysuru, India,0009-0001-2614-6169

DOI:

https://doi.org/10.29284/sza6kh02

Keywords:

Object Detection, Computer Vision, YOLO, Detection Transformer.

Abstract

In this digital era, the verification of a person's identity based on authenticated ID cards plays an important role. The verification can be done in many ways; out of that, the signature present on the documents is used for personal authenticity. The main aim of this research article is to localize and detect signatures from various types of authenticated ID cards, such as PAN cards, driving licenses, college IDs, and so on. The deep learning models were used, such as YOLO variants (YOLOv5, YOLOv8, YOLOv9, YOLOv10, YOLOv11), DETR (Detection Transformer), RT-DETR (Real- time Detection Transformer), and Detectron2. The proposed model detects the signature from newly collected datasets, some images sourced from the MASK-RCNN-Dataset and the Roboflow platform. The model was tested with our newly collected datasets without any preprocessing. To measure the effectiveness of the models, we have used the most common metric in detection phase as, mean average precision (mAP50 and mAP50-95). All the models detect signatures with above 96% accuracy. This article also compares all the models and observes that the RT-DETR model gives efficient results. Specifically, using the RT- DETR_r101vd_coco_0365 backbone model, RT-DETR achieved a detection accuracy of 100% with a pre-processed dataset and also observed that all models obtained an IoU ranging from 92% to 97%.

References

1. BJ, B. N., & Jose, M. (2023, February). Multiple Object Recognition from Smart Document Images using YOLOv5s. In 2023 7th International Conference on Computing Methodologies and Communication (ICCMC) (pp. 824-828). IEEE.

2. Li, Y. Q., Chang, H. S., & Lin, D. T. (2022). Large-scale printed Chinese character recognition for ID cards using deep learning and few samples transfer learning. Applied Sciences, 12(2), 907.

3. Silva, P. G., Junior, C. A., Lima, E. B., Bezerra, B. L., & Zanchettin, C. (2019, September). Speeding-up the handwritten signature segmentation process through an optimized fully convolutional neural network. In 2019 International Conference on Document Analysis and Recognition (ICDAR) (pp. 1417-1423). IEEE.

4. Junior, C. A., da Silva, M. H. M., Bezerra, B. L. D., Fernandes, B. J. T., & Impedovo, D. (2020, July). Fcn+ rl: A fully convolutional network followed by refinement layers to offline handwritten signature segmentation. In 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1-7). IEEE.

5. Zuo, L., Chen, W., Qu, H., Huang, L., Wang, Z., & Chen, Y. (2019). An intelligent knowledge extraction framework for recognizing identification information from real-world ID card images. IEEE Access, 7, 165448-165457.

6. Satyawan, W., Pratama, M. O., Jannati, R., Muhammad, G., Fajar, B., Hamzah, H., ... & Kristian, K. (2019, June). Citizen ID card detection using image processing and optical character recognition. In Journal of Physics: Conference Series (Vol. 1235, No. 1, p. 012049). IOP Publishing.

7. Rusli, F. M., Adhiguna, K. A., & Irawan, H. (2021, August). Indonesian id card extractor using optical character recognition and natural language post-processing. In 2021 9th International Conference on Information and Communication Technology (ICoICT) (pp. 621-626). IEEE.

8. Zhu, J., Ma, H., Feng, J., & Dai, L. (2018, April). ID card number detection algorithm based on convolutional neural network. In AIP Conference Proceedings (Vol. 1955, No. 1). AIP Publishing.

9. Van Hoai, D. P., Duong, H. T., & Hoang, V. T. (2021). Text recognition for Vietnamese identity card based on deep features network. International Journal on Document Analysis and Recognition (IJDAR), 24(1), 123-131.

10. Zhang, Y., Zhang, J., Yan, K., Wang, H., & Wang, G. (2023). Handwritten Chinese signature detection with simple Copy–Paste augmentation on power plants technical documents. Service Oriented Computing and Applications, 17(4), 293-302.

11. das Neves, R. B., Verçosa, L. F., Macêdo, D., Bezerra, B. L. D., & Zanchettin, C. (2020, July). A fast fully octave convolutional neural network for document image segmentation. In 2020 International Joint Conference on Neural Networks (IJCNN) (pp. 1-6). IEEE.

12. Rezkiani, K., & Nurtanio, I. (2022, November). Logo Detection Using You Only Look Once (YOLO) Method. In 2022 2nd International Conference on Electronic and Electrical Engineering and Intelligent System (ICE3IS) (pp. 29-33). IEEE.

13. Saha, R., Mondal, A., & Jawahar, C. V. (2019, September). Graphical object detection in document images. In 2019 International Conference on Document Analysis and Recognition (ICDAR) (pp. 51-58). IEEE.

14. Yan, K., Zhang, Y., Tang, H., Ren, C., Zhang, J., Wang, G., & Wang, H. (2022). Signature detection, restoration, and verification: A novel chinese document signature forgery detection benchmark. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 5163-5172).

15. Sohan, M., Sai Ram, T., Reddy, R., & Venkata, C. (2024). A review on yolov8 and its advancements. In International Conference on Data Intelligence and Cognitive Informatics (pp. 529-545). Springer, Singapore.

16. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020, August). End-to-end object detection with transformers. In European conference on computer vision (pp. 213-229). Cham: Springer International Publishing.

17. Zhao, Y., Lv, W., Xu, S., Wei, J., Wang, G., Dang, Q., & Chen, J. (2024). Detrs beat yolos on real-time object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 16965-16974).

18. Thejashwini, B. L., Nagendraswamy, H. S., Rakshitha, P., & Manjuchethan, N. (2024). Integrating YOLO and GAN for robust signature detection and similarity assessment. International Journal of Research and Analysis in Science and Engineering, 4(5), Sept 2024. ISSN: 2582-8118.

19. M. M. Tall, I. Ngom, O. Sadio, I. Diagne, A. Coulibaly and M. Ndiaye, "Detection of Fish Species Captured by Artisanal Canoes Using YOLOv11," 2024 International Conference on Intelligent Computing and Next Generation Networks (ICNGN), Bangkok, Thailand, 2024, pp. 1-5, doi: 10.1109/ICNGN63705.2024.10871297.