An Identity Level Protocol for Calibrated Open Set Face Recognition in Identity Scarce Scenarios

Divya Saleela; Rani Saritha R; Rahila N A; Reji R; Teena Joseph; Sujith Kumar P S; Gunjan Mittal Roy; Syed Arshad Ali; Chinchu M S

doi:10.29284/g1pnt122

Authors

Divya Saleela Assistant Professor, College of Engineering Aranmula, Kerala, India
Rani Saritha R Associate Professor, Saintgits College of Engineering (Autonomous), Kerala, India
Rahila N A Research Scholar, Thangal Kunju Musaliar College of Engineering, Kerala, India
Reji R Professor, Thangal Kunju Musaliar Institute of Technology, Kerala, India
Teena Joseph Professor, St. Thomas College of Engineering & Technology, Kerala, India
Sujith Kumar P S Professor, Sree Buddha College of Engineering, Kerala, India
Gunjan Mittal Roy Associate Professor, IILM University, Greater Noida, India
Syed Arshad Ali Assistant Professor, Sharda University, Greater Noida, India
Chinchu M S Assistant Professor, Sree Buddha College of Engineering, Kerala, India

DOI:

https://doi.org/10.29284/g1pnt122

Keywords:

Open-set face recognition, identity scarcity, identity-level evaluation, face embeddings, decision calibration, LFW dataset, unknown identity rejection, reliability-aware recognition.

Abstract

Open-set face recognition in unconstrained environments is hindered by identity scarcity, unreliable rejection of unseen individuals, and poorly calibrated decision confidence. This paper introduces an identity-level open-set evaluation protocol on the Labeled Faces in the Wild (LFW) dataset that explicitly models unknown identities during inference and enforces strict separation between training, gallery, and probe identities. The protocol enables joint assessment of open-set verification, identification accuracy, and decision calibration under both balanced and full-open operating conditions. Using this framework, a face-specific embedding model based on FaceNet with VGGFace2 pretraining is evaluated against strong generic and foundation-model baselines. Experimental results show that, using the proposed evaluation protocol, the best-performing face-specialized baseline (FaceNet trained on VGGFace2) achieves robust open-set discrimination and identification performance, with a ROC-AUC of 0.964, an equal error rate of 0.059, a Top-1 identification accuracy of 0.933, and an accepted-known identification accuracy of 0.982 under full-open evaluation. Furthermore, post-hoc calibration significantly improves confidence reliability without affecting discriminative performance. The proposed protocol and results demonstrate the importance of identity-aware evaluation and calibrated decision mechanisms for practical open-set face recognition.

References

[1] A. Zhalgas, B. Amirgaliyev, and A. Sovet, ‘Robust Face Recognition Under Challenging Conditions: A Comprehensive Review of Deep Learning Methods and Challenges’, Appl. Sci., vol. 15, no. 17, p. 9390, Aug. 2025, doi: 10.3390/app15179390.

[2] R. H. Vareto, Y. Linghu, T. E. Boult, W. R. Schwartz, and M. Günther, ‘Open-set face recognition with maximal entropy and Objectosphere loss’, Image Vis. Comput., vol. 141, p. 104862, Jan. 2024, doi: 10.1016/j.imavis.2023.104862.

[3] M. H. Fahim, A. A. Donkol, A. Bedair, and M. Abdel-Nasser, ‘Image Processing A Decade-by-Decade Review with a Focus on Face Recognition’, SVU-Int. J. Eng. Sci. Appl., vol. 6, no. 2, pp. 29–46, Dec. 2025, doi: 10.21608/svusrc.2025.339279.1251.

[4] M. Kim, A. Jain, and X. Liu, ‘50 Years of Automated Face Recognition’, 2025, arXiv. doi: 10.48550/ARXIV.2505.24247.

[5] I. Adjabi, A. Ouahabi, A. Benzaoui, and A. Taleb-Ahmed, ‘Past, Present, and Future of Face Recognition: A Review’, Electronics, vol. 9, no. 8, p. 1188, Jul. 2020, doi: 10.3390/electronics9081188.

[6] A. S. Khan, M. J. Abbass, and A. H. Khan, ‘Towards Fault-Aware Image Captioning: A Review on Integrating Facial Expression Recognition (FER) and Object Detection’, Sensors, vol. 25, no. 19, p. 5992, Sep. 2025, doi: 10.3390/s25195992.

[7] A. Chouldechova, S. Deng, Y. Wang, W. Xia, and P. Perona, ‘Unsupervised and Semi-supervised Bias Benchmarking in Face Recognition’, in Computer Vision – ECCV 2022, vol. 13673, S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds, in Lecture Notes in Computer Science, vol. 13673. , Cham: Springer Nature Switzerland, 2022, pp. 289–306. doi: 10.1007/978-3-031-19778-9_17.

[8] Y. Kortli, M. Jridi, A. Al Falou, and M. Atri, ‘Face Recognition Systems: A Survey’, Sensors, vol. 20, no. 2, p. 342, Jan. 2020, doi: 10.3390/s20020342.

[9] A. K. Bahjat, ‘A survey of facial recognition techniques’, Int. J. Commun. Inf. Technol., vol. 6, no. 2, pp. 214–225, Jul. 2025, doi: 10.33545/2707661X.2025.v6.i2c.167.

[10] A. A. A. Khalifa, ‘Efficient and robust face recognition in the wild’, Universitäts- und Landesbibliothek Sachsen-Anhalt, 2025. doi: 10.25673/118968.

[11] A. P. Fard and M. H. Mahoor, ‘Ad-Corre: Adaptive Correlation-Based Loss for Facial Expression Recognition in the Wild’, IEEE Access, vol. 10, pp. 26756–26768, 2022, doi: 10.1109/ACCESS.2022.3156598.

[12] Fayin Li and H. Wechsler, ‘Open set face recognition using transduction’, IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 11, pp. 1686–1697, Nov. 2005, doi: 10.1109/TPAMI.2005.224.

[13] W. J. Scheirer, L. P. Jain, and T. E. Boult, ‘Probability Models for Open Set Recognition’, IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 11, pp. 2317–2324, Nov. 2014, doi: 10.1109/TPAMI.2014.2321392.

[14] C. Geng, S.-J. Huang, and S. Chen, ‘Recent Advances in Open Set Recognition: A Survey’, IEEE Trans. Pattern Anal. Mach. Intell., vol. 43, no. 10, pp. 3614–3631, Oct. 2021, doi: 10.1109/TPAMI.2020.2981604.

[15] E. Lopez-Lopez, X. M. Pardo, and C. V. Regueiro, ‘Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition’, Pattern Recognit., vol. 131, p. 108885, Nov. 2022, doi: 10.1016/j.patcog.2022.108885.

[16] L. Best-Rowden, H. Han, C. Otto, B. F. Klare, and A. K. Jain, ‘Unconstrained Face Recognition: Identifying a Person of Interest From a Media Collection’, IEEE Trans. Inf. Forensics Secur., vol. 9, no. 12, pp. 2144–2157, Dec. 2014, doi: 10.1109/TIFS.2014.2359577.

[17] L. Boudry, J. D. Nador, and M. Ramon, ‘Determinants of Face Recognition: The Role of Target Prevalence and Similarity’, J. Cogn., vol. 7, no. 1, p. 27, Feb. 2024, doi: 10.5334/joc.339.

[18] L. Laishram, M. Shaheryar, J. T. Lee, and S. K. Jung, ‘Toward a Privacy-Preserving Face Recognition System: A Survey of Leakages and Solutions’, ACM Comput. Surv., vol. 57, no. 6, pp. 1–38, Jun. 2025, doi: 10.1145/3673224.

[19] Y. Yang et al., ‘Recent Advances of Foundation Language Models-based Continual Learning: A Survey’, ACM Comput. Surv., vol. 57, no. 5, pp. 1–38, May 2025, doi: 10.1145/3705725.

[20] S. M. H. Abidi, S. A. Hassan, S. M. Raza, and M. J. Beliatis, ‘Advances in Face Recognition: A Comprehensive Review of Feature Extraction and Dataset Evaluation’, Electronics, vol. 15, no. 2, p. 338, Jan. 2026, doi: 10.3390/electronics15020338.

[21] N. Weber and N. Brewer, ‘The effect of judgment type and confidence scale on confidence-accuracy calibration in face recognition.’, J. Appl. Psychol., vol. 88, no. 3, pp. 490–499, 2003, doi: 10.1037/0021-9010.88.3.490.

[22] C. Eliades, L. Lenc, P. Král, and H. Papadopoulos, ‘Automatic face recognition with well-calibrated confidence measures’, Mach. Learn., vol. 108, no. 3, pp. 511–534, Mar. 2019, doi: 10.1007/s10994-018-5756-7.

[23] C. Tomani, D. Cremers, and F. Buettner, ‘Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration’, in Computer Vision – ECCV 2022, vol. 13673, S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, and T. Hassner, Eds, in Lecture Notes in Computer Science, vol. 13673. , Cham: Springer Nature Switzerland, 2022, pp. 555–569. doi: 10.1007/978-3-031-19778-9_32.

[24] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, ‘Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments’, University of Massachusetts, Amherst, MA, USA, 07–49, Oct. 2007.