Optimizing Credit Card Approval Decisions Through Advanced Gradient Boosting Techniques
DOI:
https://doi.org/10.29284/e535hr68Keywords:
credit card approval, risk assessment of credit card application, gradient boosting, catboost, imbalance in class, cost sensitive learning, probability calibration, interpretability of model, machine learning.Abstract
Credit card approval systems need accurate, reliable and interpretable predictive models to support financial decision making under the condition of class imbalance. This work examines the performance of more sophisticated approaches to gradient boosting for credit card approval decisions based on the UCI Credit Card Default dataset made freely available to the public. The quantitative, experiment framework is taken to evaluate AdaBoost, LightGBM, XGBoost, and optimized CatBoost model under the same preprocessing, sampling and evaluation condition. Cost-sensitive learning is used to solve class imbalance, categorical feature correction, early stopping, and decision threshold optimization are combined to improve the detection of minority class. Experimental results show that the proposed CatBoost framework can achieve better performance compared with baseline and competing models, which achieves a ROC-AUC of 0.7809, F1-score of 0.5490 and balanced accuracy of 0.7157 on a held-out test set. Statistical validation through McNemar's and Wilcoxon tests prove that the observed gains in performance are statistically significant. Calibration and interpretability analyses provide further support for the reliability and transparency of the proposed approach. The results show that optimized gradient boosting models, especially CatBoost, are a powerful and practical solution for credit card approval systems that can offer better risk sensitivity, interpretability, and decision reliability for financial applications in the real world.
References
1. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd ed. New York, NY, USA: Springer, 2009.
2. Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119–139, 1997.
3. J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” Annals of Statistics, vol. 29, no. 5, pp. 1189–1232, 2001.
4. J. H. Friedman, “Stochastic gradient boosting,” Computational Statistics & Data Analysis, vol. 38, no. 4, pp. 367–378, 2002.
5. [5] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1263–1284, 2009.
6. F. Provost and T. Fawcett, “Robust classification for imprecise environments,” Machine Learning, vol. 42, no. 3, pp. 203–231, 2001.
7. C. Elkan, “The foundations of cost-sensitive learning,” in Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI), 2001, pp. 973–978.
8. G. Ke et al., “LightGBM: A highly efficient gradient boosting decision tree,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.
9. L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “CatBoost: Unbiased boosting with categorical features,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 31, 2018.
10. S. Lessmann, B. Baesens, H. V. Seow, and L. C. Thomas, “Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research,” European Journal of Operational Research, vol. 247, no. 1, pp. 124–136, 2015.
11. B. Baesens, T. Van Gestel, M. Stepanova, D. Van den Poel, and J. Vanthienen, “Benchmarking state-of-the-art classification algorithms for credit scoring,” Journal of the Operational Research Society, vol. 54, no. 6, pp. 627–635, 2003.
12. [12] D. J. Hand and W. E. Henley, “Statistical classification methods in consumer credit scoring: A review,” Journal of the Royal Statistical Society, vol. 160, no. 3, pp. 523–541, 1997.
13. C.-F. Tsai and J.-W. Wu, “Using neural network ensembles for bankruptcy prediction and credit scoring,” Expert Systems with Applications, vol. 34, no. 4, pp. 2639–2649, 2008.
14. A. Dal Pozzolo, O. Caelen, R. A. Johnson, and G. Bontempi, “Calibrating probability with undersampling for unbalanced classification,” in Proceedings of the IEEE Symposium Series on Computational Intelligence, 2015, pp. 159–166.
15. B. Zadrozny and C. Elkan, “Transforming classifier scores into accurate multiclass probability estimates,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2002, pp. 694–699.
16. A. Niculescu-Mizil and R. Caruana, “Predicting good probabilities with supervised learning,” in Proceedings of the International Conference on Machine Learning (ICML), 2005.
17. S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30, 2017.
18. M. T. Ribeiro, S. Singh, and C. Guestrin, “Why should I trust you? Explaining the predictions of any classifier,” in Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135–1144.
19. A. C. Bahnsen, D. Aouada, and B. Ottersten, “Cost-sensitive decision trees for fraud detection,” Expert Systems with Applications, vol. 42, no. 11, pp. 5380–5390, 2015.
20. UCI Machine Learning Repository, “Default of Credit Card Clients Dataset.” [Online]. Available: https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
