An Optimized XGBoost Framework for Real-Time Credit Card Fraud Detection: Addressing Class Imbalance with Hybrid SMOTE-ENN Resampling

Abstract

Financial fraud, especially in the form of credit card fraud, presents considerable threats to both consumers and businesses, thus requiring sophisticated detection mechanisms. This paper introduces an enhanced framework based on XGBoost to tackle essential issues in fraud detection: class imbalance, the need for real-time processing, and the importance of model interpretability. Using the Kaggle Credit Card Fraud Dataset (which includes 285,807 transactions with a fraud rate of 0.17%), we apply a SMOTE-ENN hybrid resampling method to equalize class distributions and create features (derived from PCA on V1–V28 and temporal metrics) for effective training. The model incorporates cost-sensitive learning by imposing penalties that are 10 times greater for false negatives, and it employs Youden’s J statistic to fine-tune decision thresholds (0.32). The experimental findings show a 92% F1-score and an AUC-ROC of 0.98, surpassing LightGBM (89% F1) and LSTM networks (90% F1) while achieving a crucial inference latency of 12ms for real-time payment gateways. Analysis of errors indicates strategies to address false positives (by whitelisting high amounts) and false negatives (through spend velocity features). The scalability of the framework is confirmed through a simulation of 1 million transactions per second on AWS, which demonstrates a throughput of 985,000 transactions per second.