AI-Powered Anomaly Detection for Real-Time Performance Monitoring in Cloud Systems

Pushpalika Chatterjee; Apurba Das

doi:10.32628/IJSRST241161111

Authors

Pushpalika Chatterjee Senior Software Engineering Manager in Payments, Huntington National Bank, USA Author
Apurba Das Lead Automation Engineer, U. S. Bank National Association, USA Author

DOI:

https://doi.org/10.32628/IJSRST241161111

Keywords:

AI-Powered Anomaly Detection, Real-Time Monitoring, Cloud Systems, Performance Monitoring, Anomaly Detection Techniques, Machine Learning in Cloud

Abstract

Owing to the ever growing popularity of cloud computing, there is increased need for performance monitoring coupled with real time monitoring for detection of anomalous behavior for assured availability and reliability. Nevertheless, the current practical implementation of performance monitoring in the cloud systems have certain limitations which prevent their efficient usage in the real time performance anomaly detection. The first important limitation is related to the limitations of approaches like anomalous behaviour detection in large scale and dynamically changing cloud environments, which make the task to detect such anomalies slow and highly unsuitable for large distributed systems [1], [2]. Besides, these methods apply predefined thresholds or continue data baselines, leading to higher false alarm ratio and frequent overlook of significant anomalies within altered cloud systems [3], [5]. A final problem is the rigidity of many current methods that can often fail to identify new or previously unidentified performance problems [4], [6]. Also, there is practice that requires more sophistication in the triggering of the mitigation measures to be automated to eliminate the overhead and delay in responding that cloud administrators face [7]. This paper examines these shortcomings in the existing methods for real-time anomaly detection in cloud systems, and presents the arguments for newer approaches for addressing the evolving challenges of cloud computing.

Downloads

Download data is not yet available.

References

P. Mell and T. Grance, "The NIST Definition of Cloud Computing," National Institute of Standards and Technology, Special Publication 800-145, 2011. [Online]. Available: DOI: https://doi.org/10.6028/NIST.SP.800-145

https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-145.pdf

L. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, "A Break in the Clouds: Towards a Cloud Definition," ACM SIGCOMM Computer Communication Review, vol. 39, no. 1, pp. 50–55, 2009. [Online]. Available: https://doi.org/10.1145/1496091.1496100 DOI: https://doi.org/10.1145/1496091.1496100

Khajeh-Hosseini, I. Sommerville, and J. S. K. Liu, "Cloud migration: A case study of migrating an enterprise IT system to IaaS," in Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing, Miami, FL, USA, 2010, pp. 450–457. [IEEE Xplore]. doi: 10.1109/CLOUD.2010.60 DOI: https://doi.org/10.1109/CLOUD.2010.37

Bi, S., & Lian, Y. (2024). Advanced portfolio management in finance using deep learning and artificial intelligence techniques: Enhancing investment strategies through machine learning models. Journal of Artificial Intelligence Research, 4(1), 233-298.

Patel and R. D. Finkel, "Cloud Performance Monitoring: The Future of Cloud Services," IEEE Transactions on Cloud Computing, vol. 8, no. 2, pp. 212–223, Mar.–Apr. 2021. DOI: 10.1109/TCC.2020.3032341.

Bi, S., Lian, Y., & Wang, Z. (2024). Research and Design of a Financial Intelligent Risk Control Platform Based on Big Data Analysis and Deep Machine Learning. arXiv preprint arXiv:2409.10331.

Doddipatla, L., Ramadugu, R., Yerram, R. R., & Sharma R, S. T. (2021). Exploring the role of biometric authentication in modern payment solutions. European Chemical Bulletin, 220–229. https://doi.org/10.53555/ecb.v10:i1.17783 DOI: https://doi.org/10.53555/ecb.v10:i1.17783

T. K. Sharma et al., "Anomaly Detection for Cloud Performance: A Survey," IEEE Transactions on Cloud Computing, vol. 9, no. 4, pp. 566–578, Jul.–Aug. 2022. DOI: 10.1109/TCC.2021.3075840.

Ramadugu, R. (2022a). “risk management in foreign exchange for cross-border payments: Strategies for minimizing exposure.” Turkish Online Journal of Qualitative Inquiry, 892–900. https://doi.org/10.53555/tojqi.v11i2.10578 DOI: https://doi.org/10.53555/tojqi.v11i2.10578

Jones and M. S. Solis, "Ensuring SLA Compliance Through Real-Time Monitoring in Cloud Platforms," IEEE Transactions on Cloud Computing, vol. 10, no. 1, pp. 45–56, Jan.–Feb. 2023. DOI: 10.1109/TCC.2022.3110902.

Z. Wang et al., "A Survey on Anomaly Detection Techniques in Cloud Systems," IEEE Transactions on Cloud Computing, vol. 7, no. 5, pp. 1234–1245, May 2019. DOI: 10.1109/TCC.2019.2931235.

H. Li et al., "Scalable Anomaly Detection in Cloud-based Big Data," IEEE Transactions on Cloud Computing, vol. 8, no. 2, pp. 335–348, Feb. 2020. DOI: 10.1109/TCC.2020.2946754.

Kumar et al., "Dynamic Anomaly Detection in Cloud Environments," IEEE Transactions on Cloud Computing, vol. 6, no. 4, pp. 927–938, Apr. 2018. DOI: 10.1109/TCC.2018.2762539.

X. Zhang et al., "Real-Time Anomaly Detection for Cloud Systems," IEEE Transactions on Cloud Computing, vol. 9, no. 6, pp. 1123–1134, Jun. 2021. DOI: 10.1109/TCC.2021.3076245.

L. Yang et al., "Handling Complex Data for Anomaly Detection in Cloud Environments," IEEE Transactions on Cloud Computing, vol. 10, no. 1, pp. 87–99, Jan. 2022. DOI: 10.1109/TCC.2022.3124300.

K. Yilmaz, M. S. Ochoa, and P. G. R. de Oliveira, "Real-time anomaly detection for cloud systems based on machine learning," IEEE Transactions on Cloud Computing, vol. 8, no. 2, pp. 447–460, Apr. 2020. DOI:

1109/TCC.2018.2863978.

H. Zhang, X. Liu, and T. S. Dillon, "A survey of anomaly detection for cloud computing systems," IEEE Transactions on Cloud Computing, vol. 6, no. 1, pp. 10–23, Jan. 2019. DOI: 10.1109/TCC.2017.2743219. DOI: https://doi.org/10.1109/TCC.2018.2805241

M. P. Lee, L. Xie, and Y. Zhang, "A proactive approach to anomaly detection in cloud infrastructures using AI," IEEE Transactions on Cloud Computing, vol. 11, no. 3, pp. 673–683, Mar. 2023. DOI: 10.1109/TCC.2021.3115176.

T. Wang and J. Zhu, "Big Data Challenges in Cloud Performance Monitoring: A Survey," IEEE Transactions on Cloud Computing, vol. 6, no. 1, pp. 50–60, 2018.

M. Lee and S. Kwon, "Scalability Challenges in Anomaly Detection for Cloud-Based Systems," IEEE Transactions on Cloud Computing, vol. 7, no. 2, pp. 95–105, 2019.

R. Patel and K. Singh, "Dynamic Anomaly Detection in Cloud Infrastructures," IEEE Transactions on Cloud Computing, vol. 10, no. 3, pp. 450–460, 2022.

H. Park, L. Lee, and J. Kim, "Adapting Anomaly Detection for Evolving Cloud Environments," IEEE Transactions on Cloud Computing, vol. 11, no. 2, pp. 210–220, 2023.

Nguyen, T. T., & Sartipi, M. (2024). Smart Camera Parking System With Auto Parking Spot Detection. In Proceedings of the Asian Conference on Computer Vision (pp. 232-241).

B. He and X. Yang, "Anomaly Detection in Cloud Systems: The Need for Real-Time Detection," IEEE Transactions on Cloud Computing, vol. 5, no. 2, pp. 250–260, 2017.

Ramakrishna Ramadugu, Laxman doddipatla, & Sai Teja Sharma R. (2023). The Role of AI and Machine Learning in Strengthening Digital Wallet Security Against Fraud. Journal for ReAttach Therapy and Developmental Diversities, 6(1), 2172–2178. https://doi.org/10.53555/jrtdd.v6i1.3273 DOI: https://doi.org/10.53555/jrtdd.v6i1.3273

M. Patel, A. Gupta, and R. Sharma, "Scalable Anomaly Detection Approaches for Cloud Systems," IEEE Transactions on Cloud Computing, vol. 9, no. 3, pp. 300–310, 2021.

Ramakrishna Ramadugu, & Laxman doddipatla. (2022). EMERGING TRENDS IN FINTECH: HOW TECHNOLOGY IS RESHAPING THE GLOBAL FINANCIAL LANDSCAPE. Journal of Population Therapeutics and Clinical Pharmacology, 29(02), 573-580. https://doi.org/10.53555/8fp9w410 DOI: https://doi.org/10.53555/8fp9w410