Study on Big Data Frameworks

Authors

  • Adriano Fernandes  Department of Computer Science, Don Bosco College of Engineering, Goa, India
  • Jonathan Barretto  
  • Jonas Fernandes  

DOI:

https://doi.org/10.32628/IJSRST218475

Keywords:

Latency, Scalability, Fault-Tolerance

Abstract

Big data analytics is becoming more and more popular every day as a tool for evaluating large volumes of data on demand. Apache Hadoop, Spark, Storm, and Flink are four of the most widely used big data processing frameworks. Although all four architectures support big data analysis, they vary in how they are used and the infrastructure that supports it. This paper defines a general collection of main performance metrics, which include Processing Time, CPU Use, Latency, Execution Time, Performance, Scalability, and Fault-tolerance, and contrasting the four big data architectures against these KPIs in a literature review. When compared to Apache Hadoop and Apache Storm frameworks for non-real-time results, Spark was found to be the winner over multiple KPIs, including processing time, CPU usage, Latency, Execution time, and Scalability. In terms of processing time, CPU consumption, latency, execution time, and performance, Flink surpassed Apache Spark and Apache Storm architectures.

References

  1. Safaa Alkatheri,Samah Abbas,Muazzam Siddiqui "A comparative study of big data frameworks” International Journal of Computer Science and Information Security, Vol 17,No.1,pp. 66-71 2019.
  2. Sara Landset, Taghi M. Khoshgoftaar, Aaron N. Richter* and Tawfiq Hasanin “A survey of open source tools for machine learning with big data in the Hadoop ecosystem” pp. 13-16 ,2015
  3. P. Carbone, A. Katsifodimos, S. Ewen, V. Markl, S. Haridi, and K. Tzoumas, “Apache flink: Stream and batch processing in a single engine,” Bull. IEEE Comput. Soc. Tech. Comm. Data Eng., vol. 36, no. 4, pp. 30, 2015.
  4. Ahmed Oussous , Fatima-Zahra Benjelloun , Ayoub Ait Lahcen , Samir Belfkih , “Big Data technologies: A survey” , pp.437-438,2018.
  5. Katrina Sin , Loganathan Muthu “Application of big data in education data mining and learning analytics – a literature review” Vol. 5, No. 4, pp. 1035-1036 , 2015.
  6. Wissem Inoubli,Haithem Mezni,Sabeur Aridhi,Alexander Jung “Big Data Frameworks:A Comparitive Study” ,Future Generation Computer Systems, pp. 6-7, 2018.

International Journal of Scientific Research in Science and Technology

Downloads

Published

2021-08-30

Issue

Volume 8, Issue 4, July-August-2021

Section

Research Articles

License

Copyright (c) IJSRST

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Adriano Fernandes, Jonathan Barretto, Jonas Fernandes "Study on Big Data Frameworks" International Journal of Scientific Research in Science and Technology(IJSRST), Online ISSN : 2395-602X, Print ISSN : 2395-6011,Volume 8, Issue 4, pp.491-499, July-August-2021. Available at doi : https://doi.org/10.32628/IJSRST218475