Weather Data Management Using Hadoop MapReduce in Zimbabwe

Authors

  • Muchaneta Mayepi  Department of Information Technology, Harare Institute of Technology, P.O Box BE277, Belvedere, Harare, Zimbabwe
  • Eng. Mainford Mutandavari  Department of Software Engineering, Harare Institute of Technology, P.O Box BE277, Belvedere, Harare, Zimbabwe
  • Tinashe Blessing Chuwe  Department of Information Technology, Harare Institute of Technology, P.O Box BE277, Belvedere, Harare, Zimbabwe

DOI:

https://doi.org//10.32628/IJSRST173430

Keywords:

Hadoop, MapReduce, Meteorological Services Department (MSD), automatic weather station, Application Programme Interface.

Abstract

The Meteorological Services Department (MSD) in Zimbabwe is experiencing huge costs in transferring data of over Gigabytes from its different models of automatic weather stations into centralized database at Head Quarters (HQ) Belvedere in Harare. However, surveys indicate that emerging technologies of big data analytics such as Hadoop MapReduce can be extended into weather data management offering multiplicity of functions that includes: prediction, management of transmission of data and also managing the range of weather elements to monitor at anytime and anywhere. This study therefore sought to design and develop an experimental set-up to understand how Hadoop MapReduce can be used to manage data files from various weather stations simulated as Application Programming Interface (APIs). The developed system was conducted entirely in a cloud environment and the results suggest that Hadoop MapReduce function is an appropriate big data tool to manage weather data and offers better options especially for cash strapped organizations such as Meteorological Services Department (MSD). Adoption of MapReduce is economic and elastic to accord changing weather data management requirements. The research used a small dataset of 100 records which might offer comprehensive model for use for practical purposes. Further studies with more realistic huge datasets might be pertinent.

References

  1. K. D. Foote, “A brief history of data management,” Dataversity , New York , 2020.
  2. M. Aguirre-Munizaga and R. Gomez, “A Cloud Computing Based Framework for Storage and Processing of Meteorological Data,” Monash , Washington DC, 2020.
  3. E. E. Filho, A. Albuquerque and M. Nagano, “Identifying SME mortality factors in the life cycle stages: an empirical approach of relevant factors for small business owner-managers in Brazil,” Journal of global entrepreneurship research, vol. 7, no. 5, 2017.
  4. National Climate Data Center, “Managing legacy climate data in the 20th century,” National Climate Data Center (NCDC), Washington DC, 2020.
  5. D. Liberto and J. Berry-Johnson, “Small and Mid-size Enterprise (SME): What is a Small and Mid-size Enterprise (SME)?,” Intercompany Solutions, New York, 2020.
  6. G. Kenneth, I. Hubbard, J. Norman and A. Rosenberg, “Automated weather data network for agriculture,” National Aeronautics and Space Agency (NASA), New York, 2015.
  7. L. E. Wood, “Automatic weather stations,” Friez Instrument, Baltimore, 2016.
  8. V. Dagade, M. Lagali, S. Avadhani and P. Kalekar, “Big data weather analytics using Hadoop,” International journal of emerging technology in computer science and electronics, vol. 14, no. 2, 2015.
  9. B. Reddy and B. A. Patil, “Weather prediction based on big data using Hadoop Map Reduce technique,” International Journal of Advanced Research in computer and communication engineering , vol. 5, no. 6, 2016.
  10. A. Mohammed, K. Adam, N. Abu and M. Majid, “Weather data analyis using Hadoop: Applications and challenges,” Materials science and engineering , vol. 551, 2019.
  11. P. A. Riyaz and S. M. Varghese, “Leveraging MapReduce with Hadoop for weather data analytics,” Journal of computer engineering , vol. 17, no. 3, pp. 6-12, 2015.
  12. National Aeronautics and Space Administration , “Weather forecasting through ages,” National Aeronautics and Space Administration (NASA), New York , 2018.
  13. S. N. Jyothi, B. Divija, S. K. Tharun and S. Reddy, “Weather data analysis using Hadoop MapReduce and Spark,” Journal of Xian University of Architecture and Technology, vol. 12, no. 4, 2020.
  14. S. Abbate, M. Avvenuti, L. Carturan and D. Cesarini, “Deploying a Communicating Automatic Weather Station on an Alpine Glacier by,” Procedia computer science, vol. 19, pp. 1190-1195, 2013.
  15. D. Stuber, A. Mhanda and C. Lefebvre, “Climate Data Management Systems: Status of implementation in developing countries,” Climate Research, vol. 3, no. 1, 2011.
  16. E. Murdyantoro, R. Setiawan and I. Rosyadi, “Prototype weather station uses LoRa wireless,” Journal of Physics, vol. 3, no. 2, p. 9, 2019.
  17. K. Lagouvardos, V. Kotroni, I. Bezes and T. Koletsis, “The automatic weather stations NOANN network of the National Observatory of Athens: operation and database,” Geoscience data Journal, vol. 6, no. 3, 2017.
  18. N. Kumari and S. Gosavi, “Real-Time Cloud based Weather Monitoring System,” International Journal of Climate Change, vol. 4, no. 2, 2019.
  19. R. Posada, D. Nascimento, F. O. Neto, O. Jens, F. Riede and F. Kaspar, “Improving the climate data management in the meteorological service of Angola: experience from SASSCAL,” Advances in science and research, vol. 13, pp. 97-105, 2016.

International Journal of Scientific Research in Science and Technology

Downloads

Published

2022-12-30

Issue

Volume 9, Issue 6, November-December-2022

Section

Research Articles

License

Copyright (c) IJSRST

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

[1]
Muchaneta Mayepi, Eng. Mainford Mutandavari, Tinashe Blessing Chuwe, " Weather Data Management Using Hadoop MapReduce in Zimbabwe, International Journal of Scientific Research in Science and Technology(IJSRST), Online ISSN : 2395-602X, Print ISSN : 2395-6011, Volume 9, Issue 6, pp.305-310, November-December-2022. Available at doi : https://doi.org/10.32628/IJSRST173430