MapReduce Framework to Improve the Efficiency of Large Scale Item Sets in IoT Using Parallel Mining of Representative Patterns in Big Data

A Geetha; Ravindra Changala; Goda Gangaram; Dr. Mahesh Kotha

doi:10.32628/IJSRST229618

Authors

A Geetha Assistant Professor, CSIT Department, CVR College of Engineering, Hyderabad, Telangana, India
Ravindra Changala Assistant Professor, IT Department, Guru Nanak Institutions Technical Campus, Hyderabad, Telangana, India
Goda Gangaram Assistant Professor, Department of CSE (AI&ML), CMR Technical Campus, Hyderabad, Telangana, India
Dr. Mahesh Kotha Assistant Professor, Department of CSE (AI&ML), CMR Technical Campus, Hyderabad, Telangana, India

DOI:

https://doi.org//10.32628/IJSRST229618

Keywords:

Big data, IoT data analysis, MapReduce, online parallel mining, representative pattern.

Abstract

With the coming of the time of huge information, individuals can gather rich and various information from a wide assortment of assortment gadgets, like those of the Internet of Things. Information concealed in enormous information is extremely helpful and important. Frequent pattern mining, as a basic method of data mining, is applied to every aspect of society. However, the application of traditional frequent pattern mining methods to big data involves bottlenecks due to the large number of result sets. Such bottlenecks make it challenging to deliver commonsense worth underway and life. We proposed a new approach which involved representative patterns using mining technique. This framework can make the runtime difficult to evaluate in large data environments. Our approach gives best solution for the above gaps with the help of online representative pattern-set parallel-mining algorithm. We used MapReduce framework which performs horizontal segmentation over data bases. Finally, several performance optimization strategies are proposed. As shown by numerous experiments on the actual dataset, the algorithm proposed in this paper improves the time efficiency by one order of magnitude. Several optimization strategies reduce the execution time to varying degrees.

References

A.Yassine, S. Singh, and A. Alamri, ``Mining human activity patterns from smart home big data for health care applications,'' IEEE Access, vol. 5, pp. 13131-13141, 2017.
Ravindra Changala, “Retrieval of Valid Information from Clustered and Distributed Databases” , Journal of Innovation in Computer Science and Engineering, Vol. 6(1), July – Dec 2016.
U. Yun, G. Lee, and K.-M. Lee, ``Efcient representative pattern mining based on weight and maximality conditions,'' Expert Syst., vol. 33, no. 5, pp. 439462, Jun. 2016.
D. Incebacak, R. Zilan, B. Tavli, J. M. Barcelo-Ordinas, and J. Garcia- Vidal, ``Optimal data compression for lifetime maximization in wireless sensor networks operating in stealth mode,'' Ad Hoc Netw., vol. 24, pp. 134-147, Jan. 2014.
D. Xin, J. Han, X. Yan, and H. Cheng, ``Mining compressed frequent pattern sets,'' in Proc. Int. Conf. Very Large Data Bases. Sep. 2005, pp. 709-720.
C. Severance, ``The Apache software foundation: Brian Behlendorf,'' Computer, vol. 45, no. 10, pp. 8-9, Oct. 2012.
Ravindra Changala, “A Survey on Development of Pattern Evolving Model For Discovery of Patterns in Text Mining Using Data Mining Techniques” , Journal of Theoretical & Applied Information Technology, Volume 95, Issue 16.
T. D. T. Do, A. Termier, and A. Laurent, ``PGLCM: Efcient parallel mining of closed frequent gradual itemsets,'' in Proc. ICDM, vol. 43, no. 3, Dec. 2010, pp. 138147.
J. Wang, J. Han, and J. Pei, ``CLOSETC: Searching for the best strategies for mining frequent closed itemsets,'' in Proc. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Jul. 2003, pp. 92369245.
K. Iwanuma, Y. Yamamoto, and S. Fukuda, ``A completeness on an online "-approximation algorithm for closed frequent itemset mining in a transactional stream,'' Trans. Jpn. Soc. Artif. Intell., vol. 31, no. 5, Aug. 2016. [11] C.-H. Xu, ``A novel approach for data stream maximal frequent itemsets mining,'' Int. J. Wireless Mobile Comput., vol. 10, no. 3, pp. 224231, Jan. 2016.
Ravindra Changala, “Data Mining Challenges With Big Data” , International Journal for Research in Applied Science & Engineering Technology (IJRASET), Volume 3 Issue VI, June 2015, ISSN: 2321-9653.
K. Gouda and M. J. Zaki, ``Efciently mining maximal frequent itemsets,'' in Proc. IEEE Int. Conf. Data Mining, San Jose, CA, USA, Nov./Dec. 2001, pp. 163170.
Y. Yan, Z. Li, and H. Chen, ``Fast mining maximal frequent itemsets based on FP-tree,'' in Proc. Int. Conf. Conceptual Modeling, Shanghai, China, Nov. 2004, pp. 348-361.
D. Xin, H. Cheng, X. Yan, and J. Han, ``Extracting redundancy-aware top k patterns,'' in Proc. ACM SIGKDD Int. Conf. KnowI. Discovery Data Mining, Philadelphia, PA, USA, Aug. 2006, pp. 20-23.
V. Tseng, C.-W. Wu, P. Fournier-Viger, and S. Y. Philip, ``Efficient algorithms for mining top-K high utility itemsets,'' IEEE Trans. Knowl. Data Eng., vol. 28, no. 1, pp. 54-67, Jan. 2016.