Data Migration from SQL to No SQL using Snapshot- Live Stream Migration

Abstract

Due to massive use of new technologies, the huge amount of data is available and needs to be processed appropriately. Such tremendous amounts of data add a great number of challenges to the traditional database paradigm. Data migration is the procedure of transforming data from one type of database to another. Companies are moving their database from one database (e.g., RDBMS) to another new one (e.g., NoSQL) because of many reasons such as new database can handle more data, that could be fast, scalable etc. Relational database has been used by many organizations for storing and analyzing enterprise database since last few decades. Relational database stores information in a structural or relational way because it follows the structured model but it has many restrictions as compared to the non-structured model. NoSQL uses document model, graph model, keyvalue etc., as compared to RDBMS databases. NoSQL was primarily designed for storing and retrieving a large amount of data. Using NoSQL database in new enterprises is not a major issue because the new application design will be based on NoSQL database. But the issue appears when existing systems that are built on relational database are restructuring database system to implement NoSQL database. They need to reanalyze the system requirements to build up the new database schema. In this research, we will study an approach for migrating real time as well as old data from SQL to NoSQL database using database snapshot and live-stream of database changes. This research also helps us to understand how snapshot and live data migration shows the higher performance as compared to other methods.