Paper Info
Title
DLToDW: Transferring Relational and NoSQL Databases from a Data Lake
Abstract
Over the past decade, digital transformation has led to the evolution of databases towards Big Data. A need to collect and analyze data from different sources has emerged. At the same time, traditional decision support systems are unable to meet the growing needs of modern businesses to integrate and analyze a wide variety of generated data. As a result, most organizations need to convert their data stored in relational systems to NoSQL or "Not only SQL" systems that are based on flexible models and schemas. Our work is part of a medical application that must allow health professionals to analyze complex data for decision making. We propose mechanisms to extract data from a Data Lake and store them in a NoSQL Data Warehouse. This will allow to perform, in a second time, decisional analysis facilitated by the features offered by NoSQL systems (richness of data structures, query language, access performances). In this article, we present a process for ingesting data from a Data Lake into a Data Warehouse. The ingestion consists, first, in transferring relational and NoSQL DBs extracted from the Data Lake into a single NoSQL DB (the Data Warehouse), second, in merging so-called "similar" classes and third, in converting the links into references between objects. To automate this process, we used the Model Driven Architecture (MDA) which provides a schema transformation environment. From the physical schemas describing a Data Lake, we propose transformation rules that allow to create a Data Warehouse stored under a document-oriented NoSQL system. An experimentation has been performed for a medical application.
Year
DOI
Venue
2022
10.1007/s42979-022-01287-7
SN Computer Science
Keywords
DocType
Volume
Data Lake, Data Warehouse, Big Data, NoSQL, Relational databases, MDA, QVT
Journal
3
Issue
ISSN
Citations 
5
2661-8907
0
PageRank 
References 
Authors
0.34
2
3
Name
Order
Citations
PageRank
Jemmali Rym100.34
Fatma Abdelhédi256.18
Zurfluh Gilles300.34