ELASTIC PERFORMANCE FOR ETL+Q PROCESSING

 


ELASTIC PERFORMANCE FOR ETL+Q PROCESSING

Pedro Martins, Maryam Abbasi, Pedro Furtado

Department of Informatics, University of Coimbra, Portugal

ABSTRACT

Most data warehouse deployments are not prepared to scale automatically, although some applications have large or increasing requirements concerning data volume, processing times, data rates, freshness and need for fast responses. The solution is to use parallel architectures and mechanisms to speed-up data integration and to handle fresh data efficiently. Those parallel approaches should scale automatically. In this work, we investigate how to provide scalability and data freshness automatically, and how to manage high-rate data efficiently in very large data warehouses. The framework proposed in this work handles parallelization and scales of the data-warehouse when necessary. It does not only scale-out to increase the processing capacity, but it also scales in when resources are underused. In general, data freshness is also not guaranteed in those contexts, because data loading, transformation, and integration are heavy tasks that are done only periodically, instead of row-by-row. The framework we propose is designed to provide data freshness as well.

KEYWORDS

Scalability, ETL, freshness, high-rate, performance, parallel processing, distributed systems, database, load-balance, algorithm

Full Text: https://aircconline.com/ijdms/V8N1/8116ijdms02.pdf

Volume Link: https://airccse.org/journal/ijdms/current2016.html

 

Comments

Post a Comment

Popular posts from this blog

3rd International Conference on Computer Science, Engineering and Artificial Intelligence (CSEAI 2025)

Image
 * Registration is currently open * Call for Research Papers!!! Free – Extended Paper will be published as free of cost. 3rd International Conference on Computer Science, Engineering and Artificial Intelligence (CSEAI 2025)   May 30 ~ 31, 2025, Virtual Conference Submission Deadline: January 25, 2025 Submission System Link:  https://cseai2025.org/submission/index.php Contact us: Here's where you can reach us : cseai@cseai2025.org or cseaiconf@gmail.com For More details visit here: https://cseai2025.org/index #computerscience #computersciencemajor #computersciencejoke #computersciencehumor #computerscienceforall #computersciencestudent #computerscienceengineering #computerscienceacademy #computersciencemajorproblems #computersciences #datamining #dataminingfordummies #internetofthings #iot #iota #iotd #iotaphitheta #iotworld #iotiran #iotsecurity #iotnews #iotas #iotmcz #ioteeilmare #iotifolecce #iotproject #iothinks #iotweek #iotproducts #iotsolutions #iotafitness #iotiam...
Read more

A REVIEW OF THE USE OF R PPROGRAMMING FOR DATA SCIENCE RESEARCH IN BOTSWANA

Image
  International Journal of Database Management Systems (IJDMS) IJDMS is indexed World Journal Clout Index (WJCI) ISSN: 0975-5705 (Online); 0975-5985 (Print) http://airccse.org/journal/ijdms/index.html A REVIEW OF THE USE OF R PPROGRAMMING FOR DATA SCIENCE RESEARCH IN BOTSWANA Simisani Ndaba Department of Computer Science, University of Botswana, Gaborone, Botswana https://aircconline.com/ijdms/V15N1/15123ijdms01.pdf
Read more

HYBRID ENCRYPTION ALGORITHMS FOR MEDICAL DATA STORAGE SECURITY IN CLOUD DATABASE

Image
  HYBRID ENCRYPTION ALGORITHMS FOR MEDICAL DATA STORAGE SECURITY IN CLOUD DATABASE Fenghua Zhang1，Yaming Chen2，Weiming Meng3 and Qingtao Wu4 1Computer Technology, Henan University of Science and Technology, Luo Yang, China 2,3Computer Science and Technology, Henan University of Science and Technology, Luo Yang, China 4Professor, Henan University of Science and Technology, Luo Yang, China ABSTRACT Cloud database are derivatives of Cloud computing. At present, most medical institutions store data in cloud database. Although the cloud database improves the efficiency of use, it also poses a huge impact and challenge to the secure storage of data. The article proposes a hybrid algorithm to solve the data security problem in the hospital cloud database. First, the AES algorithm is improved. The improved algorithm is called P-AES algorithm. The P-AES algorithm is then combined with the RSA algorithm, called a hybrid algorithm. The experimental results show that the hybrid enc...
Read more