Scaling Distributed Database Joins by Decoupling Computation and Communication

 Scaling Distributed Database Joins by Decoupling Computation and Communication

Abhirup Chakraborty, ACM Member

Abstract

To process a large volume of data, modern data management systems use a collection of machines connected through a network. This paper proposes frameworks and algorithms for processing distributed joins—a compute- and communication-intensive workload in modern data-intensive systems. By exploiting multiple processing cores within the individual machines, we implement a system to process database joins that parallelizes computation within each node, pipelines the computation with communication, parallelizes the communication by allowing multiple simultaneous data transfers (send/receive). Our experimental results show that using only four threads per node the framework achieves a 3.5x gains in intra-node performance while compared with a single-threaded counterpart. Moreover, with the join processing workload the cluster-wide performance (and speedup) is observed to be dictated by the intra-node computational loads; this property brings a near-linear speedup with increasing nodes in the system, a feature much desired in modern large-scale data processing system.


Keywords

Distributed joins; Multi-core; Database; Pipelining; Parallel processing


Issue URL : https://airccse.org/journal/ijdms/current2023.html

Abstract URL: https://aircconline.com/abstract/ijdms/v15n1/15123ijdms02.html

Full Article: https://aircconline.com/ijdms/V15N1/15123ijdms02.pdf

http://airccse.org/journal/ijdms/index.html

#distributedjoins #multicore #database #pipelining #parallelprocessing



Comments

Post a Comment

Popular posts from this blog

3rd International Conference on Computer Science, Engineering and Artificial Intelligence (CSEAI 2025)

Image
 * Registration is currently open * Call for Research Papers!!! Free – Extended Paper will be published as free of cost. 3rd International Conference on Computer Science, Engineering and Artificial Intelligence (CSEAI 2025)   May 30 ~ 31, 2025, Virtual Conference Submission Deadline: January 25, 2025 Submission System Link:  https://cseai2025.org/submission/index.php Contact us: Here's where you can reach us : cseai@cseai2025.org or cseaiconf@gmail.com For More details visit here: https://cseai2025.org/index #computerscience #computersciencemajor #computersciencejoke #computersciencehumor #computerscienceforall #computersciencestudent #computerscienceengineering #computerscienceacademy #computersciencemajorproblems #computersciences #datamining #dataminingfordummies #internetofthings #iot #iota #iotd #iotaphitheta #iotworld #iotiran #iotsecurity #iotnews #iotas #iotmcz #ioteeilmare #iotifolecce #iotproject #iothinks #iotweek #iotproducts #iotsolutions #iotafitness #iotiam...
Read more

A REVIEW OF THE USE OF R PPROGRAMMING FOR DATA SCIENCE RESEARCH IN BOTSWANA

Image
  International Journal of Database Management Systems (IJDMS) IJDMS is indexed World Journal Clout Index (WJCI) ISSN: 0975-5705 (Online); 0975-5985 (Print) http://airccse.org/journal/ijdms/index.html A REVIEW OF THE USE OF R PPROGRAMMING FOR DATA SCIENCE RESEARCH IN BOTSWANA Simisani Ndaba Department of Computer Science, University of Botswana, Gaborone, Botswana https://aircconline.com/ijdms/V15N1/15123ijdms01.pdf
Read more

HYBRID ENCRYPTION ALGORITHMS FOR MEDICAL DATA STORAGE SECURITY IN CLOUD DATABASE

Image
  HYBRID ENCRYPTION ALGORITHMS FOR MEDICAL DATA STORAGE SECURITY IN CLOUD DATABASE Fenghua Zhang1，Yaming Chen2，Weiming Meng3 and Qingtao Wu4 1Computer Technology, Henan University of Science and Technology, Luo Yang, China 2,3Computer Science and Technology, Henan University of Science and Technology, Luo Yang, China 4Professor, Henan University of Science and Technology, Luo Yang, China ABSTRACT Cloud database are derivatives of Cloud computing. At present, most medical institutions store data in cloud database. Although the cloud database improves the efficiency of use, it also poses a huge impact and challenge to the secure storage of data. The article proposes a hybrid algorithm to solve the data security problem in the hospital cloud database. First, the AES algorithm is improved. The improved algorithm is called P-AES algorithm. The P-AES algorithm is then combined with the RSA algorithm, called a hybrid algorithm. The experimental results show that the hybrid enc...
Read more