Paper Info
Title
Comparison of dataflow control techniques in distributed data-intensive systems
Abstract
In dataflow architectures, each dataflow node (i.e., operation) is typically executed on a single physical node. We are concerned with distributed data-intensive systems, in which each base (i.e., persistent) set of data has been declustered over many physical nodes to achieve load balancing. Because of large base set size, each operation is executed where the base set resides, and intermediate results are transferred between physical nodes. In such systems, each dataflow node is typically executed on many physical nodes. Furthermore, because computations are data-dependent, we cannot know until run time which subset of the physical nodes containing a particular base set will be involved in a given dataflow node. This uncertainty affects program loading, task activation and termination, and data transfer among the nodes.In this paper we focus on the problem of how a dataflow node in such an environment knows when it has received data from all the physical nodes from which it is ever going to receive. We call this the dataflow control problem. The interesting part of the problem is trying to achieve correctness efficiently. We propose three solutions to this problem, and compare them quantitatively by the metrics of total message traffic, message system throughput and data transfer response time.
Year
DOI
Venue
1988
10.1145/55595.55614
SIGMETRICS
Keywords
Field
DocType
single physical node,physical node,data transfer response time,dataflow node,data-intensive system,dataflow architecture,particular base set,dataflow control problem,base set resides,large base set size,dataflow control technique,data transfer,load balance
Dataflow architecture,Data transmission,Computer science,Load balancing (computing),Correctness,Computer network,Response time,Real-time computing,Dataflow,Throughput,Distributed computing,Computation
Conference
Volume
Issue
ISSN
16
1
0163-5999
ISBN
Citations 
PageRank 
0-89791-254-3
7
8.33
References 
Authors
8
2
Name
Order
Citations
PageRank
W. Alexander178.33
George P. Copeland2353156.88