Title
A model, design, and implementation of an efficient multithreaded workflow execution engine with data streaming, caching, and storage constraints
Abstract
The paper proposes a model, design, and implementation of an efficient multithreaded engine for execution of distributed service-based workflows with data streaming defined on a per task basis. The implementation takes into account capacity constraints of the servers on which services are installed and the workflow data footprint if needed. Furthermore, it also considers storage space of the workflow execution engine and its cost. Caching service output data is implemented to speed up the execution of the workflow. Input data is partitioned into data packets, which are passed and processed by services previously selected for workflow tasks so that the aforementioned constraints are met. Performance impact of the proposed mechanisms is investigated for workflow structures common in acyclic directed graph workflow applications. It is shown for a real workflow with distributed processing of digital media content that the initial budget needs to be properly distributed between both the cost of services, but also the cost of intermediate storage to obtain good workflow execution times.
Year
DOI
Venue
2013
10.1007/s11227-012-0837-z
The Journal of Supercomputing
Keywords
Field
DocType
Workflow execution,Data streaming,Storage constraints,Service selection
Workflow technology,Computer science,Network packet,Parallel computing,Server,Windows Workflow Foundation,Directed acyclic graph,Workflow engine,Workflow management system,Workflow,Operating system,Distributed computing
Journal
Volume
Issue
ISSN
63
3
0920-8542
Citations 
PageRank 
References 
7
0.52
22
Authors
1
Name
Order
Citations
PageRank
Pawel Czarnul112121.11