TY - GEN
T1 - A workflow runtime environment for manycore parallel architectures
AU - Janetschek, Matthias
AU - Prodan, Radu
AU - Benedict, Shajulin
N1 - Publisher Copyright:
© 2015 ACM.
PY - 2015/11/15
Y1 - 2015/11/15
N2 - We introduce a new Manycore Workflow Runtime Environment (MWRE) to efficiently enact traditional scientific workflows on modern manycore computing architectures. In contrast to existing engines that enact workflows acting as external services, MWRE is compiler-based and translates workflows specified in the XMLbased InteroperableWorkflow Intermediate Representation (IWIR) into an equivalent C++-based program. This program efficiently enacts the workflow as a stand-alone executable by means of a new callback mechanism that resolves dependencies, transfers data, and handles composite activities. Experimental results on a number of real-world workflows demonstrate that MWRE clearly outperforms existing Java-based workflow engines designed for distributed (Grid/Cloud) computing infrastructures in terms of enactment time, is generally better than an existing script-based engine for manycore architectures (Swift), and sometimes gets even close to an artificial baseline implementation of the workflows in the standard OpenMP language for shared memory systems.
AB - We introduce a new Manycore Workflow Runtime Environment (MWRE) to efficiently enact traditional scientific workflows on modern manycore computing architectures. In contrast to existing engines that enact workflows acting as external services, MWRE is compiler-based and translates workflows specified in the XMLbased InteroperableWorkflow Intermediate Representation (IWIR) into an equivalent C++-based program. This program efficiently enacts the workflow as a stand-alone executable by means of a new callback mechanism that resolves dependencies, transfers data, and handles composite activities. Experimental results on a number of real-world workflows demonstrate that MWRE clearly outperforms existing Java-based workflow engines designed for distributed (Grid/Cloud) computing infrastructures in terms of enactment time, is generally better than an existing script-based engine for manycore architectures (Swift), and sometimes gets even close to an artificial baseline implementation of the workflows in the standard OpenMP language for shared memory systems.
KW - Enactment engine
KW - Heterogeneous manycore parallel architectures
KW - Scientific workflows
KW - Source-to-source compiler
UR - https://www.scopus.com/pages/publications/84960855326
U2 - 10.1145/2822332.2822333
DO - 10.1145/2822332.2822333
M3 - Conference contribution
AN - SCOPUS:84960855326
T3 - Proceedings of WORKS 2015: 10th Workshop on Workflows in Support of Large-Scale Science - Held in conjunction with SC 2015: The International Conference for High Performance Computing, Networking, Storage and Analysis
BT - Proceedings of WORKS 2015
PB - Association for Computing Machinery
T2 - 10th Workshop on Workflows in Support of Large-Scale Science, WORKS 2015 Held in conjunction with The International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2015
Y2 - 15 November 2015 through 15 November 2015
ER -