SplitJoin: A scalable, low-latency stream join architecture with adjustable ordering precision

Mohammadreza Najafi, Mohammad Sadoghi, Hans Arno Jacobsen

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

26 Zitate (Scopus)

Abstract

There is a rising interest in accelerating stream processing through modern parallel hardware, yet it remains a challenge as how to exploit the available resources to achieve higher throughput without sacrificing latency due to the increased length of processing pipeline and communication path and the need for central coordination. To achieve these objectives, we introduce a novel top-down data flow model for stream join processing (arguably, one of the most resource-intensive operators in stream processing), called SplitJoin, that operates by splitting the join operation into independent storing and processing steps that gracefully scale with respect to the number of cores. Furthermore, SplitJoin eliminates the need for global coordination while preserving the order of input streams by re-thinking how streams are channeled into distributed join computation cores and maintaining the order of output streams by proposing a novel distributed punctuation technique. Throughout our experimental analysis, SplitJoin offered up to 60% improvement in throughput while reducing latency by up to 3.3X compared to state-of-the-art solutions.

OriginalspracheEnglisch
TitelProceedings of the 2016 USENIX Annual Technical Conference, USENIX ATC 2016
Herausgeber (Verlag)USENIX Association
Seiten493-505
Seitenumfang13
ISBN (elektronisch)9781931971300
PublikationsstatusVeröffentlicht - 2016
Extern publiziertJa
Veranstaltung2016 USENIX Annual Technical Conference, USENIX ATC 2016 - Denver, USA/Vereinigte Staaten
Dauer: 22 Juni 201624 Juni 2016

Publikationsreihe

NameProceedings of the 2016 USENIX Annual Technical Conference, USENIX ATC 2016

Konferenz

Konferenz2016 USENIX Annual Technical Conference, USENIX ATC 2016
Land/GebietUSA/Vereinigte Staaten
OrtDenver
Zeitraum22/06/1624/06/16

Fingerprint

Untersuchen Sie die Forschungsthemen von „SplitJoin: A scalable, low-latency stream join architecture with adjustable ordering precision“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren