SplitJoin: A scalable, low-latency stream join architecture with adjustable ordering precision

Mohammadreza Najafi, Mohammad Sadoghi, Hans Arno Jacobsen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

23 Scopus citations

Abstract

There is a rising interest in accelerating stream processing through modern parallel hardware, yet it remains a challenge as how to exploit the available resources to achieve higher throughput without sacrificing latency due to the increased length of processing pipeline and communication path and the need for central coordination. To achieve these objectives, we introduce a novel top-down data flow model for stream join processing (arguably, one of the most resource-intensive operators in stream processing), called SplitJoin, that operates by splitting the join operation into independent storing and processing steps that gracefully scale with respect to the number of cores. Furthermore, SplitJoin eliminates the need for global coordination while preserving the order of input streams by re-thinking how streams are channeled into distributed join computation cores and maintaining the order of output streams by proposing a novel distributed punctuation technique. Throughout our experimental analysis, SplitJoin offered up to 60% improvement in throughput while reducing latency by up to 3.3X compared to state-of-the-art solutions.

Original languageEnglish
Title of host publicationProceedings of the 2016 USENIX Annual Technical Conference, USENIX ATC 2016
PublisherUSENIX Association
Pages493-505
Number of pages13
ISBN (Electronic)9781931971300
StatePublished - 2016
Externally publishedYes
Event2016 USENIX Annual Technical Conference, USENIX ATC 2016 - Denver, United States
Duration: 22 Jun 201624 Jun 2016

Publication series

NameProceedings of the 2016 USENIX Annual Technical Conference, USENIX ATC 2016

Conference

Conference2016 USENIX Annual Technical Conference, USENIX ATC 2016
Country/TerritoryUnited States
CityDenver
Period22/06/1624/06/16

Fingerprint

Dive into the research topics of 'SplitJoin: A scalable, low-latency stream join architecture with adjustable ordering precision'. Together they form a unique fingerprint.

Cite this