TY - GEN
T1 - Massively parallel numa-aware hash joins
AU - Lang, Harald
AU - Leis, Viktor
AU - Albutiu, Martina Cezara
AU - Neumann, Thomas
AU - Kemper, Alfons
N1 - Publisher Copyright:
© Springer International Publishing Switzerland 2015.
PY - 2015
Y1 - 2015
N2 - Driven by the two main hardware trends increasing main memory and massively parallel multi-core processing in the past few years, there has been much research effort in parallelizing well-known join algorithms. However, the non-uniform memory access (NUMA) of these architectures to main memory has only gained limited attention in the design of these algorithms. We study recent proposals of main memory hash join implementations and identify their major performance problems on NUMA architectures. We then develop a NUMA-aware hash join for massively parallel environments, and show how the specific implementation details affect the performance on a NUMA system. Our experimental evaluation shows that a carefully engineered hash join implementation outperforms previous high performance hash joins by a factor of more than two, resulting in an unprecedented throughput of 3/4 billion join argument quintuples per second.
AB - Driven by the two main hardware trends increasing main memory and massively parallel multi-core processing in the past few years, there has been much research effort in parallelizing well-known join algorithms. However, the non-uniform memory access (NUMA) of these architectures to main memory has only gained limited attention in the design of these algorithms. We study recent proposals of main memory hash join implementations and identify their major performance problems on NUMA architectures. We then develop a NUMA-aware hash join for massively parallel environments, and show how the specific implementation details affect the performance on a NUMA system. Our experimental evaluation shows that a carefully engineered hash join implementation outperforms previous high performance hash joins by a factor of more than two, resulting in an unprecedented throughput of 3/4 billion join argument quintuples per second.
UR - http://www.scopus.com/inward/record.url?scp=84921871319&partnerID=8YFLogxK
U2 - 10.1007/978-3-319-13960-9_1
DO - 10.1007/978-3-319-13960-9_1
M3 - Conference contribution
AN - SCOPUS:84921871319
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 3
EP - 14
BT - In Memory Data Management and Analysis - 1st and 2nd International Workshops, IMDM 2013, IMDM 2014, Revised Selected Papers
A2 - Neumann, Thomas
A2 - Pavlo, Andrew
A2 - Levandoski, Justin
A2 - Jagatheesan, Arun
PB - Springer Verlag
T2 - 1st International Workshop on In-Memory Data Management and Analytics, IMDM 2013 and 2nd International Workshop on In-Memory Data Management and Analytics, IMDM 2014
Y2 - 1 September 2014 through 1 September 2014
ER -