TY - JOUR
T1 - Everything you always wanted to know about compiled and vectorized queries but were afraid to ask
AU - Kersten, Timo
AU - Leis, Viktor
AU - Kemper, Alfons
AU - Neumann, Thomas
AU - Pavlo, Andrew
AU - Boncz, Peter
N1 - Publisher Copyright:
© 2018, Association for Computing Machinery.
PY - 2018
Y1 - 2018
N2 - The query engines of most modern database systems are either based on vectorization or data-centric code generation. These two state-of-the-art query processing paradigms are fundamentally different in terms of system structure and query execution code. Both paradigms were used to build fast systems. However, until today it is not clear which paradigm yields faster query execution, as many implementation-specific choices obstruct a direct comparison of architectures. In this paper, we experimentally compare the two models by implementing both within the same test system. This allows us to use for both models the same query processing algorithms, the same data structures, and the same parallelization framework to ultimately create an apples-to-apples comparison. We find that both are efficient, but have different strengths and weaknesses. Vectorization is better at hiding cache miss latency, whereas data-centric compilation requires fewer CPU instructions, which benefits cacheresident workloads. Besides raw, single-threaded performance, we also investigate SIMD as well as multi-core parallelization and different hardware architectures. Finally, we analyze qualitative differences as a guide for system architects.
AB - The query engines of most modern database systems are either based on vectorization or data-centric code generation. These two state-of-the-art query processing paradigms are fundamentally different in terms of system structure and query execution code. Both paradigms were used to build fast systems. However, until today it is not clear which paradigm yields faster query execution, as many implementation-specific choices obstruct a direct comparison of architectures. In this paper, we experimentally compare the two models by implementing both within the same test system. This allows us to use for both models the same query processing algorithms, the same data structures, and the same parallelization framework to ultimately create an apples-to-apples comparison. We find that both are efficient, but have different strengths and weaknesses. Vectorization is better at hiding cache miss latency, whereas data-centric compilation requires fewer CPU instructions, which benefits cacheresident workloads. Besides raw, single-threaded performance, we also investigate SIMD as well as multi-core parallelization and different hardware architectures. Finally, we analyze qualitative differences as a guide for system architects.
UR - http://www.scopus.com/inward/record.url?scp=85064903905&partnerID=8YFLogxK
U2 - 10.14778/3275366.3275370
DO - 10.14778/3275366.3275370
M3 - Conference article
AN - SCOPUS:85064903905
SN - 2150-8097
VL - 11
SP - 2209
EP - 2222
JO - Proceedings of the VLDB Endowment
JF - Proceedings of the VLDB Endowment
IS - 13
T2 - 44th International Conference on Very Large Data Bases, VLDB 2018
Y2 - 27 August 2018 through 31 August 2018
ER -