Making Compiling Query Engines Practical

Andre Kohn, Viktor Leis, Thomas Neumann

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Compiling queries to machine code is a very efficient way for executing queries. One often overlooked problem with compilation is the time it takes to generate machine code. Even with fast compilation frameworks like LLVM, generating machine code for complex queries often takes hundreds of milliseconds. Such durations can be a major disadvantage for workloads that execute many complex, but quick queries. To solve this problem, we propose an adaptive execution framework, which dynamically switches from interpretation to compilation. We also propose a fast bytecode interpreter for LLVM, which can execute queries without costly translation to machine code and dramatically reduces the query latency. Adaptive execution is fine-grained, and can execute code paths of the same query using different execution modes. Our evaluation shows that this approach achieves optimal performance in a wide variety of settings - low latency for small data sets and maximum throughput for large data sizes. Besides compilation time, we also focus on debugging, which is another important challenge of compilation-based query engines. To address this problem, we present a novel, database-specific debugger for compiling query engines.

Original languageEnglish
Article number8667737
Pages (from-to)597-612
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume33
Issue number2
DOIs
StatePublished - 1 Feb 2021

Keywords

  • Query processing
  • bytecode interpreter
  • debugging
  • query compilation
  • register allocation

Fingerprint

Dive into the research topics of 'Making Compiling Query Engines Practical'. Together they form a unique fingerprint.

Cite this