TY - GEN
T1 - One size fits all, again! the architecture of the hybrid OLTP&OLAP database management system HyPer
AU - Kemper, Alfons
AU - Neumann, Thomas
PY - 2011
Y1 - 2011
N2 - Real time business intelligence demands to execute OLAP queries on a current, up-to-date state of the transactional OLTP data. The currently exercised separation of transaction processing on the OLTP database and BI query processing on the data warehouse that is only periodically refreshed violates this goal. We propose to enhance the transactional database with highly effective query processing capabilities. We contrast different architectures proposed for achieving the real-time BI goal: versioning of the data and thereby separating the query from the transactions workload, continuous DW refreshing, heterogeneous workload management, update staging by periodically merging the update delta into the queryable main database, update and query batching, and our newly developed virtual memory snapshot mechanism based on hardware-supported shadowed pages. In our HyPer main memory database management system, the OLAP queries are executed on an arbitrarily current database snapshot that is created by forking the OLTP process. This snapshot is efficiently maintained by the operating system's copy on write mechanism that is supported by the MMU hardware. To assess the performance of such hybrid DBMSs we propose a new OLTP&OLAP benchmark that combines the transaction processing functionality of the TPC-C benchmark with the query workload of the TPC-H benchmark in one mixed workload. Based on this benchmark we substantiate the claim that it is indeed possible to architect a hybrid system that can achieve the transaction throughput of dedicated in-memory OLTP systems and, in parallel, execute a BI workload on the same data at the same performance as dedicated OLAP systems, such as in-memory column stores.
AB - Real time business intelligence demands to execute OLAP queries on a current, up-to-date state of the transactional OLTP data. The currently exercised separation of transaction processing on the OLTP database and BI query processing on the data warehouse that is only periodically refreshed violates this goal. We propose to enhance the transactional database with highly effective query processing capabilities. We contrast different architectures proposed for achieving the real-time BI goal: versioning of the data and thereby separating the query from the transactions workload, continuous DW refreshing, heterogeneous workload management, update staging by periodically merging the update delta into the queryable main database, update and query batching, and our newly developed virtual memory snapshot mechanism based on hardware-supported shadowed pages. In our HyPer main memory database management system, the OLAP queries are executed on an arbitrarily current database snapshot that is created by forking the OLTP process. This snapshot is efficiently maintained by the operating system's copy on write mechanism that is supported by the MMU hardware. To assess the performance of such hybrid DBMSs we propose a new OLTP&OLAP benchmark that combines the transaction processing functionality of the TPC-C benchmark with the query workload of the TPC-H benchmark in one mixed workload. Based on this benchmark we substantiate the claim that it is indeed possible to architect a hybrid system that can achieve the transaction throughput of dedicated in-memory OLTP systems and, in parallel, execute a BI workload on the same data at the same performance as dedicated OLAP systems, such as in-memory column stores.
UR - http://www.scopus.com/inward/record.url?scp=79961199936&partnerID=8YFLogxK
U2 - 10.1007/978-3-642-22970-1_2
DO - 10.1007/978-3-642-22970-1_2
M3 - Conference contribution
AN - SCOPUS:79961199936
SN - 9783642229695
T3 - Lecture Notes in Business Information Processing
SP - 7
EP - 23
BT - Enabling Real-Time Business Intelligence - 4th International Workshop, BIRTE 2010, Held at the 36th International Conference on Very Large Databases, VLDB 2010, Revised Selected Papers
PB - Springer Verlag
T2 - 4th International Workshop on Enabling Real-Time Business Intelligence, BIRTE 2010, Held at the 36th International Conference on Very Large Databases, VLDB 2010
Y2 - 13 September 2010
ER -