FlexLog: A Shared Log for Stateful Serverless Computing

Dimitra Giantsidi, Emmanouil Giortamis, Nathaniel Tornow, Florin Dinu, Pramod Bhatotia

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Stateful serverless applications need to persist their state and data. The existing approach is to store the data in general purpose storage systems. However, these approaches are not designed to meet the demands of serverless applications in terms of consistency, fault tolerance and performance. We present FlexLog, a storage system, specifically a distributed shared log, distinctively designed to meet the requirements of stateful serverless computing while mitigating the relevant system bottlenecks. FlexLog's data layer leverages the state-of-the-art persistent memory (PM) to offer low latency I/O and improve performance. To match the performance, FlexLog's ordering layer employs a scalable design, namely a tree-structure set of sequencer nodes. Importantly, this design provides serverless applications with the flexibility to implement different consistency guarantees and to seamlessly support multi-tenancy configurations. We implement FlexLog from the ground up on a real hardware testbed and we also prove the correctness of our protocols. In particular, we evaluate FlexLog on a cluster of 6 machines with 800 GB Intel Optane DC PM over a 10 Gbps interconnect. Our evaluation shows that FlexLog scales to millions of operations per second while maintaining minimal latency. Our comparison with the state-of-the-art shared log for serverless, Boki, shows that we achieve 10x better throughput in the storage layer and 2x-4x lower latency in the ordering layer, while also providing flexibility to support different consistency properties and multi-tenancy.

Original languageEnglish
Title of host publicationHPDC 2023 - Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
PublisherAssociation for Computing Machinery, Inc
Pages195-209
Number of pages15
ISBN (Electronic)9798400701559
DOIs
StatePublished - 7 Aug 2023
Event32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023 - Orlando, United States
Duration: 16 Jun 202323 Jun 2023

Publication series

NameHPDC 2023 - Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing

Conference

Conference32nd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2023
Country/TerritoryUnited States
CityOrlando
Period16/06/2323/06/23

Keywords

  • persistent memory
  • serverless computing
  • shared distributed log

Fingerprint

Dive into the research topics of 'FlexLog: A Shared Log for Stateful Serverless Computing'. Together they form a unique fingerprint.

Cite this