AB testing for process versions with contextual multi-armed bandit algorithms

Suhrid Satyal, Ingo Weber, Hye young Paik, Claudio Di Ciccio, Jan Mendling

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Business process improvement ideas can be validated through sequential experiment techniques like AB Testing. Such approaches have the inherent risk of exposing customers to an inferior process version, which is why the inferior version should be discarded as quickly as possible. In this paper, we propose a contextual multi-armed bandit algorithm that can observe the performance of process versions and dynamically adjust the routing policy so that the customers are directed to the version that can best serve them. Our algorithm learns the best routing policy in the presence of complications such as multiple process performance indicators, delays in indicator observation, incomplete or partial observations, and contextual factors. We also propose a pluggable architecture that supports such routing algorithms. We evaluate our approach with a case study. Furthermore, we demonstrate that our approach identifies the best routing policy given the process performance and that it scales horizontally.

Original languageEnglish
Title of host publicationAdvanced Information Systems Engineering - 30th International Conference, CAiSE 2018, Proceedings
EditorsJohn Krogstie, Hajo A. Reijers
PublisherSpringer Verlag
Pages19-34
Number of pages16
ISBN (Print)9783319915623
DOIs
StatePublished - 2018
Externally publishedYes
Event30th International Conference on Advanced Information Systems Engineering, CAiSE 2018 - Tallinn, Estonia
Duration: 11 Jun 201815 Jun 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10816 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference30th International Conference on Advanced Information Systems Engineering, CAiSE 2018
Country/TerritoryEstonia
CityTallinn
Period11/06/1815/06/18

Keywords

  • AB testing
  • Business process management
  • Multi-armed bandit
  • Process performance indicators

Fingerprint

Dive into the research topics of 'AB testing for process versions with contextual multi-armed bandit algorithms'. Together they form a unique fingerprint.

Cite this