Learning Explainable and Better Performing Representations of POMDP Strategies

Alexander Bork, Debraj Chakraborty, Kush Grover, Jan Křetínský, Stefanie Mohr

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Strategies for partially observable Markov decision processes (POMDP) typically require memory. One way to represent this memory is via automata. We present a method to learn an automaton representation of a strategy using a modification of the L-algorithm. Compared to the tabular representation of a strategy, the resulting automaton is dramatically smaller and thus also more explainable. Moreover, in the learning process, our heuristics may even improve the strategy’s performance. We compare our approach to an existing approach that synthesizes an automaton directly from the POMDP, thereby solving it. Our experiments show that our approach can lead to significant improvements in the size and quality of the resulting strategy representations.

Original languageEnglish
Title of host publicationTools and Algorithms for the Construction and Analysis of Systems - 30th International Conference, TACAS 2024, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024, Proceedings
EditorsBernd Finkbeiner, Laura Kovács
PublisherSpringer Science and Business Media Deutschland GmbH
Pages299-319
Number of pages21
ISBN (Print)9783031572487
DOIs
StatePublished - 2024
Event30th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2024, which was held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024 - Luxembourg City, Luxembourg
Duration: 6 Apr 202411 Apr 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14571 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference30th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2024, which was held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024
Country/TerritoryLuxembourg
CityLuxembourg City
Period6/04/2411/04/24

Fingerprint

Dive into the research topics of 'Learning Explainable and Better Performing Representations of POMDP Strategies'. Together they form a unique fingerprint.

Cite this