TY - GEN
T1 - Learning Explainable and Better Performing Representations of POMDP Strategies
AU - Bork, Alexander
AU - Chakraborty, Debraj
AU - Grover, Kush
AU - Křetínský, Jan
AU - Mohr, Stefanie
N1 - Publisher Copyright:
© The Author(s) 2024.
PY - 2024
Y1 - 2024
N2 - Strategies for partially observable Markov decision processes (POMDP) typically require memory. One way to represent this memory is via automata. We present a method to learn an automaton representation of a strategy using a modification of the L∗-algorithm. Compared to the tabular representation of a strategy, the resulting automaton is dramatically smaller and thus also more explainable. Moreover, in the learning process, our heuristics may even improve the strategy’s performance. We compare our approach to an existing approach that synthesizes an automaton directly from the POMDP, thereby solving it. Our experiments show that our approach can lead to significant improvements in the size and quality of the resulting strategy representations.
AB - Strategies for partially observable Markov decision processes (POMDP) typically require memory. One way to represent this memory is via automata. We present a method to learn an automaton representation of a strategy using a modification of the L∗-algorithm. Compared to the tabular representation of a strategy, the resulting automaton is dramatically smaller and thus also more explainable. Moreover, in the learning process, our heuristics may even improve the strategy’s performance. We compare our approach to an existing approach that synthesizes an automaton directly from the POMDP, thereby solving it. Our experiments show that our approach can lead to significant improvements in the size and quality of the resulting strategy representations.
UR - http://www.scopus.com/inward/record.url?scp=85192255092&partnerID=8YFLogxK
U2 - 10.1007/978-3-031-57249-4_15
DO - 10.1007/978-3-031-57249-4_15
M3 - Conference contribution
AN - SCOPUS:85192255092
SN - 9783031572487
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 299
EP - 319
BT - Tools and Algorithms for the Construction and Analysis of Systems - 30th International Conference, TACAS 2024, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024, Proceedings
A2 - Finkbeiner, Bernd
A2 - Kovács, Laura
PB - Springer Science and Business Media Deutschland GmbH
T2 - 30th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2024, which was held as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2024
Y2 - 6 April 2024 through 11 April 2024
ER -