Monitizer: Automating Design and Evaluation of Neural Network Monitors

Muqsit Azeem, Marta Grobelna, Sudeep Kanav, Jan Křetínský, Stefanie Mohr, Sabine Rieder

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

1 Zitat (Scopus)

Abstract

The behavior of neural networks (NNs) on previously unseen types of data (out-of-distribution or OOD) is typically unpredictable. This can be dangerous if the network’s output is used for decision making in a safety-critical system. Hence, detecting that an input is OOD is crucial for the safe application of the NN. Verification approaches do not scale to practical NNs, making runtime monitoring more appealing for practical use. While various monitors have been suggested recently, their optimization for a given problem, as well as comparison with each other and reproduction of results, remain challenging. We present a tool for users and developers of NN monitors. It allows for (i) application of various types of monitors from the literature to a given input NN, (ii) optimization of the monitor’s hyperparameters, and (iii) experimental evaluation and comparison to other approaches. Besides, it facilitates the development of new monitoring approaches. We demonstrate the tool’s usability on several use cases of different types of users as well as on a case study comparing different approaches from recent literature.

OriginalspracheEnglisch
TitelComputer Aided Verification - 36th International Conference, CAV 2024, Proceedings
Redakteure/-innenArie Gurfinkel, Vijay Ganesh
Herausgeber (Verlag)Springer Science and Business Media Deutschland GmbH
Seiten265-279
Seitenumfang15
ISBN (Print)9783031656293
DOIs
PublikationsstatusVeröffentlicht - 2024
Veranstaltung36th International Conference on Computer Aided Verification, CAV 2024 - Montreal, Kanada
Dauer: 24 Juli 202427 Juli 2024

Publikationsreihe

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Band14682 LNCS
ISSN (Print)0302-9743
ISSN (elektronisch)1611-3349

Konferenz

Konferenz36th International Conference on Computer Aided Verification, CAV 2024
Land/GebietKanada
OrtMontreal
Zeitraum24/07/2427/07/24

Fingerprint

Untersuchen Sie die Forschungsthemen von „Monitizer: Automating Design and Evaluation of Neural Network Monitors“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren