Adversarial Robustness of Multi-bit Convolutional Neural Networks

Lukas Frickenstein, Shambhavi Balamuthu Sampath, Pierpaolo Mori, Manoj Rohit Vemparala, Nael Fasfous, Alexander Frickenstein, Christian Unger, Claudio Passerone, Walter Stechele

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

Abstract

Deploying convolutional neural networks (CNNs) on resource-constrained, embedded hardware constitutes challenges in balancing task-related accuracy and resource-efficiency. For safety-critical applications, a third optimization objective is crucial, namely the robustness of CNNs. To address these challenges, this paper investigates the tripartite optimization problem of task-related accuracy, resource-efficiency, and adversarial robustness of CNNs by utilizing multi-bit networks (MBNs). To better navigate the tripartite optimization space, this work thoroughly studies the design space of MBNs by varying the number of weight and activation bases. First, the pro-active defensive model MBN3x1 is identified, by conducting a systematic evaluation of the design space. This model achieves better adversarial accuracy (+10.3pp) against the first-order attack PGD-20 and has 1.3 × lower bit-operations, with a slight degradation of natural accuracy (–2.4pp) when compared to a 2-bit fixed-point quantized implementation of ResNet-20 on CIFAR-10. Similar observations hold for deeper and wider ResNets trained on different datasets, such as CIFAR-100 and ImageNet. Second, this work shows that the defensive capability of MBNs can be increased by adopting a state-of-the-art adversarial training (AT) method. This results in an improvement of adversarial accuracy (+13.6pp) for MBN3 × 3, with a slight degradation in natural accuracy (–2.4pp) compared to the costly full-precision ResNet-56 on CIFAR-10, which has 7 × more bit-operations. To the best of our knowledge, this is the first paper highlighting the improved robustness of differently configured MBNs and providing an analysis on their gradient flows.

OriginalspracheEnglisch
TitelIntelligent Systems and Applications - Proceedings of the 2023 Intelligent Systems Conference IntelliSys Volume 3
Redakteure/-innenKohei Arai
Herausgeber (Verlag)Springer Science and Business Media Deutschland GmbH
Seiten157-174
Seitenumfang18
ISBN (Print)9783031477140
DOIs
PublikationsstatusVeröffentlicht - 2024
VeranstaltungIntelligent Systems Conference, IntelliSys 2023 - Amsterdam, Niederlande
Dauer: 7 Sept. 20238 Sept. 2023

Publikationsreihe

NameLecture Notes in Networks and Systems
Band824 LNNS
ISSN (Print)2367-3370
ISSN (elektronisch)2367-3389

Konferenz

KonferenzIntelligent Systems Conference, IntelliSys 2023
Land/GebietNiederlande
OrtAmsterdam
Zeitraum7/09/238/09/23

Fingerprint

Untersuchen Sie die Forschungsthemen von „Adversarial Robustness of Multi-bit Convolutional Neural Networks“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren