Scalable Vector Gaussian Information Bottleneck

Mohammad Mahdi Mahvari, Mari Kobayashi, Abdellatif Zaidi

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung

2 Zitate (Scopus)

Abstract

In the context of statistical learning, the Information Bottleneck (IB) method seeks a right balance between accuracy and generalization capability through a suitable tradeoff between compression complexity, measured by minimum description length, and distortion evaluated under logarithmic loss measure. In this paper, we study a variation of the problem, called scalable information bottleneck, in which the encoder outputs multiple descriptions of the observation with increasingly richer features. The model, which is of successive-refinement type with degraded side information streams at the decoders, is motivated by some application scenarios that require varying levels of accuracy depending on the allowed level of complexity. We establish an analytic characterization of the optimal relevance-complexity region for vector Gaussian sources. Then, we derive a variational inference type algorithm for general sources with unknown distribution; and show means of parametrizing it using neural networks. Finally, we provide experimental results on the MNIST dataset which illustrate that the proposed method generalizes better to unseen data compared to the standard IB with a single description.

OriginalspracheEnglisch
Titel2021 IEEE International Symposium on Information Theory, ISIT 2021 - Proceedings
Herausgeber (Verlag)Institute of Electrical and Electronics Engineers Inc.
Seiten37-42
Seitenumfang6
ISBN (elektronisch)9781538682098
DOIs
PublikationsstatusVeröffentlicht - 12 Juli 2021
Extern publiziertJa
Veranstaltung2021 IEEE International Symposium on Information Theory, ISIT 2021 - Virtual, Melbourne, Australien
Dauer: 12 Juli 202120 Juli 2021

Publikationsreihe

NameIEEE International Symposium on Information Theory - Proceedings
Band2021-July
ISSN (Print)2157-8095

Konferenz

Konferenz2021 IEEE International Symposium on Information Theory, ISIT 2021
Land/GebietAustralien
OrtVirtual, Melbourne
Zeitraum12/07/2120/07/21

Fingerprint

Untersuchen Sie die Forschungsthemen von „Scalable Vector Gaussian Information Bottleneck“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren