Shift Invariance and Deformation Error Properties of Deep Convolutional Neural Networks Based on Wavelets

Johannes Grobmann, Michael Koller, Ullrich Monich, Holger Boche

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

An important step towards a mathematical theory of deep convolutional neural networks (DCNNs) was achieved by investigating so-called scattering networks. For scattering networks, a deformation error stability bound has been established. It remained an open question for which functions in L2 (R d) the bound actually is finite. For practical applications, it is further relevant to know whether the deformation error can be controlled for a 'large' set of functions or only for a 'small' set. Recently, there has been progress regarding the mathematical understanding of scattering networks and new decay bounds on the energy per network layer were discovered. We show how these bounds can be used to control the deformation error by constructing an upper bound on the existing deformation error bounds. The structure of the new deformation error bound is less complex and allows us to conduct a qualitative mathematical analysis using the functional analytic tool of Baire categories and determine the 'size' of the set of functions for which finiteness holds. Our results reveal that the new bound is finite only on a set of first Baire category (meager set). In addition, our investigations focus on shift invariance which is an important property for many signal processing applications. We study the deformation error bounds for shift-invariant closed subspace of L2R) as input for DCNNs. This turns out to be closely related to the Paley-Wiener spaces of bandlimited functions.

Original languageEnglish
Title of host publication2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)9781538635124
DOIs
StatePublished - 24 Aug 2018
Externally publishedYes
Event19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018 - Kalamata, Greece
Duration: 25 Jun 201828 Jun 2018

Publication series

NameIEEE Workshop on Signal Processing Advances in Wireless Communications, SPAWC
Volume2018-June

Conference

Conference19th IEEE International Workshop on Signal Processing Advances in Wireless Communications, SPAWC 2018
Country/TerritoryGreece
CityKalamata
Period25/06/1828/06/18

Keywords

  • Deep convolutional neural network
  • deformation stability
  • scattering network
  • shift-invariant spaces

Fingerprint

Dive into the research topics of 'Shift Invariance and Deformation Error Properties of Deep Convolutional Neural Networks Based on Wavelets'. Together they form a unique fingerprint.

Cite this