Backpropagation Through States: Training Neural Networks with Sequentially Semiseparable Weight Matrices

Matthias Kissel, Martin Gottwald, Biljana Gjeroska, Philipp Paukner, Klaus Diepold

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

Matrix-Vector multiplications usually represent the dominant part of computational operations needed to propagate information through a neural network. This number of operations can be reduced if the weight matrices are structured. In this paper, we introduce a training algorithm for neural networks with sequentially semiseparable weight matrices based on the backpropagation algorithm. By exploiting the structures in the weight matrices, the computational complexity for computing the matrix-vector product can be reduced to the subquadratic domain. We show that this can lead to computing time reductions on a microcontroller. Furthermore, we analyze the generalization capabilities of neural networks with sequentially semiseparable matrices. Our experiments show that neural networks with structured weight matrices can outperform standard feed-forward neural networks in terms of test prediction accuracy for several real-world datasets.

Original languageEnglish
Title of host publicationProgress in Artificial Intelligence - 21st EPIA Conference on Artificial Intelligence, EPIA 2022, Proceedings
EditorsGoreti Marreiros, Bruno Martins, Ana Paiva, Alberto Sardinha, Bernardete Ribeiro
PublisherSpringer Science and Business Media Deutschland GmbH
Pages476-487
Number of pages12
ISBN (Print)9783031164736
DOIs
StatePublished - 2022
Event21st EPIA Conference on Artificial Intelligence, EPIA 2022 - Lisbon, Portugal
Duration: 31 Aug 20222 Sep 2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13566 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference21st EPIA Conference on Artificial Intelligence, EPIA 2022
Country/TerritoryPortugal
CityLisbon
Period31/08/222/09/22

Keywords

  • Efficient inference
  • Neural networks
  • Structured matrices

Fingerprint

Dive into the research topics of 'Backpropagation Through States: Training Neural Networks with Sequentially Semiseparable Weight Matrices'. Together they form a unique fingerprint.

Cite this