Lightweight Instruction Set for Flexible Dilated Convolutions and Mixed-Precision Operands

Simon Friedrich, Shambhavi Balamuthu Sampath, Robert Wittig, Manoj Rohit Vemparala, Nael Fasfous, Emil Matus, Walter Stechele, Gerhard Fettweis

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Modern deep neural networks specialized for object detection and semantic segmentation require specific operations to increase or preserve the resolution of their feature maps. Hence, more generic convolution layers called transposed and dilated convolutions are employed, adding a large number of zeros between the elements of the input features or weights. Usually, standard neural network hardware accelerators process these convolutions in a straightforward manner, without paying attention to the added zeros, resulting in an increased computation time. To cope with this problem, recent works propose to skip the redundant elements with additional hardware or solve the problem efficiently only for a limited range of dilation rates. We present a general approach for accelerating transposed and dilated convolutions that does not introduce any hardware overhead while supporting all dilation rates. To achieve this, we introduce a novel precision-scalable lightweight instruction set and memory scheme that can be applied to the different convolution variants. This results in a speed-up of 5 times in DeepLabV3+ outperforming the recently proposed design methods. The support of precision-scalable execution of all workloads further increases the speedup in computation time shown for the PointPillars, DeepLabV3+, and ENet networks. Compared to the state-of-the-art commercial EdgeTPU, the instruction footprint of ResNet-50 of our designed accelerator is reduced by 60 percent.

Original languageEnglish
Title of host publicationProceedings of the 24th International Symposium on Quality Electronic Design, ISQED 2023
PublisherIEEE Computer Society
ISBN (Electronic)9798350334753
DOIs
StatePublished - 2023
Event24th International Symposium on Quality Electronic Design, ISQED 2023 - San Francisco, United States
Duration: 5 Apr 20237 Apr 2023

Publication series

NameProceedings - International Symposium on Quality Electronic Design, ISQED
Volume2023-April
ISSN (Print)1948-3287
ISSN (Electronic)1948-3295

Conference

Conference24th International Symposium on Quality Electronic Design, ISQED 2023
Country/TerritoryUnited States
CitySan Francisco
Period5/04/237/04/23

Keywords

  • DNN
  • accelerator
  • address generation
  • dilated convolution
  • instruction set
  • memory alignment
  • mixed-precision
  • stride
  • transposed convolution

Fingerprint

Dive into the research topics of 'Lightweight Instruction Set for Flexible Dilated Convolutions and Mixed-Precision Operands'. Together they form a unique fingerprint.

Cite this