TY - GEN
T1 - Computational and Storage Efficient Quadratic Neurons for Deep Neural Networks
AU - Chen, Chuangtao
AU - Zhang, Grace Li
AU - Yin, Xunzhao
AU - Zhuo, Cheng
AU - Schlichtmann, Ulf
AU - Li, Bing
N1 - Publisher Copyright:
© 2024 EDAA.
PY - 2024
Y1 - 2024
N2 - Deep neural networks (DNNs) have been widely deployed across diverse domains such as computer vision and natural language processing. However, the impressive accomplishments of DNNs have been realized alongside extensive computational demands, thereby impeding their applicability on resource-constrained devices. To address this challenge, many researchers have been focusing on basic neuron structures, the fundamental building blocks of neural networks, to alleviate the computational and storage cost. In this work, an efficient quadratic neuron architecture distinguished by its enhanced utilization of second-order computational information is introduced. By virtue of their better expressivity, DNNs employing the proposed quadratic neurons can attain similar accuracy with fewer neurons and computational cost. Experimental results have demonstrated that the proposed quadratic neuron structure exhibits superior computational and storage efficiency across various tasks when compared with both linear and non-linear neurons in prior work.
AB - Deep neural networks (DNNs) have been widely deployed across diverse domains such as computer vision and natural language processing. However, the impressive accomplishments of DNNs have been realized alongside extensive computational demands, thereby impeding their applicability on resource-constrained devices. To address this challenge, many researchers have been focusing on basic neuron structures, the fundamental building blocks of neural networks, to alleviate the computational and storage cost. In this work, an efficient quadratic neuron architecture distinguished by its enhanced utilization of second-order computational information is introduced. By virtue of their better expressivity, DNNs employing the proposed quadratic neurons can attain similar accuracy with fewer neurons and computational cost. Experimental results have demonstrated that the proposed quadratic neuron structure exhibits superior computational and storage efficiency across various tasks when compared with both linear and non-linear neurons in prior work.
UR - http://www.scopus.com/inward/record.url?scp=85196479314&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85196479314
T3 - Proceedings -Design, Automation and Test in Europe, DATE
BT - 2024 Design, Automation and Test in Europe Conference and Exhibition, DATE 2024 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2024 Design, Automation and Test in Europe Conference and Exhibition, DATE 2024
Y2 - 25 March 2024 through 27 March 2024
ER -