A Smart HW-Accelerator for Non-uniform Linear Interpolation of ML-Activation Functions

The compulsive nonlinearity in neural networks (NN) is introduced by well-known nonlinear functions called activation functions. Performing AI-inferences on edge devices calls for efficient approximation of those complex functions on highly restrictive hardware (HW) platforms. When designing such systems, balancing area, footprint and power consumption at an application appropriate latency is key. To address those challenges, we propose a HW-based interpolation component capable of approximating arbitrary mathematical functions. A combinatorial search-based optimization algorithm is employed to find the optimal set of interpolation points for a set of functions while also considering non-uniform distributions. The proposed solution is accompanied by a Python-based HW generator, that facilitates the process of deploying software-computed search results on HW and provides room for generating different flavors of application-optimized HW. In an effort to reduce area footprint and delay, the proposed approach exploits symmetry and biased symmetry properties of functions and applies bit width optimizations to reduce the size of the utilized computational units. Additionally, property-aware reprogrammable solutions for multifunctional use cases are incorporated into our design. Experimental analyses show that our proposed method permits achieving better area utilization and deviation error results than state-of-the-art implementations.