Inference software library for power-efficient neural networks

The embARC Machine Learning Inference (MLI) software library provides developers with optimised functions to implement neural network layer types, significantly reducing processor cycle counts for applications that require low power and area, such as voice detection, speech recognition, and sensor data processing.

The embARC MLI software library is available through its dedicated website that provides software developers centralised access to free and open source software, drivers, operating systems, and middleware supporting ARC processors.

"To provide our customers with an ultra-low power AI solution for voice triggering and recognition, we need power- and area-efficient processor IP like ARC EM DSP processors," said Albert Liu, Founder and Chief Executive Officer at Kneron. "By offering the embARC Machine Learning Inference software library, Synopsys gives SoC developers the fundamental kernels needed to quickly implement machine learning algorithms on ARC-based designs."

The embARC MLI software library supports ARC EMxD and HS4xD processors and provides a set of essential kernels for effective inference of small- or mid-sized machine learning models. It enables the efficient implementation of operations such as convolutions, long short-term memory (LSTM) cells, pooling, activation functions such as rectified linear units (ReLU), and data routing operations, including padding, transposing, and concatenation, while reducing power and memory footprint. 

As an example, low-power neural network benchmarks such as CIFAR-10 running on an ARC EM9D processor can achieve up to a 4X reduction in cycle count compared to competitive processors in the same class. Additionally, the MLI library provides a performance improvement across a wide range of neural network layers, such as depth-wise 2D convolution, fully connected, basic RNN cells, and LSTM cells with a maximum performance boost of up to 16X for 2D convolution layers.

John Koeter, Vice President of Marketing for IP at Synopsys, added: "Power consumption and area are critical considerations for embedded machine learning functionality in edge devices. By enabling broad classes of neural networks to run on power-efficient ARC EM and HS DSP processors, Synopsys is expanding the set of ARC processors that developers can choose to create their energy-efficient AI designs."