CEA-Leti barn-owl inspired object localisation system

Presented in a paper published recently in Nature Communications, the research team describes development of an auditory-processing system that increases energy efficiency by up to five orders of magnitude compared to conventional localisation systems.

“Real-world sensory-processing applications require compact, low-latency, and low-power computing systems,” the paper, ‘Neuromorphic Object Localisation Using Resistive Memories and Ultrasonic Transducers’, explains. “Enabled by their in-memory, event-driven computing abilities, hybrid memristive-complementary metal-oxide semiconductor (CMOS) neuromorphic architectures provide an ideal hardware substrate for such tasks.”

Neurobiology offers a spectrum of ultralow-power solutions to efficiently process sensory information, as different animals and insects have evolved to effectively perform difficult tasks with limited power. At the heart of biological signal processing are two fundamental concepts: event-driven sensing and analogue in-memory computing.

“We drew inspiration from biology to incorporate these two aspects of computation into our hardware, leveraging CEA-Leti’s state-of-the-art ultrasound sensors and resistive memory technologies,” said Elisa Vianello, senior scientist and Edge AI program coordinator, and senior author of the paper. 'In particular, we focused on the acoustic-based, object-localisation task. Owls efficiently solve this problem and thus we extrapolated their computational principles into our system.”

CEA-Leti built and tested this object localisation system with the help of CEA-List, University of Zurich, University of Tours and University of Udine researchers. The team leveraged CEA-Leti’s successes in developing piezoelectric micromachined ultrasound transducer (pMUT) sensors and its advancements in spiking neural networks based on resistive memory technologies.

The researchers’ first challenge was developing a pre-processing pipeline that extracts the key information from pMUTs, which encode information based on brief events or spikes. This temporal signal coding leads to higher energy-efficiencies compared to traditional continuous analogue or digital data, so that only relevant data are processed.

Bio-inspired analogue RRAM-based circuit

“Our system, which could have future use in sensor-fusion applications, mimics the owl’s extremely energy efficient prey-capture mechanism, which is preceded by combined auditory and visual search,” said Filippo Moro, lead author of the paper. “The ultralow power consumption auditory search is always active and when a specific auditory neuron fires, the owl has the information it needs to start the visual search, which is more precise but more costly in terms of energy consumption.”

The second challenge was designing and fabricating a bio-inspired analogue RRAM-based circuit to efficiently process the extracted events and estimate an object’s location. Resistive memory provides a compact solution to store the synaptic weights and RRAMs are non-volatile devices, a feature that matches the asynchronous event-driven nature of the team’s proposed system, resulting in no power consumption when the system is idle.

“To minimise the energy consumption of the object localisation system, researchers envisioned, designed, and implemented an efficient RRAM-based neuromorphic circuit that processes signal information produced by embedded sensors to calculate a targeted object’s position in real time,” the paper reports. “Whereas conventional processing techniques would continuously sample the detected signal and crunch calculations to extract the useful information, the proposed neuromorphic solution computes asynchronously as the useful information arrives: this has allowed us to increase the system’s energy efficiency by up to five orders of magnitude.”

Over the past decade, CEA-Leti has made substantial progress in pMUT sensors and spiking neural networks based on resistive memory technologies. The current work shows that combining visual sensors, such as DVS cameras, and the proposed pMUT-based audition sensor should be explored to develop future consumer robotics.

In addition, mimicking a barn owl’s precise and efficient object-localisation system is another example of the institute’s work to prove that bio-inspired concepts can dramatically improve performance of Edge-AI systems. In March, Vianello received a €3m grant from the European Research Council (ERC) to build nanoscale memory devices inspired by insect nervous systems for such applications as consumer robotics, implantable medical diagnostic microchips and wearable electronics.