IBM’s brain-inspired chip for more efficient AI

AI is a constantly evolving environment and innovation is key to unlocking the true potential of AI systems. Conventional hardware, while powerful, was not originally designed with the huge demands of modern AI in mind. This limitation led to scalability issues and soaring costs which required a need for innovation … and what’s more innovative than the human brain?

And that’s what IBM researcher, Dharmendra Modha, thought when he created: NorthPole, a chip prototype developed by Modha and his team at IBMs Research's lab in Almaden, California, that has the potential to change the AI hardware industry.

Traditional structures no longer work

Traditional computer chips have followed a basic structure where processing units and memory are discrete components. However, when data continuously shuttles back and forth between memory, processing units, and other devices within the chip, it creates a bottleneck which hinders progress. The constant movement of data consumes time and energy, which limits the efficiency of AI processing systems.

So, Modha and his team at IBM Research embarked on a journey to create a chip that mimics the efficiency of the human brain because it is an energy-efficient processor with memory and processing functions interwoven – a departure from the traditional von Neumann architecture – laying a new path in AI hardware development.

A chip that stands out

The chip showcased its remarkable energy and space efficiency, and reduced latency in comparison to other chips on the market in a recent publication of Science, demonstrating significant improvements over conventional chips.

The key to NorthPole's efficiency lies in its design. Unlike conventional chips, which rely on separate memory components, NorthPole integrates all its memory on the chip itself. This design eliminates the traditional von Neumann bottleneck and allows the chip to perform AI inferencing much faster than its counterparts.

With a fabrication process of 12-nm node technology, NorthPole boasts 22 billion transistors in an 800 square millimetre chip. It features 256 cores capable of performing 2,048 operations per core per cycle at 8-bit precision. Moreover, the chip has the potential to double and quadruple the number of operations with 4-bit and 2-bit precision, respectively. This comprehensive architecture transforms NorthPole into an entire network on a single chip.

Bridging compute and memory

At the individual core level, NorthPole’s architecture appears as memory-near-compute, while at the input-output level, it appears as an active memory. This design greatly simplifies the chip's integration into systems and reduces the load on the host machine.

When NorthPoles energy efficiency was benchmarked using the ResNet-50 model, it outperformed common 12-nm GPUs and 14-nm CPUs by 25 times, in terms of the number of frames interpreted per joule of power required. The chip's improved latency and space efficiency standout when compared to chips using more advanced technology processes, such as a 4-nm GPU. NorthPole's efficiency not only translates into a reduced need for power-hungry cooling systems, but it also enables its deployment in relatively small spaces.

Applications and future prospects

While NorthPole excels in various AI tasks, it's not a one-size-fits-all solution. It may not be suitable for running extremely large models like GPT-4, but it's a perfect fit for many enterprise-level AI models. The chip is tailored for inferencing, making it a game-changer for applications like computer vision, natural language processing, and speech recognition.

The potential applications of NorthPole are vast and varied. It's well-suited for edge computing, opening doors for real-time processing in autonomous vehicles, robotics, digital assistants, and spatial computing. NorthPole could modernise industries by enabling tasks such as monitoring agriculture, managing wildlife populations, ensuring safer road transportation, operating robots securely, and detecting cyber threats.

A significant step in AI

However, NorthPole still has room for improvement in terms of efficiency and performance, and this is just the beginning of a journey that promises even more breakthroughs in AI hardware.

As AI continues to evolve, NorthPole is well positioned to be at the head of the transformation – unlocking new possibilities for the future of AI.