Transforming Edge AI performance with Winbond's CUBE technology

It significantly boosts memory-interface bandwidth, providing remarkable power efficiency, exceptional performance, compact size, and cost-effectiveness. This technology significantly enhances system capabilities, response times, and energy efficiency, catering to a diverse spectrum of sectors, including consumer, industrial, financial, healthcare, and governmental activities.

CUBE improves the performance of front-end 3D structures such as chip-on-wafer (CoW) and wafer-on-wafer (WoW), as well as back-end 2.5D/3D advanced packaging and fan-out solutions. Designed to meet the growing demands of edge AI computing devices, CUBE is compatible with memory densities ranging from 1Gb to 8Gb per single die by 20nm process. Additionally, it can be 3D stacked to achieve densities of up to 8GB and boasts exceptional bandwidth capabilities, ranging from 256GB/s to a staggering 1TB/s. CUBE offers reduced power consumption during data transfer, making it an ideal solution for powerful edge AI applications.

Meeting the demands of generative AI and large models

The surge in generative AI adoption highlights the need for cutting-edge memory solutions. Generative AI models, known for their complex architectures and large model sizes, place heavy demands on system resources. These models require substantial memory bandwidth for rapid data access and intensive computations to generate responses and create content. Consequently, processing resources are strained, potentially affecting overall system performance. Additionally, the need to store and manipulate large model weights further amplifies memory usage.

Broadly speaking, various AI applications rely on large models to manage intricate patterns and relationships within their domains, increasing computational demands and resource requirements. For example, computer vision applications use convolutional neural networks (CNNs) for image recognition, employing large models to learn detailed patterns in visual data. Natural language processing (NLP) models handle tasks such as sentiment analysis and speech recognition, using deep neural networks with substantial parameters to enhance accuracy. Reinforcement learning applications use neural networks to represent complex policies or value functions, and recommendation systems in streaming services and e-commerce platforms employ large architectures to analyse user preferences.

Limitations of existing memory technologies

Current memory solutions face several limitations that impact their effectiveness for AI applications, particularly in Edge computing scenarios.

These limitations include:

• Bandwidth constraints: Traditional memory solutions struggle to provide the necessary bandwidth for AI applications. Factors such as the number of IC pins, data transfer rate, and memory bus width play a crucial role in determining interface bandwidth.
• Power efficiency: Increasing bandwidth often results in higher power consumption, which can introduce thermal management challenges and compromise the operation of battery-powered edge devices.
• Form factor: Existing solutions may contribute to larger form factors, limiting their suitability for compact devices.
• Signal integrity: At higher speeds, signal integrity issues such as attenuation, crosstalk, and reflections can limit achievable bandwidth.
• Access latency: Off-chip memory introduces significant access latency compared to on-chip SRAM, making it unsuitable for L1 and L2 cache. While 3D stacked SRAM offers high density for L3 cache, it may have slightly higher latency compared to traditional planar SRAM.

As AI workloads continue to grow, these constraints are likely to become more significant, necessitating more powerful and energy-efficient memory solutions.

CUBE: The high-bandwidth, power-efficient solution

CUBE technology addresses the limitations of conventional memory solutions through several innovative approaches:

• High I/O count and data speed: CUBE increases I/O count and raises data speed, utilising Through-Silicon Via (TSV) technology as an option. Its 3D architecture reduces thermal dissipation issues, enabling higher performance and power efficiency.
• Advanced 3D structures: CUBE enhances the performance of front-end 3D structures such as chip-on-wafer (CoW) and wafer-on-wafer (WoW), as well as back-end 2.5D/3D chip-on-Si-interposer-on-substrate and fan-out solutions. This advanced packaging facilitates higher bandwidth and improved thermal management.
• Compact and customisable design: CUBE's 3D stacking options and compact size make it ideal for portable and space-constrained devices. Its flexible design allows customisation to meet specific requirements, providing tailored solutions for various applications.
• Power efficiency: CUBE consumes less than 1pJ/bit, making it particularly well-suited for energy-sensitive applications. The integration of TSVs improves power delivery and signal integrity, contributing to overall system efficiency.
• Memory densities: CUBE can be designed with densities ranging from 1-8Gb/die based on the D20 process, or 16Gb/die on the D16 process. This flexibility enables optimisation of memory bandwidth for various applications.
• High data rates: CUBE's IO interface supports a data rate of 2Gbps with 1k – 4K I/O, providing total bandwidth ranging from 256GB/s to 1TB/s per die. This ensures accelerated performance that exceeds industry standards.

Shaping the future of AI-driven technologies

Winbond's CUBE technology is positioned to shape the future of AI-driven technologies. By providing high bandwidth, low power consumption, and advanced 3D architecture, CUBE facilitates efficient data transfer and enhanced system performance. This makes it pivotal for deploying powerful AI models across different platforms and use cases, including edge devices and hybrid edge/cloud scenarios.

CUBE's integration into the designs of chip makers, module makers, and system builders ensures that edge AI devices can handle the increasing demands of AI applications. Winbond's collaboration with partners such as IP design houses, foundries, and OSATs fosters a comprehensive ecosystem that supports the development and deployment of advanced AI solutions.

With its focus on power efficiency, high performance, and flexible design, CUBE is ready to unlock the full potential of AI technologies, making advanced AI applications more accessible and efficient.