Performance boost for AI inference in the low-power range

By florian Drittenthaler, Product Line Manager, congatec

Intel has once again boosted the performance of its low-power processors, codenamed Amston Lake, delivering improvements of 49% in single-threaded and 61% in multi-threaded applications. This performance boost aligns with expectations given the increased clock rate and number of cores, allowing for
significant progress within the same power budget. However, the real breakthrough for embedded and Edge computers is the up to 9.83-fold increase in image classification performance of the new Intel Core and Intel Atom processors. Such powerful AI performance was previously limited to systems with significantly higher thermal design power (TDP). The standardisation of the underlying microar- chitecture (codenamed Gracemont) for the Efficient-cores (E-cores) across all

Intel client processor designs – from the smallest Intel Atom to the most powerful Intel Core i9 – brings x86 innovations such as Vector Neural Network Instructions (VNNI) for artificial intelligence to the low-power segment up to 15 watts. As a result, small form factor designs based
on SMARC modules are now entering performance territories previously exclusive to larger footprints such as COM Express or COM-HPC.

Developing smarter systems

High-performance AI inference logic can now be utilised in many innovative, energy-efficient, and robust industrial applications where AI-based intelligence is crucial. In industrial production, object detection and recognition play an increasingly important role in pick-and-place operations and testing and measurement tasks. In the latter case, vision-based solutions are replacing complex mechanical processes, helping to reduce quality assurance costs. Collaborative robots, autonomous guided vehicles (AGVs), and service robots also require advanced image processing capabilities to perceive their environment comprehensively and precisely in real-time, and to ensure operational safety as well as efficient route planning and navigation.

Powerful image segmentation also enhances automated anomaly detection in medical technology, such as mobile ultrasound devices, accelerating diagnosis. In smart city surveillance and security systems, improved image classification enables not only faster detection of moving objects, but also
more nuanced object categorisation based on multiple criteria. This capability also benefits point-of-sale (POS) applications, like fully automated vision-based checkout terminals, which replace traditional barcode scanner systems and reduce labour costs. These and countless other applications, including in-vehicle systems deployed in the construction, agriculture, and forestry sectors, as well as classic human-machine interfaces (HMIs), greatly benefit from the AI performance boost now provided by credit-card-sized SMARC modules.

Comprehensive x86 ecosystem for inference logic

The hardware performance increase can be swiftly implemented in real applications. Developers in higher performance classes have long used Intel’s AI inference logic solutions. Porting such inference logic to an extremely compact footprint with low power consumption and industrial temperature range (-40 to +85°C) design now requires significantly less software development effort. This is because the CPU’s up to eight E-cores and the Intel Gen 12 UHD GPU’s up to 32 execution units are identical to those in higher performance class processors based on the Gracemont microarchitecture, also supporting INT8 deep learning inference processing. Other AI technologies like optimised Intel AVX2 (Advanced Vector Extensions 2) and Intel VNNI instruction sets can also be used uniformly across platforms. Moreover, a complete ecosystem is available, from kernels for library developers and libraries for data-level specialists to toolkits such as OpenVino for application developers. This enables low-power application developers to achieve goals faster by leveraging the harmonised underlying processor microar- chitecture. However, fast AI inference logic isn’t the only advantage of the new SMARC modules.

Eight cores expand virtualisation options

The new SMARC modules based on the Intel Core i3 processor, and the Intel Atom x7000RE processor series (codenamed Amston Lake) offer up to eight cores, doubling the previous generation’s processor cores while maintaining power consumption. This performance increase can be integrated into existing system designs without revising the cooling layout, accelerating both the integration of this new performance generation and the introduction of AI in 24/7 industrial applications requiring particularly robust design. The eight cores also provide attractive virtualisation technology options.

The advantages of virtualisation are manifold:

• It offers significant cyber security benefits (aligning with the Cyber Resilience Act) by reducing the risk of cross-system security breaches
• Communication gateways, controllers, and user interfaces can operate in separate, independent instances, increasing cyber threat resilience
• By consolidating several systems on a single piece of physical hardware, companies can save on hardware costs and increase efficiency
• Code base complexity is minimised, while application scalability and portability are improved by dividing individual tasks granularly into separate software products and operating them in independent virtual machines instead of monolithic software

In the Industrial Internet of Things (IIoT) field, virtualisation also enables creating dedicated virtual machines for maintenance, increasing system availability, and simplifying updates. Additionally, developers can work in a controlled, reproducible environment, accelerating software development and improving end product quality.

SMARC modules with Hypervisor-on-Module technology

congatec is incorporating a hypervisor directly into the firmware of its SMARC modules. This virtualisation-ready feature significantly simplifies virtual machine implementation for developers. It also enables the design of more reliable, cost-effective, and durable solutions, thus reducing the total cost of ownership
(TCO). An additional benefit for industrial applications is the integrated hypervisor’s real-time capability.

The application of this Hypervisor-on-Mod- ule technology is not limited to systems requiring real-time functionality or those integrating Intel Time Coordinated Computing (Intel TCC) and Time-Sensitive Networking (TSN). Developers can use this hypervisor to manage all required functions, including those not demanding real-time performance. Compared to containerisa- tion, this approach achieves a significantly higher level of cyber security, as all virtual machines operate as completely separate security islands without sharing a host OS.

congatec also offers an aReady.COM version of the conga-SA8 SMARC module. Customers can purchase individually configured modules, for example, with Bosch Rexroth’s ctrlX OS and virtual machines for tasks such as real-time control, HMI, AI, IIoT data exchange, firewall, and maintenance/management functions, all ready for immediate deployment. Naturally, the new SMARC modules also support multiple variants of Microsoft Windows. This comprehensive ecosystem greatly facilitates the development of new, ener- gy-efficient embedded and Edge computing system solutions leveraging AI inference logic and virtualisation technologies. With numerous essential functions already provided out-of-the-box, developers can concentrate on creating the distinctive aspects of their solution.

Wireless real-time with TSN over Wi-Fi 6E

The conga-SA8 is one of the first SMARC modules to support Wi-Fi 6E, including for extended temperature ranges. Compared to Wi-Fi 5 products, it offers nearly triple the data rate and more stable connections in dense or overloaded environments. It is also prepared for the upcoming TSN standard via Wi-Fi, enabling deterministic wireless connections with defined throughput. This provides a cost-effective alternative to private 5G networks or new Ethernet cabling. Additional industrial features of the conga-SA8 SMARC module include in-band ECC for increased data security and soldered DRAM for enhanced resilience in harsh environments.

This article originally appeared in the October'24 magazine issue of Electronic Specifier Design – see ES's Magazine Archives for more featured publications.