Latest server processors get embedded support

Now that AMD has extended such support to its EPYC 3000, 7002 and 7003 processor series, the company covers the complete spectrum of demanding edge computing applications with scalable hardware that is future-proof and long-term available – two qualities that are particularly important for OT.

More enterprises are decentralising their computing to move it closer to the point of action. This process, known as edge computing, reduces network latency, avoids overloads and improves application performance. However, depending on the use case and device location, the requirements of edge servers in OT applications are more individual and higher than those of server IT hardware in a fully air-conditioned cloud data centre.

Planning reliability for edge server designs

Systems that are installed on a factory production line, or outdoors where there is no air conditioning, must be more resistant to shocks and vibrations and able to operate in industrial temperature ranges. When edge servers are deployed in the medical or the test and measurement instrumentation sector, compliance with EMC/ERP directives may play a role, while critical infrastructures such as energy and water supply or the transport and traffic sectors often require additional industry-specific certifications for functional safety besides a custom design to provide fail safety and real-time capability.

Developers looking to design and deploy such future-proof OT servers need server processors that are available as embedded SKUs with an extended availability of at least five years (or longer) compared to standard IT components. This is to allow for the extra time needed to develop and certify such customised systems, and because the rollout also often takes longer. OEMs don’t want to replace system components within a product generation as this would mean starting the development and certification efforts all over again. Maintenance, too, is more complex when different systems are in the field. But apart from these requirements, there is no significant difference between OT and IT servers.

Application-specific embedded CPUs

Edge data centres have an important need for a generous selection of I/O interfaces to connect GPGPUs for artificial intelligence and machine learning using neural networks, fast solid-state storage via NVMe and fast network connectivity. The storage sector is following the same trend. Software-defined storage solutions are deployed to move SAN storage clusters and their connected NAS systems to the network edge, for instance to support critical applications and data locally and to minimise downtimes. Running as virtual SANs on multiple virtual machines (VMs), they enable customised storage solutions that can provide scalable data and application security across multiple physical locations. However, this also increases security requirements.

Embedded security

This is one of the reasons why high-performance applications and server systems deployed at the edge of the cloud require hardware-integrated security features, including the capability to individually encrypt multiple VMs in the processor and their dedicated RAM. Secure migration of VMs and containers between dispersed systems is another increasingly important factor in modern network and storage infrastructures. When adding or expanding a vSAN cluster, it then becomes possible to migrate VMs securely to additional physical locations without interrupting running services.

The transition between ensuring data security and continuity of the services that manage and process this data is fluid and must therefore be adapted to the needs of the customer. This is one of the main reasons why system integrators are calling for long-term available CPU components that are as design-compatible as possible across a processor series so that a single design can meet the requirements of varying performance classes and price levels.

The Zen microarchitecture

AMD meets these demands with its AMD Embedded EPYC processor families. The entire EPYC series is based on the Zen microarchitecture, which AMD first introduced in 2017. Using energy-efficient FinFET transistor technology, it offers more performance per watt and footprint. Along with the Zen microarchitecture, AMD also introduced the so-called Infinity Fabric, which integrates all components on a system-on-chip for lowest latency and high bandwidth. Specifically, this includes the CPU cores, the memory controller as well as I/O and system hubs – for example for PCIe and Ethernet – and graphics cores. However, the latter are not present in the EPYC processors.

This integration is not only available on one die, but also across multiple CPU dies. Infinity Fabric further controls the interconnect between CPUs in dual-socket operation. This high-bandwidth interconnect, together with the modular CPU architecture, is behind the high scalability of all AMD processors with Zen microarchitecture.

The new Zen microarchitecture also introduces so-called core complexes (CCXs) to group several CPU cores together. Each core within a CCX has the same low-latency access to 512KB L2 cache per core and 8MB shared L3 cache. Next, multiple CCXs are combined into a core complex die (CCD), or chiplet. Several of these chiplets are connected via the Infinity Fabric to make up the actual processor, which is designed as a multi-chip module (MCM).

EPYC Embedded 3000 – ultra-robust and long-term available

The models of the AMD EPYC Embedded 3000 processor series offer up to 16 CPU cores for 32 threads in this structure, with support for up to 1 TB DDR4-2666MHz RAM over 4 memory channels. 64 PCIe 3.0 lanes provide high I/O connectivity. In total, AMD offers 7 model variants with different core counts (4, 8, 12 or 16 cores), CPU clock speeds (2.1 to 3.0GHz maximum boost) and power consumption (30 to 100 Watt TDP). They further integrate 8 x 10Gb Ethernet and 16x SATA ports. All variants are fully pin compatible and BGA soldered onto the boards. This makes the designs extremely robust, predestining them for use in extremely harsh environments. They can even be used in the extended temperature range of -40°C to 85°C. The fact that the AMD processors are offered with an extended long-term availability of up to 10 years makes them an ideal platform for compact and highly robust telco and industrial edge servers. Hence, they are also supported on a variety of different board form factors, including µ-ITX, Mini-ITX and COM-Express Type 7 Server-on-Modules.

AMD also introduced secure encryption virtualization (SEV) in the EPYC Embedded 3000 CPUs for the first time. SEV makes it possible to encrypt VMs at the hardware level and to isolate them from the hypervisor. A separate security processor provides an individual AES 128-bit key for the VM that needs securing. That key also encrypts all user data in the VM’s RAM (secure memory encryption) to prevent that such data can be read via the hypervisor.

EPYC 7002 processor family with Zen 2 microarchitecture

Based on the Zen 2 microarchitecture, EPYC 7002 processors (code name Rome) implement a central I/O die that significantly simplifies CCD off-chip communication compared to the first EPYC generation. Since the EPYC 7002 CCDs are manufactured in 7nm process technology, they offer up to 64 cores. While the Zen 1 design had four CCDs, the number of multi-core blocks in Rome has been increased to eight. In addition, the processor-integrated L3 cache has quadrupled in size from 64MByte in Zen 1 to up to 256MByte today; cache is also distributed more efficiently across the cores. The up to four cores within a CCX now share 16MByte L3 cache instead of 8MByte per CCX as before.

The number of PCIe lanes on the I/O die has not changed, but the 128 lanes are now connected via fourth generation PCIe and the eight memory channels now support up to 4 TB DDR-4 RAM with a faster clock rate of up to 3200MHz. This results in major performance improvements between the first and the second AMD EPYC Embedded generation and significantly wider scalability for memory-intensive workloads.

Developers can choose between 19 model specifications, which differ in the number of cores (8, 12, 16, 24, 32 or 64 cores) and the clock frequency. 14 out of the 19 model variants (those without the suffix P) can also be operated as dual-socket solutions with up to 128 (!) cores. The TDP of the AMD EPYC 7002 family ranges from 85 to 280 Watt. AMD offers 9 processor variants with an extended long-term availability of 5 years. This gives embedded edge server designs access to the same outstanding performance that is otherwise only found in traditional data centers with shorter upgrade cycles.

Table: The powerful AMD Embedded EPYC 7002 processors with a long-term availability of 5 years come in 9 variants that scale from 8 cores to a maximum of 64 cores with 128 threads.

AMD has also upped the security. For instance, processors of the AMD EPYC 7002 series can activate an additional module to encrypt the register that the hypervisor assigns to a VM. This feature prevents disclosure of CPU register information to other software components, such as the hypervisor. This means that even if the hypervisor itself is compromised, it cannot access VM memory contents, since the GHCB that sits between the two only shares encrypted contents in response to requests. This provides additional security for the use of embedded systems at the edge.

Third generation EPYC Embedded (Zen 3)

Next, there are the brand-new AMD EYPC 7003 CPUs (code name Milan). They introduce the new Zen 3 microarchitecture, which also uses the ultra-energy-efficient 7nm process technology and retains the basic design with CCX, CCD and multi-chip module. With 19% higher IPC than Zen 2, the new Zen 3 cores can execute more instructions per clock, which is partly due to a higher number of CPU cores per CCX, i.e., 8 instead of the previous 4. This doubles the available L3 cache in the CCX from 16MB (Zen 2) to 32MB. At the same time, AMD has accelerated the Infinity Fabric clock to 1,600MHz. This allows synchronous connection of the faster DDR4-3200MHz memory, which results in even lower latency.

AMD has also expanded the security features with secure nested paging. This function enables the processors to cryptographically isolate and secure more than 500 VMs per server, especially as the necessary computing power is now also available. This addresses the growing security demands at the network edges and in integrated solutions.

Table: This tabular overview compares various AMD EPYC processor series, revealing hardware differences and similarities.

Figure 1: Unlike the four-die design of the EPYC 7001 Series (left) with up to 32 cores, AMD EPYC 7002 processors (right) feature up to 64 cores – divided into 8 blocks of 8 cores each – which results in higher computing performance and improved cache management.

Figure 2: Secure encrypted virtualization (SEV) is used to encode each virtual machine (VM) with a separate AES-128 key that is only known to the hardware-isolated secure processor. Originally introduced for Zen 1 CPUs, this security technology supports the encryption of more than 500 VMs in the newer Zen 2 and Zen 3 EPYC generations.