Roofline enables agile adaptation to new Edge AI models and hardware

Traditional Edge AI deployment methods, mainly based on TensorFlow Lite, can’t keep up with the pace. Low adaptability, limited performance, and a painful user experience make them barriers to adoption for Edge AI.

RooflineAI, a spin-off from the RWTH Aachen University, revolutionises this process with a software development kit (SDK) that offers unmatched flexibility, top performance, and ease of use. Models can be imported from any AI framework such as TensorFlow, PyTorch, or ONNX. The one-stop AI backend enables deployment across diverse hardware, covering CPUs, MPUs, MCUs, GPUs, and dedicated AI hardware accelerators – all with just a single line of Python.

“Our retargetable AI compiler technology is building on the shoulders of proven technologies to create massive synergies with the open-source community and chip vendors. This approach provides the flexible software infrastructure required to overcome technology fragmentation in the edge AI space,” says Roofline CEO Moritz Joseph. “Roofline enables faster to-market times for new architectures, improves the efficiency of existing solutions, and brings the user experience to the next level.”

The compiler is key

AI compiler technology has become mission critical for deploying AI models at scale on Edge devices. Today, widely used solutions build on a legacy software stack that interprets AI models but does not allow for compilation, partly relying on handwritten and manually optimised kernels. This limits the application of the technology for state-of-the-art AI models, such as language models, as they are not compatible with the existing technology stack.

AI compilation optimises model execution at different levels of abstraction – known as “intermediate representations” – that represent specific features of the execution of an AI workload. The compiler translates the AI model through various intermediate representations to a low level that is close to the target hardware.

Using AI compilation instead of model interpretation allows users to adapt to the constant stream of new AI models. In addition, novel heterogeneous hardware platforms can be targeted as the compiler generates code for each component.

The AI compiler space is driven by open-source technologies that allow the exploitation of synergies beyond the scope of individual players in the market. Roofline is active in the open-source community and committed to providing code that drives innovation from cutting-edge AI models to novel chips.