However, traditional edge AI deployment methods, mainly based on TensorFlow Lite, have been struggling to keep up with the pace of change in the market due to low adaptability and limited performance.
RooflineAI, which is a spin-off from the RWTH Aachen University, looks to address this with a software development kit (SDK) that offers much greater flexibility, better performance, and ease of use.
According to the company, models can be imported from any AI framework such as TensorFlow, PyTorch, or ONNX and the one-stop AI backend enables deployment across diverse hardware, covering CPUs, MPUs, MCUs, GPUs, and dedicated AI hardware accelerators – all with just a single line of Python.
“Our retargetable AI compiler technology is building on the shoulders of proven technologies to create massive synergies with the open-source community and chip vendors. This approach provides the flexible software infrastructure required to overcome technology fragmentation in the edge AI space,” said Roofline CEO Moritz Joseph. “Roofline enables faster to-market times for new architectures, improves the efficiency of existing solutions, and brings the user experience to the next level.”
AI compiler technology has become critical when it comes to deploying AI models at scale on edge devices. Currently, widely used solutions build on a legacy software stack that interprets AI models but does not allow for compilation, partly relying on handwritten and manually optimised kernels. This tends to limit the application of the technology for state-of-the-art AI models, such as language models, as they are not compatible with the existing technology stack.
AI compilation optimises model execution at different levels of abstraction – known as “intermediate representations” – that represent specific features of the execution of an AI workload. The compiler translates the AI model through various intermediate representations to a low level that is close to the target hardware.
Roofline uses AI compilation instead of model interpretation to allow users to adapt to the constant stream of new AI models. In addition, novel heterogeneous hardware platforms can be targeted as the compiler generates code for each component.
The AI compiler space is driven by open-source technologies that allow the exploitation of synergies beyond the scope of individual players in the market. Roofline is active in the open-source community and committed to providing code that drives innovation from cutting-edge AI models to novel chips.
