This processor has been designed to power edge servers and laptops with high performance, cost effective, and energy efficient inference capable of running applications such as video analytics, Large Language Models (LLMs), and other Generative AI models.
The Ara-2 is intended for edge applications that run traditional AI models and state-of-the-art AI models with transformer-based architectures. With an experientially enhanced feature set and more than 5-8 times the performance of its first-generation Ara-1 processor, the Ara-2 combines real-time responsiveness with high throughput, merging its latency optimised design with balanced on-chip memories and high off-chip bandwidth to execute very large models with extremely low latency.
LLMs and Generative AI in general have become incredibly popular, but most of the associated applications are running on GPUs in data centres and are burdened with high latency, high cost, and questionable privacy.
To overcome these limitations, the Ara-2 looks to simplify the transition to the edge with its support for the 10’s of billions of parameters used by these Generative AI models. Furthermore, to seamlessly facilitate the migration from expensive GPUs for a wide variety of AI models, the compute engines in Ara-2 and the associated software development kit (SDK), are specifically designed to support high-accuracy quantization, a dynamically moderated host runtime, and direct FP32 support.
“With Ara-2 added to our family of processors, we can better provide customers with performance and cost options to meet their requirements. For example, Ara-1 is the right solution for smart cameras as well as edge AI appliances with 2-8 video streams, whereas Ara-2 is strongly suited for handling 16-32+ video streams fed into edge servers, as well as laptops, and even high-end cameras,” said Ravi Annavajjhala, Kinara’s CEO. “The Ara-2 enables better object detection, recognition, and tracking by using its advanced compute engines to process higher resolution images more quickly and with significantly higher accuracy. And as an example of its capabilities for processing Generative AI models, Ara-2 can hit roughly 0.5 seconds per iteration for Stable Diffusion and tens of tokens/sec for LLaMA-7B.”
In October, Kinara was invited by Ampere to join the AI Platform Alliance which has the primary goal of reducing system complexity and promotes better collaboration and openness with AI solutions and ultimately delivering better total performance and increased power and cost efficiency than GPUs.
Commenting Ampere’s Chief Evangelist Sean Varley said, “The performance and feature set of Kinara’s Ara-2 is a step in the right direction to help us bring better AI alternatives to the industry than the GPU-based status quo.”
The Ara-2 also offers secure boot, encrypted memory access, and a secure host interface to enable enterprise AI deployments with even greater security.
Kinara also supports Ara-2 with a comprehensive SDK that includes a model compiler and compute-unit scheduler, flexible quantization options that include the integrated Kinara quantizer as well as support for pre-quantized PyTorch and TFLite models, a load balancer for multi-chip systems, and a dynamically moderated host runtime.
Ara-2 is available as a stand-alone device, a USB module, an M.2 module, and a PCIe card featuring multiple Ara-2’s. Kinara will show a live demo with Ara-2 at CES.
