The SoC provides on-chip decode of 12x simultaneous 1080p30 video streams, while concurrently processing that video and running a hybrid of multiple, multimodal vision-language models (VLMs) and traditional CNNs.
The chip’s high AI processing performance supports most of the popular multimodal VLMs and large-language models (LLMs), while consuming only 20 watts of power, which is10-100x lower than cloud processors.
For example, the N1-655 reliably runs the Phi, Gemma, LLaVA-OneVision and Llama models, without the need for an internet connection, on data inputs like visuals and speech in a variety of applications.
Following its initial N1 SoC introduced last year, Ambarella said that it was building up a family of edge GenAI SoCs for tasks that go beyond what can be done on-camera. Both of the current family members process GenAI models locally to improve privacy with state-of-the-art performance per watt, significantly reducing power consumption as well as the total cost of ownership compared to cloud-based inference processing.
“The N1-655 continues Ambarella’s industry-leading levels of AI performance per watt, this time for AMRs and on-premises AI box applications that require the latest edge GenAI features while keeping power consumption under 20 watts,” said Fermi Wang, President and CEO of Ambarella. “We tailored this new SoC to run faster, while fine-tuning the most popular LLMs and VLMs to make them even smaller, eliminating the need for high-bandwidth and cost-intensive cloud processing in this class of applications.”
There is a trend toward doing more GenAI processing at the edge, due to cloud-based processing’s bandwidth cost, ongoing service fees and high-power usage, along with data security and privacy concerns.
Ambarella employs cutting-edge fine-tuning techniques such as QLoRA, which reduces the footprint sizes of popular GenAI models to enable their use in edge applications.
To support N1-655 based designs, Ambarella is also expanding its Cooper Developer Platform with the new Cooper Pro member of the Cooper developer kit family, which provides industrial-class performance. This kit is powered by the N1-655 and will begin shipping shortly.
