The AI-In-A-Box module can answer queries and solve problems in a similar way to well-known AI tools based on a large language model (LLM) but by using compression and acceleration technologies developed by Useful Sensors, it is able to host its LLM file locally, enabling its low-cost microprocessor to understand and respond instantly to spoken natural language queries or commands without reference to a data centre.
Disconnected from the internet, the module eliminates user concerns about privacy, snooping, or dependence on third-party cloud services that are prevalent with conventional LLM-based AI products and services marketed by large technology companies.
Commenting Pete Warden, founder and CEO of Useful Sensors and a former leader of AI programmes at Google, said, ‘The AI-In-A-Box module contains an advanced model that the user can talk to naturally, and with no network connection, account, or API calls. It offers the benefits of AI – an intuitive, natural way to control devices in the home, answer queries or just have a chat with a natural-language avatar - without having to share any of your conversations or other data with a Big Tech company or a cloud service provider. At home, every conversation with the box stays within the user’s four walls.’
The module offers similar natural language processing (NLP) performance to well-known LLM-based tools but while these tools require arrays of high-performance and high-cost GPUs hosted in a data centre, the AI-In-A-Box module contains a single, general-purpose Arm Cortex-A-class microprocessor running all AI operations.
This means that the module provides a ready-made, bolt-on hardware and software platform for OEMs and developers who want to provide an intuitive and natural voice-based user interface for any electronics product.
Because the software runs locally, the user avoids any need for account registration or system configuration, and privacy concerns have been eliminated.
Other capabilities of the module include real-time closed captions of nearby conversations, a potential benefit for users with hearing difficulties. Upgrades in development by Useful Sensors will add the ability to translate between English and other languages, and the ability to use the box as a voice keyboard (speech-to-text function).
The Useful Sensors compressed LLM file stored locally on the AI-In-A-Box module, and its other operating software, are supplied by Useful Sensors as open-source software under a GPL v3 license. Commercial license terms are available for OEMs.
