Besides data scientists, few companies were seriously implementing ML technology in their products, and therefore the benefits of edge computing were only qualitatively recognised.
These benefits of edge computing that we refer to include energy savings and latency reduction, where the cost of cloud processing, storage, and bandwidth preclude sending data to the cloud to make AI-enabled decisions.
Furthermore, edge computing can also yield a better user experience by supporting fast, AI-enabled decisions. Privacy and security concerns will limit what data users send and store in the cloud.
Today we can quantitatively recognise these benefits, after witnessing deployment on real systems and understanding the specific application requirements.
Counting Electrons
Energy consumption and performance are the most understood metrics for recognising the benefits of edge computing. Compared to even just the cost of transmitting data to the cloud, a lot more work can be accomplished in a given amount of energy in the form of computations at the edge.
Let’s put this into perspective using MobileNet as an example (more about MobileNet later), a popular neural network for object classification (keeping in mind that this is an oversimplification to avoid pages of analysis).
One version of MobileNet, specifically MobileNet_v1_1.0_224, requires approximately 559 million multiply-accumulates (MACs) and has an input image size of 50,176 pixels (224x224). Being conservative and considering 10 MACs on a GPU costs 100 pJoules, then 559 MMACs would consume approximately 0.001 Joules. Compare that to the image transmission cost on LTE of about 0.02 Joules (assuming 8 bits/pixel and an uncompressed image). In other words, there is an order of magnitude more energy required to transmit rather than process the image – and then once in the cloud, the inference on the neural network model would still have to be performed (on significantly more expensive equipment).
Benefits to society
For many of our customers, ML is new territory. We often get asked the question ‘does ML apply to my application’? And, of course, our answer is ‘it depends’. There are some basic prerequisites, such as a physical interface (e.g. camera, microphone, sensors), the ability to collect and label data (for model training) and determining whether the ML really adds value to the application (i.e. the monetary incentive).
Given these prerequisites are met, we are observing a very wide variety of applications that can incorporate some type of ML, the most interesting of which are those that benefit society or the user’s experience.
One area of interest, especially as awareness is increasing on climate awareness, is the use of ML in agriculture to monitor and control water conservation, targeted application of pesticides, integration of drones into crop analysis – all to increase productivity and minimise the global impact of food production. Machine learning can also be used to enhance industrial productivity for applications such as food quality sorting with visual inspection, predictive maintenance with anomaly detection, and increase safety for equipment operators.
Another area of very popular interest is in the use of ML for secure access, useful for applications ranging from home door access to commercial deployment protecting access to secure areas and blocking or allowing users access to heavy machinery operation. To augment this secure access, door-lock or user verification products are utilising two-factor access – simultaneously applying both face plus voice recognition or other biometric methods. Further enhancements to face recognition are possible using liveness detection to prevent spoofing attacks.
Public safety has become a grave concern in recent times and ML is coming to the rescue. For example, one application allows a security camera in an airport, subway platform, or other public places, to determine erratic behaviour patterns or detect if a person has abandoned a package or luggage. The system can respond appropriately if one of these conditions are met, for example, by notifying local authority of the incident and even exact location. Another camera-based application is counting cars at intersections, monitoring vehicular flow and adjusting traffic light timings – this serves both the convenience of drivers and can ultimately help improve driving efficiency.
The ML that can be implemented in an edge device is the result of tremendous advancements in both algorithms and processing solutions. However, this epoch really started expanding with the kick-off of the ImageNet Large Scale Visual Recognition Challenge in 2010. The goal of this competition was (and still is) to estimate the content of photographs containing one of 1000 possible object categories. This led to the birth of AlexNet, which is now somewhat archaic, but still used in benchmarks (although typically not used in practical applications because of its low accuracy factor). The evolution of AlexNet led to newer classification model topologies such as VGG, ResNet, and Inception.
As I referenced earlier, MobileNet is one of the more recent additions and has gained considerable popularity due its smaller size and reasonable accuracy. By the way, the accuracy of any of these models can be increased by using transfer learning to fine-tune for a specific class(es) of objects.
Along with great strides in algorithm development, as well as the training frameworks and inference engines to support them, comes the advances in processing power. While ML accelerators are becoming the norm, embedded system developers still strive to deploy ML in traditional devices, ranging from low-cost MCUs to highly-integrated application processors containing multiple CPU cores, GPUs, and DSPs.
It’s outrageous to think that there’s a $2 MCU-based solution for face recognition that can perform an inference in slightly more than 200msec with an accuracy up to 95%.
Scaling up in performance, as well as memory capacity, can expand the edge device’s ability to process more complex applications (as described above), and imagine what you could do with orders of magnitude more ML performance in an edge device.
Advancing Edge Compute
We believe that our industry is still far from realising the potential of ML. For that matter, most system developers working on industrial, consumer, IoT, and automotive applications, don’t yet comprehend how ML can enhance their products. But ML is not just a fad, it’s already proven its benefits to society and industry.
Going forward, we’ll see a growing number of processors with integrated ML acceleration. We’ll also see tremendous expansion of open source tools and libraries to support ML development. And we’ll be witness to great innovations in data analysis and distribution of collective knowledge – embracing ML technology at the edge.
Author details:
Geoff Lees, SVP and GM, Microcontrollers Business, Markus Levy, Director of AI and ML Technologies at NXP
