Today, there are cameras integrated into drones, body cameras worn by public safety workers, and hundreds of millions of multi-camera smartphones capturing video on the go. And they can all - naturally - suffer from the shakes unless something’s done about it.
Watching shaky video for long periods of time can wear out a viewer and even induce nausea. Depending on the use case, important details could be missed. It’s one thing if video of a family reunion is a little shaky, especially as you zoom in on happy faces. While the technology exists to stabilize such video in the smartphone itself, there is certainly post-production software that can smooth out unwanted camera shakes. But it’s a whole other thing if important content, such as surveillance video from drones or incident footage from bodycams, shakes to the point it’s unreliable.
Imagine drones monitoring a rapidly spreading wildfire. Compiling video feeds from multiple drones can help create a more complete picture of the affected area and inform important decisions about evacuations or the deployment of fire-fighting resources. But because of the nature of drones, their video footage is particularly prone to shakiness and video noise. By addressing video stabilization in drones, solution providers can also add other important features, such as object tracking, smart zooming, or the ability to compose a single situational picture from several video feeds.
When it comes to public safety of surveillance, imagine a SWAT team entering a building and being guided by what a command centre “sees” from the body cameras each team member wears. Strong video stabilization and other features, such as lighting and lens correction, make it easier for command centre staff to perceive the situation from afar, identify danger faster, and guide the SWAT team to a positive outcome.
In short, great demand for video from cameras in motion yields demand for great, stable video. That’s where advanced video stabilization technology comes into play. To date, innovation in video stabilization have been driven largely by the smartphone industry. The technology runs in the background of most leading smartphones, capitalizing on multiple integrated cameras to create smooth, sharp, and stable video. Similar stabilization technology can also work in other mobile camera platforms.
For companies developing new camera-based products, there are three main ways of achieving the video stabilization that customers will demand: develop a stabilizer in-house, source greater stabilization through hardware, or integrate software-based video stabilization. Each has their pros and cons.
Path 1: Develop in-house
For ultimate control over the video stabilization function of a camera product, companies can do it themselves. This may allow them to more fully customise a product’s stabilization to match a very particular use case. But it’s important to understand that good stabilization requires extensive development resources, and that it’s not finished when the product ships.
Of the three paths for video stabilization, in-house development may require the greatest capex investment. It takes a team of developers and often years of work to design and produce video stabilization that will meet market expectations. Video stabilization development presents unique challenges. For instance, a common hurdle is being able to synchronize motion sensors with a video stream — a critical, resource-intensive step that will determine the effectiveness of the final solution. In addition, developing an in-house stabilization solution requires expertise on energy optimisation. A video stabilizer can be power hungry and if, for example, you’re creating a video-enable drone, inattention to energy consumption by the in-house stabilizer could cut significantly into flight times.
Plus, one of the potential benefits of in-house stabilizer development - namely, the ability to customise - an also prove a challenge. For instance, you may have to account for different types of camera systems, or some that are used for different purposes, which will mean the development team must calibrate specifically for each use case and integrate each type of camera system differently.
And when the product is finished, assuming it’s reached the level of stabilization quality required, in most cases it will be time to development the next-generation product. This is an area where technology advances quickly. A video stabilizer is not a one-time expense. The company will need to regularly upgrade, maintain, and improve it.
Path 2: Using hardware
Alternatively, a company could introduce advanced video stabilization to its mobile camera product through the hardware systems it chooses, including cameras that have greater capabilities built into their chipsets. In general, there are two types of hardware-based video stabilization solutions to consider: optical image stabilization (OIS), which involves mechanically keeping the camera or lens still to counteract shakiness, and hardware-integrated electronic image stabilization (EIS), which involves digitally correcting shakiness with algorithms built into the device’s chipset.
OIS can deliver highly effective stabilization for small movement, but it struggles with to overcome greater movement. Moreover, relying on OIS can make it difficult to integrate other features that improve video quality, such as noise reduction or object tracking.
EIS built into chipsets is similar to a software-based solution, but less flexible. EIS in a chipset is usually designed as a one-size-fits-all solution, often for the smartphone market. But of course, using a camera for drone monitoring is completely different from using it for a ‘selfie’ video.
Both types of hardware solution - OIS and EIS pre-integrated into hardware - can require a relatively large investment. If a company is looking to add video stabilization to an existing product line, depending on how many cameras and their age, it might not be practical to replace or upgrade them now. Another problem with relying on a hardware-based solution is that there is no such thing as an out-of-the-box stabilizer that works for use case. If you want to achieve outstanding video quality, you’ll need to calibrate the solution for each application. Hardware can be inflexible and difficult to adapt.
Path 3: Integrated software
Finally, advanced video stabilization exists that can be integrated into existing camera platforms. The key advantages to this path are that it doesn’t require a significant capex investment and that software is more easily upgradable, whether the manufacturer wants to add features or improve performance.
As important as stabilization is creating high-quality content, it’s usually just one of many features for improving video quality. Understanding what else a product might require is important. A device that will be deployed at night may need to compensate for low light. Another may need to exploit new artificial intelligence capabilities to track and identify objects. Such features can be quickly and easily added through integrated software.
Although purchasing software spares a company the effort and expense of developing stabilization from scratch, that software still needs to be professionally integrated and calibrated. Experienced software vendors not only support the widest variety of camera platforms, but they also know how to calibrate hardware and software to ensure the highest stabilization performance. In addition, they specialize in tuning their solutions, taking the product development process a step further to differentiate a solution. If calibration involves testing to see if a camera product’s software and hardware are in sync, tuning involves adjusting parameters to suit specific needs.
In the end, of the three paths to video stabilization in mobile camera products, software stabilization offers the best chance to future-proof a new product. We’ve arrived at a point in time when video is applicable to a wide variety of business solutions, and quality continues to improve. To keep up with innovation in the overlapping video, camera, smartphone and “camera in motion” markets, a software-based stabilization approach allows the device maker to push regular updates, add new features like AI, calibrate and tune for any use case, and support advances in hardware without being handcuffed to any one hardware platform.
The result is greater value from mobile video for the greatest universe of potential users.
Author details: Johan Svensson is CTO, IMINT Intelligence
