Extended reality (XR) has been creeping into modern life over many years and, according to Statista, the research group, the global XR market is forecast to be worth $31 billion in 2021 and to top $300 billion by 2024.
Extended reality (XR) is used as an umbrella term to describe a wide range of immersive and interactive technologies and covers augmented reality (AR), mixed reality (MR), and virtual reality (VR) and it’s possible to access XR through a variety of different platforms whether that’s mobile devices, headsets or glasses.
While it has been long associated with entertainment and gaming XR is now expanding into new industries and is being used in design creation, training and to improve the speed and quality of installing and maintaining facilities and systems.
In November 2021, leaders from some of the world’s biggest companies gathered at the Augmented World Expo (AWE) in Santa Clara, California, to discuss current and projected use cases for XR with the discussion focused on the transformational impact XR could have in terms of how teams would be able to collaborate, design and develop products, and better serve their customers.
RealWear is six years old and having come out of Silicon Valley is now based in Portland, Oregon. Operating in over 60 countries and with offices in four it has developed a number of assisted reality platforms and solutions and serves a broad range of industrial customers.
“While our focus is mainly on the industrial space, we are now seeing some traction from other sectors,” explains Sanjay Jhawar, Co-Founder and Chief Strategy Officer at RealWear. “Within this space of head worn devices there are several different categories. VR is fully immersive, MR sees users operating with a transparent display, objects can be superimposed on real view world, and then there is AR.
“At RealWear we have coined the concept of Assisted Reality which doesn’t attempt to be immersive but looks to keep the user focused on the real world while deploying a smaller field of view for digital data, so they are not taken out of the cognitive focus in what are often difficult and dangerous environments. That focus on the real world and below the line digital information is an important driver as to how the system works.”
According to Jhawar the user interface is critical when you are supporting workers who may be wearing PPE, such as goggles or face masks.
“These people will be operating in a variety of different and dangerous environments,” explains Jhawar. “They’ll be climbing buildings, ascending wind turbines, and descending into sewers via manholes, so they wont be able to rely on their hands to interact and drive their devices.”
When it comes to existing VR solutions, from MR to AR, most require users to enable the systems via gesture and so having to hold various devices or operate wearing gloves doesn’t make that easy or, in many cases, even possible.
“Gaze and dwell – in which a cursor moves in your field of view - can be used to select something or make commands, but again that takes the wearer’s attention off the real world. So, while it might be better than using your hands there are still safety issues,” says Jhawar.
Asa a result we have focused on the development of voice recognition.
“In the industrial space voice recognition has to be able to operate in a high noise environment, and that is a problem. Traditional speech recognition systems - Alexa, Google Hubs - can’t handle those types of environments, so we’ve created a system than can operate at up to 100 decibels and can understand 17 different languages.”
Hands free
The company’s WearHF (Wearable ‘Hands Free’) technology provides the underlying foundation for the interface upon which its various applications, and third-party ones, for that matter, are built.
“It’s a command driven voice system which is conceptually similar to the navigation commands you will enter into your car with a GPS system.
“Each field looks for a specific type of answer that reduces the vocabulary that you can say but you need to know which field is active at that moment and constraining the possible things it could recognise. Simply, we put it on a screen. All it requires is for the user to briefly glance at their screen which will contain all the voice commands that they can use with that screen. Basic navigation commands will be available on those screens, and they will change dynamically, but we recommend that those commands need to be distinct.
“That’s crucial because there’s a high probability that at 100 decibels they could be easily confused. The art here, is to pick a good voice command that will be distinct. We recommend the use of words with multiple syllables, such as ‘navigate home’ and ‘navigate back’, avoiding short single syllables,” says Jhawar.
“Customers can add commands and we provide parameters for them to work within. We can work with a variety of different apps and provide user training. It’s straightforward to write a speech driven app. If you know how to work with HTML then you’ll be able to create voice commands.”
And that’s proved to be the case as to date RealWear has over 1000 enterprise customers worldwide and has deployed some 65,000 of its RealWear devices.
“Among our deployed devices are the RealWear HMT-1, HMT1Z1 (Atex zone 1 version) and the RealWear Navigator 500 assisted reality devices. These allow companies to be more responsive when it comes to supervision and they have helped to improve the efficiency of teams, increasing their safety,” explains Jhawar
“Our RealWear devices also allow on-site workers to instantly contact remote experts to assist them, whether that’s on a daily basis or during an emergency intervention.
“When faced with situations that require a high level of expertise, the devices’ built-in camera means that operators can contact an expert or experts who can then assist them in making decisions while allowing the responder to move freely and use their hands.”
Previously, in those kinds of situations, the user would have to take pictures with a phone, email them to the experts and wait for their response - significantly lengthening inspection times.
“One sector that is growing significantly for us is automotive field services,” says Jhawar.
“Ford is one of our newest and biggest customers and they are using out technology to support their dealer network. We’re in the midst of a transition from the internal combustion engine to electric vehicles and there is more electronics in each vehicle. Consequently, it’s fair to say that most mechanics have little experience when it comes to the electronics in these vehicles and will need to be retrained, so there is a huge investment being made in educating the field services force.
“Using our technology, the engineer in the field can call upon the support and experience of an experienced team of experts. Via an inbuilt camera, those experts will be able to see exactly what the engineer is seeing. The camera can be optimally positioned and it’s possible to zoom in. The camera has built-in image stabilisation technology, so the latest version can work in low lights and deliver high resolution.
“Power consumption is managed and depending on what you are doing it’s able to operate across a full shift – that is up to 10 hours.”
By enabling remote experts to assist field workers, it’s easy to work for more customers every day.
“Take a surveyor, for example,” explains Jhawar. “They might have been stuck for an entire day performing one inspection, now they can do two or more remotely. This reduces the time required to meet customer needs.”
As a result, a growing number of customers using RealWear’s solutions are finding that they are more agile in the field, requiring just an internet connection to enable them to respond quickly and with the highest possible level of expertise, regardless of location.
According to Jhawar, “The use of assisted reality solutions was studied by companies before the Covid-19 pandemic, but it is clear that the pandemic has helped to accelerate the need for remote intervention capabilities.”
So, while many devices were clunky and expensive headsets, and poor software and limited bandwidth of wireless networks slowed the adoption of XR adoption, the technology has now certainly advanced to a point where companies like RealWear are now starting to see opportunities with a growing number of potential customers who are looking to be more agile, cut waste, improve coordination, and enhance the customer experience.
