A coming together of advances in hardware and software is creating portable devices that offer personal services, based on vast amounts of data about every individual who uses them – which, today, is nearly all of us.
First generation AC products have taken the form of intelligent assistants: probably the best known are Siri (on iOS) and Google Now (on Android). Others include MindMeld, developed by San Francisco based Expect Labs, which uses natural language processing to understand a user's context and needs and deliver relevant information, often before anything specific has been asked for. Cortana, for the Windows Phone, is similar. Many developers believe AC based on advanced machine learning and using excellent speech recognition interfaces will create an irresistible combination within the next few years.
An early pioneer of AC is London based SwiftKey, which developed an input method for Android devices. Based on a blend of AI technologies, it can predict the next word the user intends to type. SwiftKey learns from previous messages and outputs predictions based on currently input text and what it has learned. Company cofounder and CTO Ben Medlock says the emergence of AC is the result of a concerted effort across both academia and industry over the last decade or so to understand the application of machine learning techniques to real world problems, driven by unprecedented access to digital data.
"The next wave of predictive systems will be based on techniques that are differentiated by their ability to learn a flexible, powerful representation of the world around them. Recent systems based on neural networks operating on vector space models using deep learning principles are one example of this," Medlock says. "I think we'll start to see some progress in hardware designed along architectural lines that are directly supportive of methods such as those described above. IBM's SyNAPSE chip is an early example."
Surprisingly, perhaps, the seeds of AC were sown in the 1980s, according to Velijko Pejovic, Assistant Professor of Computer and Information Science at the University of Ljubljana, who been studying the field and its potential for years. He also has close ties with the University of Birmingham.
"The theory behind AC has been developing since the 1980s, yet we've had very few practical realisations, mostly in the field of robotics," he says. "The main reason for the lack of AC applications was the disconnect between the real world and the anticipatory models."
Now, modern mobile technologies are bridging this gap for several reasons. First, they combine an array of sensors undreamt of a decade ago – such as accelerometers, GPS chips, cameras and microphones. This means today's smartphones can be fully aware of their surrounding context.
"Second, the culture around the mobile phone – and that of emerging wearable computing devices – makes them the most ubiquitous and personal device in human history," Pejovic says. "We are with our phones at all times. They are in our pockets, purses, beside our bed. Such a pervasive use of the device, together with the sensing capabilities, means phones can be trained to know exactly what we are doing at all times."
The result is that our mobiles will predict what we are about to do. Only now has hardware reached the point where it is possible for them to run such software efficiently.
"Different approaches – such as Bayesian networks, Markov chains and neural networks – are used for different purposes, like location prediction or image recognition," Pejovic says. "But, in the end, it is crucial for the model to be trained on a lot of quality data."
Thomas van Manen is a researcher who recently produced a report on AC for VINT, part of London based IT company Sogeti. To describe the coming together of technologies and software underlying AC, he cites advances in SMACT, (Social, Mobile, Analytics, Cloud and Things).
"SMACT makes possible highly contextualised systems that can serve us in a personalised and anticipatory way," van Manen says. "Siri and Google Now are the first steps into a new generation of AC systems."
Today's mobile hardware may be impressive, but there is more to come because we are still limited to sensing the external physical environment, Pejovic explains.
"For the future, we are developing models that aim to infer if a person is interruptible or not. For example, will sending a message at a particular time result in a person actually picking up the phone and reading the message? Also, is the person going to be annoyed by the message or not? While we found initially that data like the phone's acceleration, location and time of day can signal a user's attentiveness, many other relevant variables remain hidden from a smartphone's sensors. For example, is a person deeply engaged in working on a project with a tight deadline or are they just browsing the Web? Sensors measuring a galvanic skin response or a heart rate, which can be embedded in smart watches, for example, will provide more insights about the user's internal state and help us predict interruptibility more accurately."
What are potential obstacles to AC? The most frequently cited is privacy.
"Take sound sensing for example," Pejovic says. "Suppose there is an app that predicts the length of a discussion, based on the initial turn taking in a meeting. Here, ethics would require that all involved parties are aware of and compliant to the conversation being recorded. At the same time, we can devise technical solutions that maintain the privacy of third parties. We can discard raw speech data and keep only an indicator stating if a current sound frame carries a human voice or not, but we can assign anonymous IDs to persons in a conversation. Finally, thanks to the advances in computing hardware, we can do all the processing on the phone, without ever sending the data to a cloud where the data might leak."
Van Manen believes privacy is one of three potential obstacles to AC.
"Anticipatory/predictive computing benefits from data that is a personal as possible. Also, it needs to track a user 24/7 to learn about behaviour. This means users must give companies a carte blanche for tracking and harvesting personal data in return for great services – or at least they hope so. This is not so different from the current debate whether Gmail is free or not. Trading privacy for services is just as hard coded in the digital age as 1s and 0s.
"The second is hacking. In a scenario where biometric data is part of your personal anticipatory system, this can get ugly. It might also be a bigger deal to you if your thermostat or smart lock gets hacked, compared to an Instagram account. It might also make crime easier."
A possible scenario would be hacking into a smart energy meter to see if the temperature settings were set to 'at home' or 'away'. In that way, a burglar could ensure no one is home when they break into a house.
"The third is that AC systems are a vendor 'lock in' waiting to happen. Will all your systems, devices, and services work together, no matter which logo they are branded with? Probably not. And AC systems obviously benefit from an all things integrated approach."
SwiftKey's Medlock agrees that AC could face potential problems in terms of mainstream adoption.
"Many people are uncomfortable with companies having access to data which may be personal or sensitive in nature. Certain types of AI based computing, such as that found within SwiftKey, require this sort of access in order to operate properly. We use machine learning to improve and tailor our word predictions for users, based on what they have typed previously, resulting in a demonstrably faster and easier writing experience.
"Companies will need to find a way to communicate such benefits to users in a way that assuages privacy fears. Part and parcel with this, however, is the responsibility these organisations then have to protect and care for this data. If this power is abused, it will put the future of this industry in jeopardy."
Another challenge is overcoming a general public bias against artificial intelligence (AI), boosted by comments from the likes of Elon Musk and Stephen Hawking, who have warned that the advancement of AI is a threat to the human race. Medlock does not buy that message.
"If the history of AI is characterised by anything, it's over optimism. Many people since Turing have overestimated our ability to replicate the intelligence of the human brain."
Is AC limited to use by individuals, or could large corporations or governments use it? Pejovic thinks the latter is unlikely. He draws a distinction between prediction and anticipatory action based on what a system predicts.
"The predicted element of AC is concerned with looking ahead and inferring what is about to happen. AC, on the other hand, bases its autonomous actions on the predictions and, as such, AC for larger bodies is far more difficult to advocate for. An AC application for an individual might proactively book a taxi from an airport and adjust the taxi pickup time according to a predicted delay of the user's flight. The cost of a wrong prediction is not that high, and the impact is limited to a single individual. In the case of governments or corporations, predictions will likely become an integral part of the business intelligence. But we are unlikely to see large organisations trusting their business to automated actions of an AC system."
Van Manen believes the main difference centres on the outcome of processing big data.
"For the individual, all the big data in the world eventually boils down to presenting the small amount of data that matter to you. Also, the data sets that need to be processed for us as individuals to benefit from are a lot smaller and will require less processing and analytical power. The outcome for individuals will be a more automated environment based on preferences.
"For companies, AC might also be about replacing human work. I have seen a demo at a Dutch bank of IBM's Watson (its AI system that won the Jeopardy TV quiz show in 2011). The anticipatory/predictive computing power in this situation was all about doing routine tasks like presenting reports and updates, market analysis and risk calculation. The analytical power of Watson, combined with context awareness about upcoming events and things that happen in the real world, will result in fewer jobs for people at that bank."
If that happens on a large scale, we all might need a computer that is also a friend.
