Depth sensing projection system turns any surface into a touchscreen

The prototype technology, dubbed Omnitouch, relies on a combination of a laser based pico projector and a depth sensing camera. The camera is an advanced, custom prototype provided by PrimeSense, which reports depth in world co-ordinates, which are used when modelling a particular graphical world. The laser based projector, on the other hand, is designed to superimpose keyboards, keypads and other controls onto any surface, automatically adjusting the surface's shape and orientation to minimise distortion of the projected images. According to the researchers, the complete wearable system lets users control interactive applications just by tapping or dragging their fingers. "We wanted to capitalise on the tremendous surface area the real world provides," said Hrvoje Benko, of the Natural Interaction Research group. "The surface area of one hand alone exceeds that of typical smart phones. Tables are an order of magnitude larger than a tablet computer. If we could appropriate these ad hoc surfaces in an on demand way, we could deliver all of the benefits of mobility while expanding the user's interactive capability." One of the more interesting discussions during the project, according to Benko, was how to determine where to place the interface surface. The team eventually decided to explore two different approaches. The first was a classification driven model in which the system classified specific objects that could be used as a surface: a hand, an arm, a notepad, or a wall. The second approach took a completely user driven model, enabling the user to finger draw a working area on any surface in front of the camera/projector system. "We wanted the ability to use any surface," Benko explained. "Let the user define the area of where they want the interface to be, and have the system do its best to track it frame to frame. This creates a highly flexible, on demand user interface. "You can tap on your hand or drag your interface out to specify the top left and bottom right border. All this stems from the main idea that if everything around you is a potential interface, then the first action has to be defining an interface area." Benko and his team maintain that although the prototype is not as small as they would like it to be, there are no significant barriers to miniaturisation. The researchers also believe it is entirely possible that a future version of OmniTouch could be the size of a matchbox and as easy to wear as a pendant or a watch.