Displaying innovation

Research and development as a major activity is disappearing from the agendas of large companies. While many organisations will claim to perform R&D, reality is the emphasis is very much on the 'D'; little attention is paid to the 'R' side of things, which is increasingly expected to be performed elsewhere. The effect of this is that traditional 'blue sky' research is not being undertaken. Even universities – the traditional home of blue sky research – are beginning to move away from this, driven by the need to develop industrial sponsorships as a revenue stream. But the story is different at Microsoft, where researchers around the world are free to pursue topics which interest them. Microsoft Research was established in 1992 at the company's Seattle headquarters. Since then, labs have been set up in Europe, India and China. The second Microsoft Research lab was set up in Cambridge in 1997, headed by noted computer scientist Roger Needham. Originally occupying a small office in the centre of Cambridge, the lab now boasts more luxurious accommodation on the city's western outskirts. The Cambridge lab is directed by Microsoft Distinguished Engineer Andrew Herbert, who said the facility was all about enabling innovation. "It's about creating new concepts, exploring new ideas and combining these with changes in technology. We're rethinking computing here and it's one of the most exciting things in my career." The lab is charged with working out new things for computers to do and Herbert believes these changes come in waves – technologies and experiences that are coupled tightly. "The pc was one wave, the internet was another. Some things are opening new opportunities and these will be dependent upon software." Microsoft Research has the advantage that it is not beholden to any product group within Microsoft. "We're funded centrally," Herbert noted, "which means we can decide our agenda and the exciting things that we want to work on. We're in charge of our destiny and our long range independence." Microsoft has come a long way since its first research lab was set up. "There were about 5000 employees," Herbert believed, "and the corporation had just broken through the $1billion annual sales barrier. In terms of technology, Microsoft had just launched Windows 3.11." Microsoft Research was set up because the company understood the microprocessor would evolve and take on functions previously handled by mainframes. "Questions being asked included 'what can we do with pcs' and 'how can we exploit this technology for new opportunities?'," Herbert added. It would have been easy for Microsoft to centralise its research work at the Seattle headquarters. But that wouldn't have been the best decision, said Herbert. "Not all the best computing talent is in the US," he observed. "If you want to tap into expertise, then you need to have a lab where those people are." Amongst the justifications for the Cambridge lab were Europe's strong tradition in software verification, as well as its leading position in the 1990s in mobile telephony. Today, Microsoft Research boasts around 1000 researchers, of which 100 are based at the Cambridge lab. The latest lab to open – confusingly in Cambridge, Massachusetts, and therefore called the New England lab – is looking at how to model and understand the social phenomena going on around the web. The organisation is adding around 80 researchers a year. "That's the size of a university research group," Herbert noted, "and it's very important when you have new ideas that you provide a fertile environment for research." But the Cambridge lab isn't working in isolation; it's partnering with leading European universities and is recruiting internationally. "As the lab has grown," Herbert continued, "it has become better known for what it does, rather than where it is. Even though it's an international business, our labs still compete with each other in a friendly way. But we do try to ensure that we don't do the same projects in different labs." Image projection One of the projects being pursued at the Cambridge lab is SecondLight, a surface computing technology that can project images and detect gestures in mid air, as well as detecting interactions on the display's surface. "It's part of work looking at how to get beyond interacting on a surface," explained researcher Dave Molyneaux. The system works by using a polymer stabilised cholesteric textured liquid crystal optical switch as the projection surface. The surface is toggled between clear and diffuse states at 60Hz, quickly enough for the changes in state to not be detected by the human eye. When diffuse, images are projected on the display's surface in regular fashion. However, when clear, another image can be projected through the surface. In order to project different content on the surface in both modes, two different images need to be projected in sync with the switching diffuser. The 'on' image is displayed for 8.3ms, followed by the 'through' image for 8.3ms and so on: an effective frame rate of 120Hz. Whilst 120Hz projectors are available, the prototype is equipped with two Hitachi CPX1 60Hz projectors, each fitted with fast optical shutters to create the two interleaved 60Hz images. The system can also take images of what is on the diffuser surface and beyond using two cameras mounted behind the diffuser. Although SecondLight has the ability to capture full colour images, the work is currently restricted to infrared. One camera works with 264 leds positioned at a 6mm pitch around the diffuser to sense fingers and other objects. The second camera is triggered to capture images only when the diffuser is clear. In this way, it capture images beyond the surface. Two projectors means two images can be seen – apparently simultaneously. The 'through' image can be captured on a thin sheet of diffuse film – even tracing paper – while the 'on' image is seen on the diffuser. These images can be complementary; the demonstration at the Cambridge lab used a photograph of the stars as the 'on' image and annotations as the 'through' image. Similarly, the 'on' image could be that of a car and the 'through' image that of its internals. Of course, the problem remains of how to interact with an image projected into space. One way of solving this problem is a mobile surface. At the moment, this is a piece of perspex illuminated by a row of infrared leds and featuring total internal reflection. Two lines laser etched at either end of the surface generate infrared radiation and allow their position in space to be determined. Using a set of algorithms, any distortion in the image is corrected and the mobile surface can be held at almost any angle. And the mobile surface can also capture touch inputs. Molyneaux believed the approach could allow different ways to visualise volumes. "Think about a CAT scan," he suggested. "The result of the scan is a 3d 'volume', but how can you interact with it? It's possible with SecondLight that this 'volume' can be placed in space above the surface and the mobile surface used to slice through the image." New ways of thinking The lab's mission is to use computer science to find new ways of doing things and to solve difficult problems. Herbert said: "It's often a balance between trying to answer questions driven by the subject itself and solving practical problems." Professor Chris Bishop, chief research scientist at the lab, added: "One thing that's been key is that researchers can choose their own project. The core part of the job is deciding which questions to ask and this helps to attract the best scientists. Deciding which questions to ask is 60% of the job; figuring out the answers represents the other 40%." In Herbert's opinion: "Researchers can look at new things, they can look at things in different ways and they can look at developing disruptive technology." Examples of such disruptive technology offered by Herbert include the first version of the Internet Protocol, the first version of Microsoft's search engine and the 3d graphics used in the XBox. "We have had a significant impact on the company," he believed. Today, the Cambridge lab has five main research areas: programming principles and tools; operating system networking and distributed systems; computer moderated living; machine learning and perception; and computational science. "The dominant theme amongst these," Herbert believes, "is understanding how software, the web and mobile devices will work together. We need to understand cycles and how they work." The cycles to which Herbert referred can often be long term. "It took Windows 10years to outsell MSDOS," he pointed out. "Word is now in its ninth release in 10 years and Excel has had five releases in 10 years. It's a long path of innovation, integration and development." Meanwhile, even though Windows Embedded is now at version 6, Herbert believes the approach is 'only just beginning to come on stream'. "And there are new opportunities in education, health care, imaging and robotics." One research strand in Cambridge is how to deal with the implications of multicore computing. "The challenge is to build software than can exploit multicore parallel computing," Herbert enthused. The approach really demands a change in thinking, he continued. "With a single core, we always had to keep some processing power in reserve, so the focus was on underusing hardware. If you have a multicore processor, the problem changes dramatically. Now, we have to think about ways in which to use the hardware flat out." Potential applications here include systems that feel more 'human'. "While we can't replace humans, you can do some things in a more humanistic way." Multicore processing is also likely to support the emergence of what Herbert terms 'spatial computing'. "Systems are becoming aware of their context. For example, a mobile phone 'knows' where it is. One vision is that space is the new frontier; computing is becoming 3d and the computer is the window into a world which you can navigate. Touch and gestures will be important elements of spatial computing. The computer is ceasing to be something we drive and control. It's now becoming something we sit alongside; it's developing into an assistant, rather than a tool and we will interface with them using natural movements, such as gestures and speech." He believes a spatial web will develop, with a combination of 'computing in the cloud', virtual Earth – 'so we know where we are' – and social networks. Possibly the final element will be synthetic scene building; where computer graphics are added to images of real scenes to provide information to the user. Ultimately, technology is transferred to Microsoft's business units, where the research is turned into products. "We're doing things which take Microsoft into the future," Herbert concluded.