REDMOND, Wash. — March 2, 2010 — It’s safe to say that computers have become a bit more powerful over the past 15 years.
Cellular technology untethered our phones. The Internet brought the world into our homes and offices. And the power of microprocessors has kept leaping forward. Now we’re embarking on a new era, where all kinds of powerful computers, in all shapes and sizes, will work hand in hand with vast online databases. Craig Mundie, Microsoft’s chief research and strategy officer, calls this the “client plus cloud” era.
Mundie, whose job is to interpret the impact of emerging technology trends, says that changes under way in the tech industry today have the potential to completely change the technology world as we know it.
“We’re approaching an imminent sea change in technology that will transform everything we know today,” Mundie says. “A combination of the cloud plus very powerful client machines, along with a revolution in how people interact with computers, will define the next era of technology — and have a vast impact on society.”
Recent, significant advances in microprocessors, up to 100 times as powerful as the machines we’re using today, are emerging at every level — chip, device and data center — and are enabling our everyday client devices including PCs, phones, e-books, game systems and more to take on very complex computing tasks. At the same time, the cloud is scaling up its service capacity thanks to massive data centers. Taken together, they form a new programming paradigm, the seamless client-plus-cloud platform.
And if that’s not enough, we’re also seeing new ways to interact with computers, via a natural user interface or “NUI” that embraces gestures, anticipatory computing, expressive response, contextual and environmental awareness, and 3-D or even immersive experiences. These new forms of input, Mundie says, will create a startling transformation in how humans and computers interact.
“The transition to a natural user interface will change everything from the way students write term papers and play computer games to how scientists study global population growth and its impact on our natural resources,” Mundie says. “In the healthcare field, physicians and patients alike will also benefit from simpler and more effective tools with which to communicate and share information.”
The NUI Revolution
When it comes to the evolution of technology, hardware and software play off each other, with advancements in one area serving to stimulate change in the other, resulting in new kinds of devices and applications.
As an example, Mundie points to the release of Microsoft Surface in 2007. Before that time, Microsoft had relied on stylus input for its Tablet PC designs. While pen computing hasn’t yet taken off, touch has really resonated with consumers.
Mundie says the success of touch technology is a precursor to a world where there will be many tactile inputs, with large Surface displays on walls and tables offering two-way communication with computers.
“You can see it happening naturally, driven by HDTV and the rapidly falling cost of display devices,” he says. “The hardware trajectory will be led naturally to surfaces both horizontal and vertical, and as large as the wall itself, which will provide both display and input.”
Another input revolution on the horizon, says Mundie, is the ability of devices to recognize gestures. Microsoft has already taken a big step in this direction with the development of code-named “Project Natal,” a system originally designed for Xbox that uses a camera to recognize human gestures as a video-game controller.
The industry has long pondered how to develop a truly natural user interface to replace the decades-old graphical user interface (GUI), where a keyboard and mouse are used to input a range of symbols on screen, and language is conveyed via reading and typing.
Many approaches have been tried. While the first iterations of the NUI simply used pen input, speech or simple touch to manipulate the GUI, Mundie says the aim today is an entirely new relationship between people and computers.
“Our early versions of the NUI were really nothing more than enhanced GUIs,” says Mundie. “The new NUI will embrace gestures and expressive responses. Computers will be aware of context and your environment, and will react accordingly. Instead of working at our command, they will increasingly work on our behalf.”
In this NUI world, Mundie says, the computer will evolve from something you sit in front of or carry around — essentially a tool that you must proactively direct — to a system that actively listens, learns, understands and works for you even while you’re away, much like a human personal assistant.
“You won’t necessarily sit down at a computer terminal,” he says. “Computing will be all around you, and you’ll basically converse with that pervasive intelligence.”
In other words — you’ll talk, the computer will listen and respond appropriately. You’ll gesture, and the computer will understand your intent. The computer will remember these interactions, and anticipate what you might need in the future. Mundie believes the new NUI is one where users will interact almost as they would with a live person sitting across from them.
“Computers will be aware in whole new ways,” Mundie says. “When you talk to them, they will know where you are, what you’re doing, and who is with you. They will remember why you’re there and what you’re trying to accomplish.”
Advancing the Arts and Sciences
Mundie says that gestures, speech, touch and other forms of human-computer interaction will naturally move into other areas, such as art and science, and in fact they already are. He points to a recent research project at Microsoft called “Gustav,” which puts users inside an immersive digital painting environment. Gustav is already able to provide convincingly real modeling for pastel and oil paintings.
Microsoft’s Chief Research and Strategy Officer Craig Mundie says that changes underway today have the potential to completely change the technology world as we know it.
When it comes to science, Mundie says, new ways to visualize and work with data are sorely needed. Software abstraction, increased computing power, and data at massive scale hold great promise to help researchers do science in new ways.
“Humanity’s biggest challenges in the next century will largely be scientific ones,” Mundie says. “We have more data than ever before. Now we need new ways to visualize, analyze, and ask new types of questions that lead to deeper insight.”
Mundie points to Microsoft Research’s Computational Science Lab, based in Cambridge, U.K., which is pioneering a new generation of scientific methods and software tools to tackle fundamental problems in science. Among them, a project code-named “Computational Science Studio” integrates novel computational methods and huge amounts of disparate data to enable new kinds of multi-scale modeling of complex systems.
“Using ‘Computational Science Studio,’ scientists can spend less of their time programming, and more time focused on advancing their research,” says Mundie. “This technology can radically reduce the amount of time it takes to achieve this level of science — from months to minutes.”
The researchers are working on a model that seeks to better understand the nature, interaction and impact of stresses on the planet. They are looking at the effects of population growth, climate change, land use and deforestation on future risks to food and water resources. These kinds of models, Mundie says, are urgently needed by scientists, policymakers and governments worldwide.
Bringing New Approaches to Healthcare
The healthcare industry is another that stands to benefit from advances in technology and more natural computing interfaces — and evidence of the potential impact is already surfacing.
Microsoft made the decision five years ago to invest significantly in health. Mundie created the Health Solutions Group from scratch, and today it has more than 700 people — from doctors to software developers to health researchers — focused on some of healthcare’s toughest problems. The group’s offerings enable a “connected health” scenario, providing ready access and sharing of information within the research community (Amalga Life Sciences), across the healthcare enterprise (Amalga UIS) and out to the patient (HealthVault).
“When we look at the healthcare industry, we see that it’s about getting the right information to the right person, at the right time — and ensuring that all this information is centered on the patient,” Mundie says. “We see incredible opportunity for information technology to help improve the healthcare experience: from the physician to the patient, and across all types of institutions.”
In today’s data-intensive society, patients are often caught in the middle of the information onslaught. Even something as straightforward as hospital discharge instructions can leave patients bewildered. A recent survey found that 80 percent of patients feel like they understand their discharge instructions, when in reality only 20 percent have fully understood what they need to do. (Annals of Emergency Medicine Vol 53 No 4: April 2009).
Exploring how to close this gap, the group’s Healthcare Innovation Lab in Washington, D.C., has developed a solution using Surface technology that is currently being piloted at Washington Hospital Center in D.C.
The solution uses Surface as a discharge station, and as part of this experience, the care team can show the patient any of his or her records — EKGs, X-rays, lab results — pulling from the Amalga system within the hospital and pushing digital copies into the patient’s HealthVault account. In this way, they can also share the data with additional physicians outside the hospital walls. The solution also integrates Microsoft Bing, which offers another resource to locate pictures, videos and articles that might help illustrate and explain the patient’s situation.
The New World Ahead
So what will the computers of tomorrow look like? According to Mundie, the PC we know today may take on a dramatically different form.
“I really believe one major change coming soon is the next class of fixed computer — the room,” he says. “Whether it’s your living room, your office, your studio, your laboratory; essentially, your computer will be all around you.”
In the next few years, ultrafast microprocessors, advanced software and enhanced display technologies will enable entire rooms to act as immersive computing environments. And all of our computers will have effective human sensory capabilities, act as proactive and anticipatory helpers, and move us into an exciting new era of NUI computing.
