Kinect Effect Reaches Into Hospitals, Senior Centers

COLUMBIA, Mo. – Dec. 19, 2011 – For the elderly, a fall is never just a fall. Marilyn Rantz’s 80-year-old mother fell and broke her shoulder, and died within six months.

“Falls lead to functional issues and other health problems, and can be a precursor to mortality. My mom was a pretty classic case,” said Rantz, a University of Missouri nursing professor. “It’s an age-old problem of aging. So much spins on this particular issue.”



Tiger Place, an independent living center in Missouri, uses technology such as Kinect to closely monitor seniors’ movement to help prevent functional decline that can lead to falls and decreased mobility.

But what if technology could help prevent falls, and in some cases even prolong lives?

Rantz and her colleagues at the University of Missouri are researching just that, using Microsoft’s Kinect to measure and monitor subtle changes in the gait and movement of older people. Using technology to measure the way people walk more completely and daily, rather than at bi-yearly doctor’s appointments, can give healthcare professionals a chance to intervene sooner.

Helping seniors is just one of a growing number of healthcare applications for Kinect.

Doctors are using Kinect to help stroke patients regain movement. Surgeons are using it to access information without leaving the operating room and in the process sacrificing sterility. Healthcare workers are even using it to help with physical therapy and children with developmental disabilities or Attention Deficit Hyperactivity Disorder (ADHD).

“Honestly, what we know about here at Microsoft is but a tiny fraction of what is actually going on,” said Bill Crounse, a medical doctor and Microsoft’s senior director of worldwide health, referring to medical uses of Kinect. “Everywhere I go in the world – every hospital, college or public health organization, people are already doing something with Kinect or they plan to.”

Launched a year ago as a controller-free gaming device for Xbox, Microsoft sold a world-record 8 million Kinect devices in its first 60 days on the market. This made Kinect the fastest-selling consumer electronics device in history, according to Guinness World Records.

Even as Kinect was enjoying unprecedented consumer success, the device was taking on another life of its own outside the living room as scientists, tinkerers, educators, hobbyists and healthcare workers started dreaming up and creating non-gaming applications for the device.

Keen on encouraging the fast-growing wealth of non-gaming applications that have sprung up for Kinect, Microsoft released an academic and enthusiast software development kit for non-commercial projects in June and will release a similar kit next year for commercial uses.

Thus, the genesis of the so-called “Kinect Effect” – a term coined in the hallways and conference rooms of Microsoft to describe the device’s increasingly widespread appeal and diversity of uses.

Tiger Place

Tiger Place is a different kind of independent senior housing, and not just because the upscale senior apartment building has a library, a movie theater, a beauty parlor, a veterinary clinic and two sports bars.

Built by Americare, Inc., in cooperation with the Sinclair School of Nursing at the University of Missouri, Tiger Place is the first facility of its kind. It provides seniors with a place to live independently, but was also built with the intent to provide an environment for interdisciplinary research to study aging and eldercare technology. The group of faculty, staff and students researching at Tiger Place represents several fields of study – electrical and computer engineering, computer science, nursing, medicine, social work, physical therapy, and health informatics.

Tiger Place focuses on monitoring its residents with a network of sensors placed in apartments, a monitoring network that now includes Kinect sensors in many rooms. What’s more, Tiger Place is an “age in place” facility, meaning seniors don’t have to move to different housing as they get older and require more assistance – the new services they need as they age are brought in to them, Rantz said.

Several apartments in Tiger Place have a Kinect mounted near the ceiling in the living room, where day after day the devices gather a mountain of data about the resident’s movement and motion.



The Kinect Effect began in the device’s first 60 days on the market, when more than 8 million of them were sold. That made Kinect the fastest-selling consumer electronics device in history, according to Guinness World Records.

“The typical way to measure fall risk is a physical therapist with a stop watch and watching someone walk,” said Marjorie Skubic, a professor in the college’s Electrical and Computer Engineering department.

She said the team has been collecting gait data with a variety of sensors (including radar) since 2005, and introduced Kinect six months ago. The research is funded by the National Science Foundation, Agency for Healthcare Research and Quality, and National Institute of Nursing Research of the National Institutes of Health.

“With Kinect, we can gather finely grained, gait data – walking speed, stride length, step time, and we can see detailed trends over time to determine subtle changes and determine very early whether there is functional decline and fall risk,” Skubic said. “We’ve also tried to make it really passive. For the most part people don’t think about them being there. That’s what we’re going for.”

The Kinect sensors were specially adapted for Tiger Place by Erik Stone, an electrical and computer engineering Ph.D. student at the university. Stone, who got his master’s degree studying robotics, is interested in computer vision, which led him to the field of eldercare.

“Eldercare research has a lot of problems to look at related to computer vision. The main thing we’re looking at in our group is being able to do things passively with environmentally mounted sensors to detect falls, the onset of illness, functional decline and just over all activity level,”Stone said.

Stone said before Kinect, he was working with a pair of specially calibrated cameras placed in two different parts of the room to get in-depth images.

“Now, for $150, you can get a 3-D picture of the world. It’s really rich, and it definitely moves things forward a lot more quickly than we were moving before,” Stone said. “Based on the results we see, I think it’s something we’ll keep using.”

Rantz and Skubic have no doubt that with Kinect the Tiger Place team will be able to see more changes in residents, and see them earlier. The more detailed data the team has over time, the better the automated alerts can become.

“We’re not trying to diagnose, but we can say, ‘Hey, take a closer look, there might be something going on here,’” Skubic said.

The sensors have specially programmed parameters for the data gathered, and can even automatically email nursing staff should it detect something awry – perhaps someone whose movement has slowed, and eventually someone whose gait has changed, or even someone who has actually fallen.

The team hopes the research and results at Tiger Place will inspire other multidisciplinary, technology-rich eldercare housing. Using Kinect and other kinds of sensors to identify functional decline can mean not only improving the quality of life of older adults, but may even extend their lives.

“We’re hoping, and in fact we’re working, to catch this – to measure functional decline, to use those measurements to intervene earlier, and to help people regain function,” Rantz said.

She added, of the data the team is gathering, “this is pretty amazing stuff. Sometimes I can’t believe it as I’m writing [the research] up.”

Royal Berkshire Hospital

At the Royal Berkshire Hospital in Reading, England, stroke patients are using Kinect for Xbox 360 as part of their rehabilitation. Doctors assign patients to play different Kinect games, depending on what kind of rehabilitation they have to do.

One patient who didn’t have much arm movement played Kinectimals, a game in which wild cats respond to being petted.

“The patient thought it was marvelous and we could actually see an improvement occurring, rather than the normal stretching and pulling a physiotherapist would do to the patient,” said Malcolm Sperrin, director of medical physics at the hospital.

Another patient had problems with standing and full-body movement.

“We had him bowling,” Sperrin said. “He was able to work on coordination between the twisting of his body and the movement of his hands, plus his eyes had to look at the screen rather than where his hands are. It’s been enormously beneficial to him.”

Sperrin said Kinect is helping patients at the hospital, but also after they leave, explaining that many of them are using Kinect to continue their rehabilitation at home.

“It’s worked extremely well,” Sperrin said. “One of the reasons we like the way its developed, first of all it works for us off the shelf with no modifications at all, but it’s also good fun so people can take it home and continue their work with family and friends and of course children – everyone can help with this self-directed improvement.”

Though the hospital’s neurological rehabilitation team has had success with the consumer version of Kinect, Sperrin said he and his team are busy brainstorming ways that the device can be adapted and developed more specifically for stroke rehabilitation.

Microsoft’s Crounse said the so-called Kinect Effect in the device’s first year on the market is but a glimmer of what’s to come, especially when it comes to healthcare. As Microsoft continues to deepen Kinect’s technology, partners, researchers and even businesses will continue to find ways to adapt it for a veritable universe of healthcare-related uses.

“The Kinect effect barely describes it. It’s like the Kinect explosion,” Crounse said. “In my decade at Microsoft I have never seen anything that we are doing generate as much excitement in the consumer community, nor have I seen such excitement in the clinical and research communities as I have about Kinect and how this technology can be applied in health and healthcare.”