Kinect empowers Melbourne Zoo orang-utans with digital enrichment

Orang-utans at Melbourne Zoo this week have begun a high-tech, world first research project that is aimed at giving the animals control over the games and applications they play to challenge their minds.

Humans have a lot in common with orang-utans, in fact, we share 97 per cent of our DNA with these highly intelligent and complex beings.

Now Zoos Victoria along with researchers from the Microsoft Research Centre for Social Natural User Interfaces at the University of Melbourne are collaborating in ground-breaking research to better understand the way orang-utans learn, interact with technology and make social choices.

The orang-utans are the first in the world to experience new digital technology that will hopefully provide the types of cognitive challenges and social interaction these fascinating animals revel in.

The orang-utans are the first in the world to experience new digital technology that will hopefully provide the types of cognitive challenges and social interaction these fascinating animals revel in. Previous research has found that orang-utans are interested in using and playing with technology, particularly if it allows them to interact with humans.

The tablets or touchscreen computers used previously have had physical and logistical constraints in that the animal needs to be able to operate the device by touch. However, the orang-utans’ strength and curiosity means that tablets can only be utilised when held by a keeper positioned behind a strong mesh screen, with the animal putting its fingers or hand through the mesh in order to operate the device.

Zoos Victoria’s Animal Welfare Specialist Sally Sherwen wanted to enable the orang-utans to engage with the technology the way they want to engage, and make it richer by allowing them more full body movements when playing and also to interact directly with visitors.

Zoos Victoria's Animal Welfare Specialist Sally Sherwen
Zoos Victoria’s Animal Welfare Specialist Sally Sherwen

“They enjoyed using the tablet but we wanted to give them something more, something they can use when they choose to,” says Dr Sherwen.

The research underway this month at Melbourne Zoo takes technology interaction to a whole new level by using a natural user interface, or NUI. Zoos Victoria’s research partner in this project, the Microsoft Research Centre for SocialNUI, normally uses NUI to translate the way humans interact in the physical world.

However this time, the orang-utans will be using advanced technology incorporated in the Microsoft Xbox Kinect, which uses voice and body gestures to control the gaming console.

The research team can now project a full body-sized screen that allows the orang-utans to “bodily engage” with the projection, whether it be rolling, using it on their bodies or bringing over physical objects like leaves and bits of tarpaulin.

The projected screen will act like a touch screen, but no devices need to be physically inside the enclosure.

 At the first testing Malu, Melbourne Zoo’s twelve-year-old orang-utan didn’t disappoint. Spying the projected red dot moving on the floor he immediately went over to it and kissed it.

This week the researchers have been excited to learn how the orang-utans are taking to the projected interface. At the first testing Malu, Melbourne Zoo’s twelve-year-old orang-utan didn’t disappoint. Spying the projected red dot moving on the floor he immediately went over to it and kissed it. The dot duly exploded and when it reappeared he kissed it again, suggesting the orang-utans are indeed keen to use more than their hands to interact.

Orang-utans Feb 2016_5

The project aims to provide a new form of stimulation for the apes, giving them the ability to initiate their own fun while stimulating their problem solving.

Researchers are particularly interested in what these animals, which generally enjoy social interaction, will choose to do in relation to human interaction when they are in control of that interaction.

They are now developing computer games, painting applications and picture galleries specifically designed for the orang-utans. And because the researchers are using projections, the orang-utans won’t be limited to just using their fingers but will be able to use their whole bodies to activate the space. If the trial is successful, within a few years the orang-utans could be playing digital games with visitors whenever they want to.

The project is the latest addition to the suite of activities the zoo uses as part of its digital enrichment practice, which is, in turn, one element in a comprehensive animal welfare program.

In the wild, orang-utans must solve fundamental problems every day for their survival: where to find food, shelter, mates.

“In a zoo environment, all of these challenges are overcome for them,” explains Ms Sherwen. “So zoos need to find other ways to provide animals with mental challenges and puzzles.”

Melbourne Zoo uses a wealth of digital and non-digital activities to encourage positive and natural behaviours. For orang-utans, that means foraging, browsing, exploration and play.

Using digital tools to help draw out these behaviours has the same welfare benefits as using non-digital toys (like painting) and naturalistic enrichment (such as hiding food around the enclosure).

For an animal as smart and curious as an orang-utan, one advantage of digital enrichment is that the experience can be varied to cater to individual interests.

Many of the applications the orang-utans will use over the course of this research project have been designed to show off their unique personalities, such as individual preferences for photos or other objects.

Orang-utans Feb 2016_4

“We think that by providing new experiences and promoting positive behaviours, this form of digital enrichment may have the potential to significantly improve their welfare. The findings from this research will be applied to other animals in zoos around the world,” Ms Sherwen said.

Digital enrichment won’t be appropriate for all species but for those that need cognitive challenges, it is increasingly thought to be crucial and this project will build on previous work in the field.

Dr. Marcus Carter, Research Fellow, with the Microsoft Research Centre for SocialNUI at The University of Melbourne explained orang-utans were chosen for the pilot because they are smart and social.

Dr. Marcus Carter, Research Fellow, with the Microsoft Research Centre for SocialNUI at The University of Melbourne with Kiani
Dr. Marcus Carter, Research Fellow, with the Microsoft Research Centre for SocialNUI at The University of Melbourne with Kiani

“It is well-recognised that orang-utans, and great apes in general, require considerable enrichment including problem-solving tasks designed to challenge their highly evolved cognitive skills.

“Furthermore, research at Zoos Victoria has demonstrated orang-utans enjoy interacting with and watching visitors, meaning there is opportunity to encourage this interaction as part of their enrichment program,” Dr. Carter says.

Dr Sherwen says digital enhancement could eventually be introduced for a number of the zoo’s animals, though the opportunity to interact with visitors would always be limited to those species wanting to interact.

The SocialNUI research centre, launched two years ago by the University of Melbourne, Microsoft Australia, Microsoft Research and the Victorian State Government, explores how emerging computer technologies influence behaviour in social environments. It designs natural user interface technologies to help people communicate, play, learn and work together to support social interactions.

Orang-utans Feb 2016_7

The project, which begins this week, will run for the all of February 2016.