The Age, 1 February 2016
Digital technology presents significant opportunities to support Zoos in their goals of animal welfare and enrichment. Researchers from the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) at The University of Melbourne are excited to be collaborating with Zoos Victoria to explore how natural user interface technologies (NUIs) can improve animal welfare.
Orang-utans are highly intelligent species critically endangered due to deforestation of their natural habitat for palm oil plantations. Through our collaborative project we are studying how digital enrichment can improve the welfare of Melbourne Zoo’s 6 orang-utans and building a better understanding of the way orang-utans learn and interact with technology. By studying the design and use of digital enrichment we are seeking to better understand the zoo as a context for human-computer interaction (HCI) research, and develop methodologies for designing for non-human users.
The research team are applying ‘user-centered design’ methods to approach the development of enrichment for orang-utans. This involves observing orangutans and keepers, interviews and workshops with animal welfare experts, keepers and other personnel, to identify orang-utans’ motivations and interests and define the opportunities for NUIs in this space.
Early on in this process, the SocialNUI researchers studied how Melbourne Zoo was utilising technology with the orang-utans. As well as projecting movies into the enclosure, the keepers have explored the use of tablet computers to provide enrichment in the form of puzzles, games, musical instruments and painting. While this shows the potential for digital enrichment, it is limited as the orang-utans (who would readily break fragile tablet computers) can only interact with their fingertips through metal bars while a Zoo Keeper holds the tablet outside the enclosure. Consequently, this project utilises NUI technologies to give the orang-utans a wider range of ways to interact with technology by offering them more active, full body movement.
The proof of concept system was tested in February 2016 at Melbourne Zoo. This solution uses a Microsoft Kinect and a projector placed outside the enclosure, which avoids the need to give the orang-utans physical devices that could be broken. The system projects moving images on to the floor of the orang-utan enclosure. Using custom software, the Microsoft Kinect sensor tracks the orangutans’ movements and detects their touches and interactions with this area of the floor, so that the projection becomes a large touch-enabled surface.
The orang-utans were given complete choice over whether to interact with the system or not, and all trials were overseen by the Zoo’s animal welfare expert.
The team developed a variety of games and simple applications for this system. The applications aim to explore how orangutans prefer to interact with the system, and demonstrate opportunities for enrichment. The games are designed to encourage orang-utans to play, explore and learn about what they can do with the system, building on their natural problem-solving abilities.
The games were developed in close collaboration with the zoo keepers, drawing on their knowledge of the orang-utans different personalities, likes and dislikes. For example, researchers discovered that Kiani (Suma), Melbourne Zoo’s 37 year old female orang-utan, loves looking at photos of herself. The team developed an application that showed a gallery of photographs that the orang-utans could browse, select and interact with. The orangutans have experience of painting on canvas and seem to find this enriching, so a painting application was also developed.
Orang-utans’ willingness and ability to use this technology without training or encouragement indicates that it can provide a foundation for diverse cognitive challenges which respond to their individual needs and interests, to keep their minds and bodies active.
As part of the trial the orang-utans were offered the opportunity to interact with humans through the technology. As orang-utans seem to find humans interesting, the team created a shared digital space where orang-utans could choose to interact safely with keepers and even visitors. In the first trial of a game for humans and orang-utans, the team observed with delight that they chose to play even with people they had not met before, creating a powerful sense of connection for the human player.
Technology at the Zoo Workshop
On the 6th and 7th May 2016 the project’s researchers hosted a workshop on technology at the zoo at the ACM SIGCHI Conference, the leading academic conference on human-computer interaction (HCI). Titled ‘HCI goes to the Zoo’, the workshop included delegates from leading zoos and institutions in North America, Europe and Australasia, to discuss the opportunities and challenges for digital enrichment as part of zoo animal enrichment, zoo keeping and the visitor experience.
This project is a collaboration between The University of Melbourne’s Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) and Zoos Victoria and proudly supported by Microsoft.
Marcus Carter, Research Fellow, Microsoft Research Centre for SocialNUI, University of Melbourne
Sarah Webber, PhD Candidate, Microsoft Research Centre for SocialNUI, University of Melbourne
Zaher Joukhadar, Lead Software Engineer, Microsoft Research Centre for SocialNUI, University of Melbourne
Sally Sherwen, Animal Welfare Specialist, Wildlife Conservation & Science, Zoos Victoria
Pons, P., Carter, M., & Jaen, J. (In Press). Sound to your Objects: A Novel Design Approach to Evaluate Orangutans’ Interest in Sound-based Stimuli. In Proceedings of The Third International Congress on Animal Computer Interaction. ACM. [PDF]
Carter, M., Webber, S., & Sherwen, S. (2015). Naturalism and ACI: Augmenting Zoo Enclosures with Digital Technology. In Proceedings of The Second International Congress on Animal Computer Interaction. ACM. [PDF]
Rault, J.-L., Webber, S., & Carter, M. (2015). Cross-Disciplinary Perspectives on Animal Welfare Science and Animal-Computer Interaction. In Proceedings of the Second International Congress on Animal Computer Interaction. ACM. [PDF]
Pursuit, 1 February 2016
Kinecting with the orang-utans
Gamespot, 1 February 2016
Xbox Kinect Being Used by Orangutans at Zoo
The Age Newspaper (Print), 2 February 2016
Orang-utans get smart with technology
Microsoft News, 2 February 2016
Kinect empowers Melbourne Zoo orang-utans with digital enrichment
New York Daily News, 2 February 2016
Xbox Kinect used by orangutans at the Melbourne
Futurity, 2 February 2016
‘Kinect’ At The Zoo Could Let Orangutans Play Games With Visitors
PC & Tech Authority, 2 February 2016
Melbourne Zoo’s Orang-utans get access to Kinect-powered apps and games
Win Beta, 2 February 2016
Microsoft Kinect and Xbox One let orangutans play video games
Gizmodo, 2 February 2016
Melbourne Zoo’s Orangutans Are Playing Videogames
Tech Times, 2 February 2016
Experts Let Orangutans At Melbourne Zoo Tinker With Xbox Kinect
Mashable Australia, 2 February 2016
Scientists are using Xbox Kinect to let orangutans play video games
ABC News, 3 February 2016
Orangutans in Melbourne Zoo playing interactive videogames using Xbox technology
ABC 774 Drive, 3 February 2016
Interview with Sarah Webber: Microsoft Research Centre for SocialNUI
(at 2hrs 48mins)
Channel Ten, 3 February 2016
Daily Mail Australia, 3 February 2016
Wonder if they’re playing Ape Escape? Zookeepers teach orangutans to play video games to boost their mood
ABC Online, 3 February 2016
Orangutans in Melbourne Zoo playing interactive videogames using Xbox technology
Huffington Post, 4 February 2016
Orangutans Learning To Play Xbox To Fight Boredom In Zoo
TechFlow, 5 February 2016
Kinecting with Orang-utans
News Beat Social, 5 February 2016
WATCH: Orangutans Learn to Play Xbox Kinect
Channel 7 News, 9 February 2016
World-first technology enables highly intelligent orangutan to interact with humans at Melbourne Zoo
Zoos Victoria, 9 February 2016
World-first study gives orang-utans the choice
The Courier Mail, 9 February 2016
Orang-utans use Xbox tech to interact
Channel Ten News, 9 February 2016
World first for Melbourne Zoo orangutans
Channel Nine News, 10 February 2016
Video: Tech savvy orangutans master computer games at Melbourne Zoo
Channel Seven, 10 February 2016
Lonely Planet Travel News, 10 February 2016
Orangutans in Melbourne Zoo play Xbox for university study
Zoonation.org, 12 February 2016
BBC, 15 February 2016
Orangutans try out motion-controlled games in Australia
ABC News 24, 16 February 2016
In a world first program, orangutans are playing interactive video games at Melbourne Zoo, Interview with Dr. Marcus Carter
ABC Behind the News, 16 February 2016
National Geographic, 16 February 2016
Melbourne Zoo’s Orangutans are playing video games
The Conversation, 14 March 2016
Orang-utans play video games too, and it can enrich their lives in the zoo
SBS, 15 March 2016
Orangutans play video games too, and it can enrich their lives in the zoo
ABC TV Good Game Spawn Point, 2 April 2016
Feature Story: Melbourne Zoo
ABC RN Radio, The Science Show, 23 April 2016
Orangutans take to interactive video games
The University of Melbourne, 2 May 2016
Video: Kinecting with Orang-utans
Herald Sun, 16 May 2016
Video: Melbourne Researchers Develop Interactive Gaming System for Orangutans
International Business Times, 16 May 2016
Watch zoo orangutans playing Xbox brain games
The Wire, 19 May 2016
Orangutans: The Future of Gaming?
SMH, 28 May 2016
Do apes have culture?
ABC TV, 19 July 2016
ABC TV Catalyst
Orangutans are highly intelligent animals and keeping them mentally engaged and challenged is paramount for Melbourne zoo keepers and researchers.
ABC News, 19 July 2016
Orangutans at Melbourne Zoo to be given more video games
By Alice Donaldson, Catalyst
8 September 2016
Engineering and IT PhD research on show for 3 Minute Thesis
The grand finalists included Sarah Webber from the Department of Computing and Information Systems and the Microsoft Centre for Social Natural User Interfaces.
SocialNUI Centre Manager