Kinecting with orang-utans

Digital technology presents significant opportunities to support Zoos in their goals of animal welfare and enrichment, and researchers from the Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) at The University of Melbourne are excited to be collaborating with Zoos Victoria to explore how Social Natural User Interfaces (NUIs) can improve animal welfare and a better understanding of the way orang-utans learn and interact with technology.

Designing and developing digital enrichment for orang-utans

Orang-utans are highly intelligent species critically endangered due to deforestation of their natural habitat for palm oil plantations. Through a study of the design and application of this digital enrichment we are seeking to better understand the zoo as a context for human-computer interaction, to understand and develop methodologies for designing for non-human users, and study how digital enrichment can improve the welfare of Melbourne Zoo’s 6 orang-utans.

We applied ‘user-centered design’ methods from the discipline of Human-Computer Interaction (HCI) to approaching the development of enrichment for orang-utans. This involved interviews and workshops with animal welfare experts, Orang-utan Zoo Keepers and the Zoo’s Interpretations team to identify what the orang-utans at Melbourne Zoo might be interested in and respond to, and what the opportunities are for NUIs in this space.

Early on in this process, we were able to observe the existing ways Melbourne Zoo was utilising technology with the orang-utans. As well as projecting movies into the enclosure, the Keepers have explored the use of tablet computers to provide enrichment in the form of puzzles, games, musical instruments and painting. While this shows the potential for digital enrichment, it is limited as the Orang-utans are incredibly strong, they can only interact with their fingertips through metal bars while a Zoo Keeper holds the tablet outside the enclosure. Consequently, our project utilises NUI technologies to give the orang-utans a wider range of ways to interact with technology by offering more active, full body movement for orang-utans.

Our system projects a screen onto the floor of the orang-utan enclosure, and by using a Microsoft Kinect sensor, we turn this projection into a giant touch-aware screen. Because the Kinect detects the orang-utan’s movement at a distance, all this technology can be activated from outside the orang-utan enclosure by the research team.

orang-utan leaning on her knuckles and looking down

On this system we have developed a variety of applications that aim to explore and demonstrate enrichment opportunities. This includes mental challenges through games, to encourage play, puzzle solving and exploration, as well as a painting app to facilitate the orang-utans natural creativity. During February 2016, these applications will be trialled with the orang-utans to see how they approach and use the technology that may offer opportunities to a multitude of cognitive challenges to keep their minds and bodies active.

One of the great stories we learnt in this design process was that Kiani (Suma), Melbourne Zoo’s 37 year old female orang-utan, loves looking at photos of herself. This is an example of how intelligent the orang-utans are, and something that connects with visitors as similar human traits. Until now Kiani has not been able to choose what photos she looks at, just what photos people choose to show her. Consequently the team has developed an application that shows a gallery of photographs and videos that the orang-utans can browse; select photos (to make them large for a short length of time) and interact with (creating water-ripple effects). By giving Kiani and the other orang-utans control over what photos and videos they view, we hope to gain insights into what kinds of programs each orang-utan prefers to use.


This project is a collaboration between The University of Melbourne’s Microsoft Research Centre for Social Natural User Interfaces (SocialNUI) and Zoos Victoria and proudly supported by Microsoft.

Project team


The Age Newspaper (Print), 2 February 2016

Orang-utans get smart with technology