Utrecht University Motion Capture and Virtual Reality Lab opens its doors

Virtual characters are becoming increasingly lifelike, which makes them more and more engaging to interact with. Utrecht University recently opened its Motion Capture and Virtual Reality Lab, equipped with the latest technology available in the field. It is the only lab at this scale in the Netherlands, and one of only a few worldwide. The lab focusses on academic research on virtual human technology. This line of research also has practical applications in the gaming industry, virtual and augmented reality, and other interactive applications where 3D virtual characters are being used.

Indispensable

The new Motion Capture and Virtual Reality Lab is equipped with the latest technology for capturing human motion. It allows for the detailed capture of several actors simultaneously, including facial expressions, body movements, finger tracking and audio. This is indispensable for the further development of increasingly lifelike virtual characters, according to Scientific Director of the lab Dr. Zerrin Yumak. Currently, most datasets available are limited, she says. They focus on either the body or the face. Detailed finger motion is often missing, as well as multi-party conversational data, and motions that have emotional or expressive content. Furthermore, 3D datasets that are constructed from 2D videos are not at a sufficient quality. With the lab, Zerrin and her group are now able to collect precisely the data they need.

The effect that applications have on their users, greatly improves when the digital characters become more believable

Impact

The primary goal of the lab is to make academic and societal impact in the field of virtual human technology. The aim is to develop novel algorithms for non-verbal behaviour synthesis, and provide an in-depth understanding of virtual human technology from a human-centric point of view. The researchers will work with the industry and societal partners to develop virtual humans that are plausible, trustable and personalized.

Although 3D virtual characters are becoming more and more realistic, they still come across as unnatural and strange. Yumak aims to make these entities more believable, and evaluate how humans perceive them.

In order to make the interactions with virtual humans believable, the tiniest forms of non-verbal signals are essential, she explains. We use our entire body when interacting. We make gestures, gaze, make subtle postural movements. And all these movements are related to one another.

Practical applications

There are a variety of practical applications for the lab's technical functionalities, and the research that is conducted there. The most obvious application being the gaming industry; whether for entertainment purposes or for health and education applications, such as for therapy or making teaching more engaging for children. Yumak: The effect that these applications have on their users, greatly improves when the digital characters become more believable. Another application area, especially after the Covid-19 pandemic, is online social communication and social VR. Recently, big companies such as Nvidia, Epic Games, Microsoft and Meta, started going into the direction of building extended reality worlds, Yumak says. The technology to create realistic interaction between humans and virtual humans in these 3D digital worlds is sheerly indispensable.