News

Photo by: Cyrill Beeler

If you were one of the millions of moviegoers who contributed to the worldwide success of Avengers: Infinity War—the highest-grossing film of 2018—then you also got to witness the Medusa Performance Capture System in action. The team of computer scientists responsible for bringing to life characters like Thanos and the Hulk on the big screen was honored this year with a Sci-Tech Academy Award for the conception, design and engineering of Medusa.

Medusa, a comprehensive hardware and software system which has been developed, tweaked and expanded upon over the last 10 years, enables the precise digital replication of human faces, including detailed expressions and super fine physical details at high resolution. The Academy presented Sci-Tech Award certificates to the Medusa team, Thabo Beeler, Derek Bradley, Bernd Bickel and Markus Gross, at its Feb. 9, 2019, ceremony held at the Beverly Wilshire Hotel.

Credited for the initial concept, Gross, who is a professor of computer science at the Swiss Federal Institute of Technology Zürich (ETH), vice president for research at The Walt Disney Studios and director of DisneyResearch|Studios, worked with Bickel, then a doctoral student in his lab at ETH, to overcome this grand challenge in computer graphics: to create digital human faces that are indistinguishable from reality. Bickel is currently an assistant professor of computer science at the Institute of Science and Technology Austria. In 2017, he received the ACM SIGGRAPH New Researcher Award.

“Medusa is the culmination of many years of research on digital human faces and digital facial animation that we’ve been working on as part of the ongoing work in my lab and in collaboration with Disney Research,” says Gross, a longtime member of ACM SIGGRAPH and a 2012 ACM Fellow. “We got connected much more deeply with the arts and technology of special effects through our partnership with the Walt Disney Company. It gave us the insights to steer the research in such a way that we could make the best progress for advancement of facial technologies for film.”

Gross notes that at the time, researchers did not have a lot of success in bridging the so-called “uncanny valley”, which is a known phenomenon in the field that refers to the digital duplication of human faces that are not quite realistic, almost disturbingly fake in appearance.

With this Oscar honor, Medusa has staked its claim as an industry standard in special effects, achieving digital characters with highly realistic human features. This year, three out of the Oscar nominated films for best visual special effects used the Medusa system, and in recent years it has been used in numerous productions, including  Star Wars: The Last Jedi and Spider-Man: Homecoming.

“The methods are really tuned into highly accurate measurement of human faces. As such our technology is not generic per se,” explains Gross. “One of the insights we’ve had in exploring facial performance is that human facial expressions usually go from a neutral state into some deformed state and then go back to that neutral-rest state. There are cycles, and examining these cycles has allowed us to build methods to track the facial surface reliably over time. We further use the randomness of the pigmentation of facial skin as invisible landmarks to reference points across consecutive frames.”

Gross is also proud of the research and development that went into the analysis of facial microgeometry, such as facial pores, which eventually allows Medusa to capture the complex geometry of human faces including minute facial details. This is essential for re-creating realistic facial expressions.

Shortly following their early partnership with Disney, Gross and Bickel were joined by Beeler and Bradley, now both research scientists at Disney Research. Developing the 3D face scanner component of the capture system became Beeler’s master thesis at the time. When Bradley, then a post-doc, joined the team, he began working on the motion capture piece and helped develop the stable face-tracking technology. Together with Beeler, the duo invented the method for separating the rigid and non-rigid components of the face performance, and they have also spent a lot of time productizing the research into an artist-friendly system.

“Bringing this research into production entailed a lot of development work, but we got a lot of input regarding potential research topics in return, one of which ended up in a publication on rigid stabilization, published at SIGGRAPH,” says Beeler. “In this work we explore the relationship between the skin surface and the underlying bone structure to separate the rigid head motion from the non-rigid face deformation, an essential step when building facial rigs and an integral part of the Medusa system.”

In fact, all of the major milestones in the development of Medusa were showcased in SIGGRAPH papers. One of the team’s images had also been used on the front page of proceedings at SIGGRAPH 2011, for which Beeler was very proud.

“We’re constantly improving Medusa through new research, and simultaneously developing the next generation performance capture technology,” says Bradley. “We can’t say much, but the future is very exciting in this field.”

This Sci-Tech award marks the second Academy Award for Gross, who won a Technical Achievement Award in 2013 for the technology that more efficiently simulates smoke and explosions in films. For Gross, this current award win is very special as it both rewards years of academic research and marks the first recognition of this kind for Disney Research.

“We worked on Medusa for literally 10 years, and I’ve been working on digital human faces since I was a post-doc, which has been some 30 years,” says Gross. “It was a beautiful experience to get recognition from the Academy for all of this work.”

Still, there is more to come. “I often compare my work to a rabbit hole,” says Beeler. “We are making great progress but the deeper we go and the more we solve, the more we realize that we are far from done. My ultimate goal is to provide technology to digitize humans holistically, with minimal effort and at maximal quality—and every day we are getting a step closer to this vision.”

By Melanie A. Farmer

ACM SIGGRAPH has announced their annual award winners including an inaugural award for the ACM SIGGRAPH Distinguished Educator. Winners will be honored at the SIGGRAPH 2019 Opening Ceremony and Awards Presentation on Monday, 29 July, 9-10:30 am, West Hall B at the Los Angeles Convention Center.

ACM SIGGRAPH 2019 Awards

The Steven Anson Coons Award
Michael F. Cohen: For his groundbreaking work in numerous areas of research—radiosity, motion simulation & editing, light field rendering, matting & compositing, and computational photography.

The Computer Graphics Achievement Award
Denis Zorin: For fundamental contributions that have advanced the fields of geometry processing, multiresolution shape modeling, and geometric principles of physics-based simulation in graphics.

The Significant New Researcher Award
Wenzel Jakob: For his work in rendering and geometry.

The Outstanding Doctoral Dissertation Award
Lingqi Yan: For a unified comprehensive view of visual appearance modeling for computer graphics rendering.

The Outstanding Service Award
Jackie White: For her long term excellent dedicated service ACM SIGGRAPH.

The Distinguished Artist Award for Lifetime Achievement in Digital Art
Donna Cox: For pioneering work in the art of scientific data visualization.

ACM SIGGRAPH Practitioner Award
Stephen Hill: For multiple contributions to the field as a practitioner, including the design and implementation of ambient occlusion and hierarchical visibility systems.

ACM SIGGRAPH Distinguished Educator Award
Andries van Dam: For his impact on research practice in education as it relates to graphics and interactive techniques, cumulative contributions to the field, innovation in education, influence on the work of others, and being active in the ACM SIGGRAPH Community.

ACM SIGGRAPH Academy

• Frederick Brooks for pioneering work applying scientific rigor to virtual reality, and applying virtual reality to scientific research.
• Marie-Paule Cani for contributions in implicit surfaces, physical simulation, sketch-based interaction, and expressive modeling, and for leadership in the graphics community.
• Donna Cox for pioneering work in the art of scientific data visualization.
• Markus Gross for contributions to  point-based graphics,  3D capture and video technology, and physics-based animation, and for founding an influential industrial research laboratory.
• Dinesh Manocha for contributions to geometric modeling, GPU computing, interactive rendering of large complex scenes, and interactive sound simulation.
• Ravi Ramamoorthi for groundbreaking  theoretical work in mathematical representations of visual appearance, and for translating these into computational methods with wide practical impact.
• Hanan Samet  for founding, developing, and authoring the definitive texts in the field of storing, processing, analyzing, and retrieving  spatial data.
• Denis Zorin for fundamental contributions that have advanced the fields of geometry processing, multiresolution shape modeling, and geometric principles of physics-based simulation in graphics.