Walt Disney’s Hollywood Smiles with Natalie Woods

Disney has an undeniable penchant for innovation and invention. Since 1923, Walt Disney and his empire have been forging the way in the digital and media-based world.

“I started, actually, to make my first animated cartoon in 1920. Of course, they were very crude things then and I used sort of little puppet things.” — Walt Disney

The Disney Research Labs, part of the Walt Disney Co., provides “the science behind the magic.” But, before Disney officially went down the route of even having a research lab it was responsible for a number of firsts including;

• The first cartoon with fully synchronized sound (1928).
• The first full-colour cartoon (1932).
• The first animated feature film (1937)
• The first modern theme park (1955).
It’s no wonder that they have continued that legacy.

“We keep moving forward, opening new doors, and doing new things, because we’re curious and curiosity keeps leading us down new paths.” — Walt Disney

In fact, Disney opened Epcot Centre in 1982 to celebrate human achievement, specifically technological advancement and international culture.

The park receives around 12 million visitors every year and is represented by ‘Spaceship Earth’, the geodesic sphere featured on the front cover of this issue of DentaltownUK.

At the beginning of Epcot’s ‘time machine’ ride, a camera takes pictures using facial recognition technology. The images are then used at the end of the ride in a presentation about the future of technology which features the riders’ faces on animated characters!

“Animation offers a medium of story-telling and visual entertainment which can bring pleasure and information to people of all ages everywhere in the world.”— Walt Disney

Per its website, Disney Research aims to “drive value for The Walt Disney Co. by delivering scientific and technological innovation. ... Our world-class research talent invents and transfers the most compelling technologies enabling the Company to differentiate content, services and products.”

So why are we talking about Disney in a dental magazine? Well, after years of achieving, creating and inventing in the media world, it has set its sights on something a little surprising: model-based tooth reconstruction!

The basis of the project is to create an algorithm that successfully allows Disney to re-create an individual’s teeth from basic images or a short video. The main reasoning for this at the moment is to allow for more realistic facial reconstruction in films and video games.

“I try to build a full personality for each of our cartoon characters—to make them personalities.” — Walt Disney

You could create the most accurate facial construction of an actor, but if his teeth aren’t quite right it could alter the most basic of facial expressions and would just never look quite right.

The demands for realistic and accurate anatomy on 21st-century animation for certain films and games is evolving over the years and building on its original concept of ‘unnatural but seemingly natural’.

“Animation is different from other parts. Its language is the language of caricature. Our most difficult job was to develop the cartoon’s unnatural but seemingly natural anatomy for humans and animals.” — Walt Disney

There already are successful reconstructive methods for other aspects of the face, including eyes and hair, but oral reconstruction has yet to be widely explored.

So far, the majority of efforts to create reconstructive software for the mouth have come from the world of dentistry. Disney has explored currently available oral scanners like the 3M True Definition Scanner, the iTero Scanner and the 3Shape Trios Scanner. While they yield high-quality images, Disney found that there were limitations for requirements related to cost and time, and that the devices would require dental professionals to arrange a specific appointment to take the scan the celebrity or actor. Therefore, Disney endeavored to come up with its own solution using photographs.

“Of all of our inventions for mass communication, pictures still speak the most universally understood language.” — Walt Disney

Thabo Beeler, a research scientist at the Disney Research Zurich lab, kindly agreed to explain more to DentaltownUK and answer some questions for us.

DentatownUK honorary editors Elaine Halley (dentist and owner of Cherrybank Dental Spa) and Andrew Tawse (dental technician and owner of 4D Dental Laboratory) collaborated with DentaltownUK before we spoke to Thabo, who heads the Capture and Effects group. His primary research focuses on capture, 3-D geometry reconstruction and appearance acquisition. Thabo was a member of the team that created the algorithm for Disney’s model-based teeth reconstruction; he stated that the project is now targeted toward films and video games but told us why his team chose to pursue the re-creation of teeth.

“At the Capture & Effects group of Disney Research we are working creating more realistic digital characters for films, video games and other mediums,” Thabo said.

“Over the past few years we have been particularly focused on the face and have produced systems that can capture the geometry and deformation of skin, the structure of facial hair and also the intricacies of eyes at very high fidelity. Working on teeth has been a natural project following this tradition.”

Thabo went on to say: “Our core focus was to be extraoral since we don’t want to put our actors through the invasive procedure of intraoral scanning, and since we’re concerned with entertainment purposes, we’re interested in faithfully capturing only those parts that will be visible on the screen.”

Disney’s algorithm relies on several factors, including the shape, pose and scale variation of each individual tooth. It requires minimal user interaction and can operate on a set of uncalibrated images or even a short video clip. Images and videos taken on both cameras and handheld devices like an iPhone can provide a suitable basis to create the model, which was proven to yield strong results.

Because dentists spend a lot of time standardising and calibrating photos, Halley was especially intrigued by this aspect. She wondered, ‘How many images at a minimum are needed, and would you need a series of expressions to capture the gum line?’ While Thabo admits to it not being investigated thoroughly (which is fair considering the project is still being worked on), the team has experimented with reconstructing teeth from even a single photograph and managed to get plausible results. This is still incredibly impressive.

“Times and conditions change so rapidly that we must keep our aim constantly focused on the future.” — Walt Disney

Although it doesn’t always have a 100 percent success rate, the project is currently based on statistics and hypothesis, which we will explain further. With more tweaks and research, this can only improve.

To read the full research paper on the algorithm and processes at Disney Research Labs, visit https://goo.gl/lWLIAV. Here’s a quick summary:

To begin, the team collected a database from 86 high-resolution plaster casts of oral 3-D scans of a variety of upper and lower teeth. They also created individual tooth template meshes, categorised into molars, premolars, canines and incisors. They modeled these meshes all the way to the root of the tooth to give any reconstructed model a solid base.

This is then combined with the data gained from a provided image of the teeth to create a reconstructed model. The image is used to detect the shape and boundaries of the teeth. The outline of the teeth is taken and combined with the template mesh and the database to calculate where the teeth would naturally and most likely be positioned.

This also allows the algorithm to identify the positioning of teeth that may not be included in the image (usually molars). The use of sparse image data can be justified and worked around, should it be required, to create an accurate reconstructed model.

Thabo summarises: “Our system consists, broadly speaking, of two main steps. First, it estimates the teeth boundaries from the images using a machine learning-based feature classifier. Second, it optimises for the teeth shape and pose that best fit the extracted features while being faithful to the statistical distribution of the prior model.

“The algorithm will hypothesise the most plausible teeth where they are missing or invisible given the available visible teeth. This is typically the case for the molars, which are mostly occluded, but should a person be missing a front tooth the system would suggest a suitable exemplar given the prior model. We show examples and verifications of this in our paper.”

Thabo said that during the project, his team consulted with dental professionals to create the algorithm. We asked Thabo if the scanner and algorithm could identify things such as discolouration, misalignment or malocclusions.

“Our algorithm relies on the statistical prior to inform on what are valid and probable teeth configurations and what configurations are unlikely to occur,” he said. “Currently the prior has been constructed from healthy teeth situations, but one could easily include pathological cases in addition to expand the expressiveness of the system.”

He went on to mention the colouring of the imaged teeth, saying: “While this was not the focus of this work, we do compute the color from the images. However, teeth appearance is much more complex than simple coloring, and acquiring the complete properties of teeth is outside the scope of this work.”

While team members currently don’t interfere with or manipulate the results of the algorithm, Thabo explained that any corrections required could be easily done.

’Smile Imaging’ (smileimaging.co.uk) in fact already provides a computer software programme that does exactly this. It was developed as an effective way for dentists and team members to speedily produce imaged photos of patients, enabling them to see how their appearance will improve with the proposed treatment.

Being able to show cosmetic dental and nonsurgical facial procedures with imaged photos within 5 minutes has obvious advantages when describing to patients the possible outcome of any proposed treatment; this is explained in more detail by Elaine Halley in her ‘Dental Imaging’ article in this issue.

Looking to the future, Disney would like to branch out into additional parts of the mouth and mouth cavity, such as lips, the tongue and the wall of the cavity. Team members feel this would allow them to create more complete reconstructions of the face and also allow them to re-create the movement, collisions and contacts between different areas of the mouth and teeth.

Whatever the future holds for Disney and its venture into our world, we’ll watch its progress with interest for potential applications within dentistry.

We’ll leave you with one final musing from Disney himself:

“We have created characters and animated them in the dimension of depth, revealing through them to our perturbed world that the things we have in common far outnumber and outweigh those that divide us.” — Walt Disney

Natalie Woods