When we compiled a list of 100 CG shorts you must see in issue 137 of 3D World, we wanted to cover everything. While at SIGGRAPH 2008, we were introduced to Emily.
Used to showcase the power of Image Metrics’ FaceWare software, the video clip of an entirely CG but incredibly photoreal young woman also acted to show Digital Domain the power of the software for use on Hollywood blockbuster The Curious Case of Benjamin Button.
The demo is so fascinating that we simply couldn’t fit into the magazine the whole conversation we had with Image Metrics’ product manager Nick Ramsey. Watch the video and then read what Nick had to say about the project in his own words.
We set out to recreate a photoreal human performance imperceptible to the common viewer: to leap across the uncanny valley. The demo was also created as a proof of concept for two reasons, one to be able to work with Digital Domain on the Curious Case of Benjamin Button and two, we wanted to prove that we could.
We aimed to create a demo to highlight the fact that our software was and is capable of driving photoreal results.
To date, most of our work had been focused around facial animation for video games, so we took the leap, and as you know Benjamin Button set a new high bar for achievable quality within our software.
To create the demo, the biggest limiting factors of our software were the simple inputs – a strong performance and a good character rig.
We knew we could capture a strong performance with Emily. Not only was she attractive to look at, she was easy to work with and eager to help us on our journey. Our biggest challenge was then how we were going to be able to create a character rig that could achieve the fidelity we were aiming for.
We partnered with Paul Debevec and his team at USC ICT to provide volumetric facial scans via his Light Stage structured light scanning stage. After we captured her performance and then received the scans from ICT, we embarked on a 4 month journey of discovery to make our leap across the valley.
Emily’s pipeline was completely different to anything we’d ever done before – mostly due to the fact we had never before built a character rig from structured light scans or shaded, rendered and composited a photo-real human performance.
Our one solid foundation we could rely on was the simple fact that we would leverage Image Metrics proprietary FaceWare software to produce the 90 seconds of animation.
We spent 4 months creating the believable movement for Emily—a stunning feat when you realize the quality of animation we were attempting to achieve. However, the FaceWare Software has come a long way since Emily, we have spent a lot of development time packaging up the tool for studios to use or license from us, it’s much better and faster today!
Matt Onheiber and Cesar Bravo, two of our senior animators used what we now call Faceware Software to create an extremely dependable animation basis that Oleg Alexander and William Lambeth leveraged.
Alexander and Lambeth, our respective rigger and modelers were then able to focus efforts on the rig, shading, lighting, rendering, and compositing portions of the project. While our team was very skilled in facial rigging, we found many challenges to overcome when we needed to light, shade, render, and composite Emily.
From an Image Metrics standpoint, we had two animators; Matt Onheiber and Cesar Bravo each spend about two weeks animating Emily. William Lambeth was our lead modeler and shader developer and Oleg Alexander was the projects’ creative director and lead rigger.
William and Oleg each spent about 14 weeks developing the rig, as well as developing our shading, lighting, and rendering pipeline. Several new techniques were developed over the course of the 16-week schedule with our research team in Manchester England, led by Kevin Walker and Mike Rogers. David Barton and Peter Busch oversaw the production management of Emily from start to finish.
Paul Debevec’s team at USC-ICT collaborated with William and Oleg to determine the best pipeline for Emily and the correct deliverables from the structured light scans.
By leveraging our powerful animation software coupled with a high-end photo real character rig built with the aid of volumetric scans in USC-ICTs Light Stage scanner. We broke the rig building process down into 7 main components:
1. Data Acquisition – this encompassed three main objectives: Video Performance and roughly 30 Light Stage scans. We did the video performance first to identify key expressions that we needed to cover in the 30 Light Stage scans. We also did a dental cast of Emily’s teeth and scanned it in via Light Stage. This was crucial for the quality we wanted to achieve.
2. Stabilization – we needed to stabilize each scan and calibrate them back to the neutral scan. We developed many automated techniques in order to optimize this difficult task.
3. Re-mesh/UV layout – in order for all the data to work in harmony, there were several steps for the optimal mesh density as well as consistent UV layouts per scan.
4. Clean up (part 1) – Eye and interior of the mouth needed to be cleaned up by hand – these are two areas that we think could still use improvement.
5. Map extraction – from the consistent UV layouts we could then kick out the correct displacement, normal, and color maps from the scans.
6. Clean up (part 2) – we needed to paint out the ink dots we used in step 2 calibration on each of the color maps. We also had to further clean up the displacements and color maps.
7. Split shapes and rig build – in our already strong facial pipeline, we had automated tools to split up each of the scans into localized regions of control. We then added our animator friendly GUI and finished out the rig build.
Emily was a viral experience for Image Metrics with a very positive reception, but there were negative comments (like anything viral). Internally we wanted to push ourselves to do better. Arguably half of our time on Emily was simply spent developing our theoretical pipeline that we had developed with our team in Manchester and with ICT.
The main aspect we’d like to have improved on was the area around the eyes, and the lighting and shading area as well. Having the luxury to further develop skin shading techniques could have strengthened the end product.
Many opportunities have come out of Emily – both from a business standpoint and from an internal development standpoint.
We all learned a bit about this process and what we could do better next time. We still continue to stay close with the team at ICT to develop future endeavours. Emily allowed our team to set the bar high for fidelity that is achievable within a CG environment.
We can now assimilate this knowledge to our entire client base to further push the envelope and tighten the gap on the uncanny valley. We are pushing the game engines to using our tech and pipelines to improve facial animation in many AAA Games. We also picked up a lot of commercial work that is seen by millions of people at home every day.
Related links:
Image Metrics homepage
The Emily Project page
Paul Debevec’s website
Digital Domain
Posted on Tuesday, November 23rd, 2010 at 2:46 pm under Features, Making of. You can subscribe to comments. You can leave a comment, or trackback from your own site.
Tags: Animated short, Emily, FaceWare, Light Stage, Paul Debevec, photoreal
The world's leading magazine for CG artists Subscribe Now
