Rise of the Planet of the Apes is just one of the movies up for a VFX Feature Film Award in our CG Awards 2012. Mark Ramshaw speaks to the team responsible for ensuring that Rise of the Planet of the Apes raises the CG bar once again
Building on its experiences with Gollum, Kong and the inhabitants of Avatar, Weta Digital proved it’s back in the lead digital actor business.
It’s in good company too – check out the other nominees:
The Avengers – ILM/Weta Digital
Battleship – ILM
Hugo – Pixomondo/Lola
Wrath of the Titans – Framestore/Method Studios/MPC
No animals were harmed in the making of this movie. In fact, no animals – at least of the ape variety – were even used for Rise of the Planet of the Apes, Twentieth Century Fox’s reboot of the beloved movie franchise.
Given a story that pits mankind as ignorant overlords, with apes as the oppressed, director Rupert Wyatt (best-known for sleight-of-hand prison breakout thriller The Escapist) was adamant that putting trained creatures before the camera would be untenable.
But while John Chambers’ prosthetics in the original Planet of the Apes films were groundbreaking, and Rick Baker’s man-in-suit effects worked well enough for Tim Burton’s 2001 stab at an Apes revival, the story here required a more naturalistic-looking range of creatures.
And so Wyatt turned to the undisputed experts in digital actors, performance capture and all things simian-related: Weta Digital.
Though notionally a step back from the milestone digital character work of Gollum, with neither Caesar nor any of the other apes in the film required to speak, the character challenges faced by Weta were actually far greater than on either the Lord of the Rings films or King Kong.
Gollum emoted, but was very much a fantastical creature. Kong was photoreal, but larger than life in terms of emotion as well as size.
The eyes have it. A young Caesar bonds with his surrogate father. The movie features Ceaser at various points in his life
Caesar needed to look like a chimp, yet act with great subtlety and embody both chimp and human-like qualities.
Crucially, he needed to be able to map out a huge character arc, from first steps to adulthood, that would underpin the entire movie.
Without ever uttering a word, the star of Rise of the Planet of the Apes had to be believable, understandable and utterly sympathetic, with the story essentially told from his perspective as he grows up, rebels, and ultimately leads his species in a revolution against mankind… And he needed to look good in fur while doing it.
With visual effects supervisor Joe Letteri once again at the helm, around 730 Weta crew members ultimately contributed to the film’s multitude of creature and environment-based effects shots.
But while the 18-month production schedule (with much of the effects work completed in a year) might be deemed generous for a typical show these days, the VFX demands for Rise of the Planet of the Apes were by no means typical.
“One of the biggest challenges was the sheer volume of what we needed to turn around in the time available,” says fur lead Nick Gaul. “Even Avatar felt more generously paced in comparison, because we had three years to work on it.”
This was also very much a milestone project for Weta Digital, with both the shooting method and the ultimate portrayal of the apes reliant on new techniques and technologies. In the case of the filming, Rise marked the first time that the studio’s performance-capture system would be used on a live-action set and in full daylight.
With The Lord of The Rings, Andy Serkis had to deliver his Gollum capture performance separately from the on-set footage, with facial details rotoscoped rather than automatically tracked using markers.
With Avatar, the capture technology was much improved, including head-mounted cameras to faithfully track facial motion, but the actors were still confined to a capture stage.
But for Rise, the director was able to shoot scenes containing both live actors and those playing the creatures, with Weta capturing the performance data simultaneously.
Models supervisor Marco Revelant says this improvement made a crucial difference to the way the directors, actors and digital artists worked together.
“Because we were capturing the data right there on set, Rupert had the ability to really define the movements and performance, and immediately have a strong idea of what he could expect us to deliver further down the line.
Obviously capturing on-set means you have less control of the environment and less room for mistakes. But conversely you’re going to get a better response from the actor working in the scene than from an isolated performance the second time around.”
Andy Serkis, Weta's mainstay, gives an excellent performance once again in what he calls his most challenging part to date
Andy Serkis once again donned a performance capture suit for Weta to help create the role of Caesar.
While it’s easy to assume that his time spent playing Kong provided good training, Serkis says the two characters are as distinct as Hamlet and Macbeth, with one a 25-foot-high lonely last of the species and the other an unknowing innocent who suddenly has to make tough choices to survive.
Describing the film as a domestic piece rather than an effects-driven action extravaganza, Serkis reckons playing Caesar from infancy through to adult revolutionary has been one of his most challenging roles to date.
The eyes have it. A young Caesar bonds with his surrogate father
While Caesar is undoubtedly the star, Revelant points out that it was by no means a one-ape show for the modelling team.
“Keep in mind that for Caesar alone, we have him as an infant, a toddler and so on before we get to the grown chimpanzee who features in about two thirds of the movie. We have a similar level of detail for the other hero models, including a gorilla called Buck, an ape named Rocket, and Maurice, an orangutan. On top of those, we had a number of second-level extras, with an average of four variations for each species.”
Despite the size of the digital cast, Revelant says they opted not to use a construction kit approach. “There are obviously ways to approach facial work where you have a kind of facial keyset, with noses, eyes and so on that you can choose from, but while we tried to keep some common topology, we choose to look for specific ways to capture nuances and individual details and add those by hand.”
When it came to building and rigging its cast of creatures, Revelant says that Weta was driven as much by a desire to stay faithful to real simian physiology as it was by production concept art.
“In the case of Caesar, we ended up working mainly from the original reference art for the face and head; then, once we went down into the body, we looked more towards medical reference, including MRI scans. We spent a lot of time building a realistic, working skeleton and creating a musculature system on top. We really wanted to get as close to the real deal as possible, so spent a very long time on it from a physiologically realistic point of view.”
Weta's character, Caesar displays unprecedented emotion and intelligence, using a new technology involving a performance capture suit
Wellington Zoo in Weta’s home town also proved a valuable source of reference, giving the modelling and animation teams the ability to study the appearance, movement and behaviours of the real creatures up close. At one point, one of the chimps at the zoo had to undergo surgery, and so Weta was afforded an even closer look.
“We took the opportunity of him being under general anaesthetic to take many close-up photos, getting a good look at the skin and fine detail,” says Revelant.
While King Kong was an impressive creation, clocking in at around 50 thousand quads, Revelant says that the cast of Apes are in another league entirely: “That was six or seven years ago, and a lot has changed, especially after taking a big leap with Avatar. I’d estimate Caesar’s face has the same number of polygons as there were in all of Kong.”
One of the payoffs of building such realistic creatures was the ability to assign a single rig to each creature.
This is particularly noticeable in the case of the adult Caesar, who was required to exhibit both ape-like and human qualities in many scenes.
“Because Caesar’s rig was naturally the most complex, we were able to use that to create derivatives for the other characters,” says Revelant. “Obviously the rig for species is slightly different, but by taking this approach, any changes made to the structure of the main rig could be easily propagated through to the others.”
It seems the similarity between ape species also extends to humans, with Weta finding it relatively straightforward to map performance capture data from the human actors’ bodies (albeit fitted with prosthetics to elongate the arms) onto the creature rigs. The facial performance data was another story.
“With the FACS system we’d used on Avatar, we knew how to activate the various facial muscles and controls to obtain any particular expression,” says Revelant.
One of the payoffs of building such realistic creatures was the ability to assign a single rig to each creature
“But with something like a chimpanzee, it’s not such a straightforward process. The mouth of a chimp is very flexible and able to do things a human’s can’t do, for one thing. And they use their muscles to express behaviours and emotions in a different way, so you can’t simply do a one-to-one conversion, activating the same muscle groups as on a human. Instead it’s necessary to do more of an indirect translation, to convey the same emotion that the actor is delivering.”
Think you've seen it before? Think again, Caesar is a revolutionary character in more ways than one
While many of the techniques used to create Caesar and his compatriots have evolved at Weta over the last decade, the system used to coat the creatures in convincingly styled and rendered fur coats is all-new. Code department supervisor Alasdair Coull began work on the bespoke toolset, dubbed Barbershop, shortly after the studio wrapped on Avatar.
Weta developed a new fur system for Rise
“Avatar itself wasn’t too demanding in terms of fur, so we were able to work with our existing technology, but afterwards we had the time to step back and consider the best way to approach it,” says Coull. “At the beginning, we were primarily working on it with Tintin in mind, but then once Apes came along, it was obviously going to have very different demands, so we really speeded up its development.”
While several articles have alluded to Weta’s involvement with computer science professor Eitan Grinspun, whose Columbia University team has researched the use of discrete differential geometry in simulations, Coull says the collaboration is concerned with ongoing research. Barbershop was developed wholly in-house.
“When we started on Barbershop I was working solo, although the team eventually grew to five,” he says. “I had done some fur stuff previously over at Framestore, but this was still a big learning curve. At the beginning, it was very much Marco and myself sketching up ideas, so everything was driven by the artist workflow.”
Coull says the approach they ultimately chose was completely different to that used on Kong. “We took a completely different perspective on what a fur system needed to be. With Kong, the approach was very much a procedural one, manipulating the fur with deformers. But the tools were, in a sense, quite limiting.
With deformers, you’re always one step removed. Plus, there was a very big learning curve for the artists. With the new software, we’ve gone for something that’s way more intuitive.”
A key challenge faced by Weta Digital’s technologists and artists was to find a way to build and groom a range of creatures that closely matched the concept artwork. “To match the artwork 100 per cent, we needed something that would allow for direct manipulation of the hair strands,” says Alasdair Coull
Marco Revelant says that the key distinction is that, rather than generating the hairs at render time, the artists are able to groom the fur in real time. “With most systems, the artists work with guide hairs, controlling the hair at a very reduced density. We wanted something that would enable the artists to direct any single strand if required.”
Barbershop, he continues, is very much an open system, as opposed to the ‘black box’ set-up previously employed.
“Making any changes required a huge amount of effort every time, whereas Barbershop allows us to access the data quite easily and then build on top of it. Even from a dynamics point of view, you can choose on the fly exactly how much to apply.”
Flexibility is Barbershop’s key benefit, but as well as giving the artist the ability to select the desired area and granularity of control, Alasdair Coull notes that Barbershop also gives a more accurate preview of what the finished render will actually look like: no more setting the guide hairs and waiting for the final render to see how they affect a full head of hair. The one downside, he says, is that all the hairs exist in 3D space through the whole pipeline – whether they like it or not.
“Even though there are ways for the artists to cut corners with the way they deal with such a volume of hair, it’s all still there on the creature. What you win in terms of artist control you do have to pay for elsewhere, and with Barbershop that area is data transfer. It’s not such a problem in terms of the on-screen preview, though. We’ve used OpenGL and GPU acceleration to make sure the artists can groom in Maya and get a real-time preview of how the final lighting will look, and even where the highlights will be on the hair.”
Revelant adds that preserving curve data for every single hair through the entire pipeline offered further benefits beyond the grooming stage. “It meant we knew where every hair was at any point in time and had every individual strand better applied in 3D space, which at the back end also came in useful for lighting and rendering.”
The fur shading on Apes represented a step forward from the techniques used on Avatar, with several different lighting and shadowing techniques used for various desired effects.
“While we still utilised deep shadows for small light sources with crisp shadows, when we needed to represent large area sources with soft shadows, we used image-based lights as well as a new addition, area lights,” explains Jedrzej Wojtowicz, head of shading at Weta Digital. “These allowed us to work in a more physically based way, for example when matching on-set light sources, scrims and so on. The area lights could utilise either of those shadowing techniques as well as PantaRay shadows.”
On Avatar, the Nvidia and Weta pre-computation engine PantaRay was used for quickly generating omni-directional occlusion, but here it was extended into generating shadows from area sources. Indirect lighting was also used where required, utilising a multi-bounce pre-compute technique that takes advantages of Pixar’s native RenderMan solution, but which is pipelined to suit Weta’s particular needs.
“For the skin shading, we utilised the quantised diffusion sub-surface scattering model BSSRDF [developed by Weta’s Eugene D’Eon and Geoffrey Irving],” says Wojtowicz. “We were able to represent higher skin and lighting detail at the surface, while it also allowed us to push the scattering depths deeper or shallower than we were able to before, without artifacts or energy loss.”
A key challenge faced by Weta Digital’s technologists and artists was to find a way to build and groom a range of creatures that closely matched the concept artwork. “To match the artwork 100 per cent, we needed something that would allow for direct manipulation of the hair strands,” says Alasdair Coull
The popularity of the original Planet of the Apes movies can be attributed, in part, to their fusion of science fiction with political allegory. Whether Rise of the Planet of the Apes pulls off the same trick remains to be seen. But what is clear is that Weta has created a digital character capable of engaging audiences far more effectively than could be achieved with man-in-a-suit prosthetics.
After the performance capture thrills of Avatar, it’s easy to forget what an achievement this is. Weta Digital has further broken down the boundaries between actors and directors on set and artists in the studio – and all within a post-production schedule of little more than a year.
Caesar is a revolutionary character in more ways than one.
Weta mainstay Andy Serkis again donned capture equipment for the underlying performance driving animation for the movie’s protagonist, Caesar. This time, he was able to perform on-set in full daylight.
Since Avatar we’ve also worked on the rigs to further minimise movement of the camera that captures the facial movements,” says Marco Revelant.
The ability to capture performances on set further breaks down the barriers between director and visual effects team. In addition to giving Rupert Wyatt the ability to direct scenes with actors present and organically reacting to one another, he was also able to use live previewing to get some idea of how the capture data would translate to the CG models.
It’s in good company too – check out the other nominees:
The Avengers – ILM/Weta Digital
Battleship – ILM
Hugo – Pixomondo/Lola
Wrath of the Titans – Framestore/Method Studios/MPC
If you liked this, you may also like reading about Weta’s work on Avatar.
You can read the full version of this article in the October 2011 issue of 3D World.
Posted on Thursday, July 26th, 2012 at 9:27 pm under Features, Making of. You can subscribe to comments. You can leave a comment, or trackback from your own site.
Tags: Avatar, CG animals, digital animals, Kong, Making of, Rick Baker, Rise of the Planet of the Apes, Weta, Weta Digital
The world's leading magazine for CG artists Subscribe Now
