Fuel VFX shares its CG secrets on Avengers Assemble

Kulsoom Middleton | Features | Making of | 21/06/2012 12:06pm

Fuel VFX delivered 150 visual effects shots for the latest blockbuster from Marvel Studios. We caught up with Fuel VFX’s CG supervisor Mike Bain and executive producer Jason Bath to ask them how they created the Loki-Thor fight sequence and more…

Sydney-based creative house Fuel VFX looked after three sequences that occur in and around Tony Stark’s New York penthouse, perched atop Stark Tower.

In a scene near the film’s opening, Tony and Pepper are celebrating the tower coming online when they are interrupted by Agent Coulson from S.H.I.E.L.D. Fuel designed and created the various holograms featured in this scene which give an insight into some of the other members of the Avengers team, as well as the mysterious tesseract artefact.

The penthouse set required sweeping views of New York to be added behind Stark's floor to ceiling glass windows, for both night and day looks in two different scenes, as well as extension to parts of the interior set

Fuel’s work can also be seen in the Loki-Thor fight sequence that takes place on the floating balcony outside of Stark’s apartment during the battle to protect New York. Read the interview below to find out how the CG was created…

Read the interview

3D World: How did Fuel VFX get the gig to work on The Avengers? Did you have to work with any other creative studio in order the produce your shots?
Jason Bath: We’ve been working with Marvel for a few years now and Avengers is our fourth film in a row with them so it’s a great relationship for us. It was really good to work with Janek Sirrs (VFX supervisor) and Susan Pickett (VFX producer) again who we got to know on Iron Man 2.

The New York City cyc asset was required in all three of our main sequences and that was supplied by ILM. However, the way we needed to use it was a bit different to their usage so we had a bit of work on our end to adapt it – mainly additional work on the Chrysler building as it was quite prominent in our shots and adding life to the cyc.

You can read about ILM and Weta Digital’s VFX work on the Avengers Assemble in the July issue of 3D World

3D World: What was your company responsible for?
JB: Aside from the scenes mentioned above, we did some extra bits and pieces – the CG Stark Jet, which was a re-use of the Fuel design used in Iron Man 2; the Security Council monitors; and some shots at the end of the film to integrate Loki into the smashed concrete floor after the Hulk has pummeled him.

Fuel designed and created the various holograms featured in this scene which give an insight into some of the other members of the Avengers team, as well as the mysterious tesseract artefact

3D World: What 3D software was used to create this work?
Mike Bain: The software we used on Avengers was pretty much out of the box. Tracking was done in SynthEyes and 3D Equalizer. All the CG work was done in Maya and rendered with Mental Ray: the CG Stark Tower, the top-up set extensions inside Stark’s penthouse, and all the 3D elements in the holograms.

All compositing was done in Nuke along with any cyc projection work. After Effects and Photoshop were large contributors to the Stark holograms.

3D World: What was the biggest technical challenge?
MB: The Loki-Thor fight sequence was the trickiest and most technically challenging. There are actually about 60 shots in the sequence and there were a lot of different elements to it:
- CG Stark Tower set extension
- New York city panoramic cyc
- Reflection replacement in practical glass
- CG scepter replacement
- Scepter blast FX; hammer/scepter clash FX
- BG alien chariots
- (Not to mention having to pull keys through Thor’s hair!)

There was a lot of camera and object tracking in the sequence and some of it was quite tricky so both SynthEyes and 3D Equalizer were used depending on what gave us the best solve.

But getting the Stark Tower set extension looking right was the biggest challenge. We quickly discovered that the partial set-piece that was built for in-camera use wasn’t a good fit with the CG designed version. This meant changing the design of the CG version somewhat as well as taking the plunge to replace more of the set than we’d originally intended so we could minimize the tricky areas where the set needed to join the CG.

For the final scene the New York cityscape needed to be added, along with FX animation from Loki's sceptre, and background CG chariots of the attacking aliens

We’d had a similar experience with our CG Burj Tower work for Mission Impossible: Ghost Protocol (in the sequence where Tom Cruise climbs the building) so we’d dealt with this issue before. The difference on Avengers though, was that any changes we needed to make to our model would affect another vendor as ILM also had Stark Tower in their shots, albeit wider framing than ours.

That meant any changes we needed to make to the design or model of the tower also had to factor in the implications for them. So there was a bit of back and forward with their supervisors and digital matte painting departments, and Janek, to make sure that we were all sitting in the same place.

But it was probably the lighting that was the trickiest aspect. The live action was filmed on a blue screen inside a sound stage with only the actors and a partial set piece, but in the final film the fight takes place outdoors, high above New York City. So we were fighting the studio lighting a bit – the plate photography was telling us one thing, but we needed to make that look like something else. To complicate matters, our shots also had to seamlessly intercut with ILM’s fully CG shots.

Fuel worked on some shots at the end of the film to integrate Loki into the smashed concrete floor after the Hulk has thrown him around like a ragdoll - one of our favourite bits of the film!

Janek was really on top of this and worked closely with us to push the live action grade and our CG lighting in one direction and kept the relevant ILM shots in line as well.

Then it was just a matter of getting all our textures on our CG Stark Tower suitable for close-up use and continuing to liaise with Janek and ILM about the look of the glass – it’s colour, reflectivity, the level of distortion of the reflections – all of that needed to be consistent.

3D World: How many people worked on each sequence
JB: The crew tends to cross-over between sequences as needed but we had about 50 people work on The Avengers throughout five months of production.

3D World: What was the most useful piece of software/kit that you used and why?
MB: I don’t think there was one piece of software specifically, but the combination of a few that made production smoother for us.

SynthEyes and Equalize were essential for tracking, Maya for 3D and Nuke for 2D. Our suite of shots were really multidisciplinary and required tracking, modeling, texturing, shading, lighting, FX, extensive roto and keying, re-grading and more.

Photoshop, After Effects and Maya were also used for the holograms, and we couldn’t have achieved any of those effects without them.

Using the multi-channeled EXR output through Maya and Mentral Ray for lighting enabled us to get through our shots more quickly, and made it easier for us to review our work – especially for the Loki-Thor fight set extensions and CG scepter shots.

3D World: Can you think of where a specific part of the pipeline was useful in fixing or achieving something in particular?
MB: From a 3D perspective, Maya enabled us to do a lot of the work in one application, from designing 3D elements, to building and rendering set extensions and doing complex FX work.

3D World: What was the most interesting shot to develop from a technical angle?
MB: Probably the Loki-Thor fight sequence for the challenges described above.

Maya enabled Fuel to do a lot of the work in one application, from designing 3D elements, to building and rendering set extensions and doing complex FX work

3D World: Can you talk us through the Loki-Thor fight sequence and explain how it was created?
MB:

The live action shots are put into camera tracking with the idea to use the CG Stark Tower geometry to match up to the partial set. This is where the differences between the two were discovered in detail and we needed to work with Janek and ILM about how to address that
Where needed, we brought in our adjusted models of the Stark Tower to ensure we could matchmove all shots accurately
Loki’s practical scepter was an incomplete, flexible version for stunt purposes so it needed to be replaced with a CG version. So an object track of the stunt scepter was also undertaken – sometimes requiring hand tracking due to the combined motion of the handheld camera and wobble of the stunt scepter. The stunt prop was painted out and replaced with the CG scepter complete with glowing jewel. There was a little bit of look dev work getting the different textures on the scepter shaft and blade rendering nicely and accurately. Then getting the scepter behind Thor’s hair in a number of close-up shots proved quite labour-intensive
Once we received the geometry of the environments, we lit them with a combination of CG lighting and high-dynamic range, on-location lighting which ILM had provided
Using assets supplied by ILM we blocked out the BG chariot animation
Stark Tower was rendered with a bunch of additional passes so that the Nuke compositors could see the final look for the set extension
In the meantime, our FX department was developing particle effects in Maya for the scepter blast, and also a spark-like effect for when the hammer and scepter clash. Once the look was approved these were rendered in Mental Ray for the required shots
The Nuke team were comping test renders along the way to give us feedback as well as getting the New York cyc sitting in each shot as a priority. There was a lot of on-set glass so there was a lot of paint out and 2D reflection replacement, as well as some tricky re-times to occupy them. They soon turned their attention to pushing the lighting in the plates in a certain direction, as per Janek, to try and better capture the outdoor lighting required. This of course had implications for our lighting of Stark Tower and we needed to adjust our rendering a number of times – especially because when the look of sky changed in the cyc, it had a significant impact on the look of the glass in the tower
The Stark Tower renders also had some look dev/art direction regarding the number of clouds reflected in the glass and the level of distortion in those reflections
Then everything went to comp basically, and for them it was a combination of making the revised plate grades work with the 3D

Thanks to Mike Bain and Jason Bath for the interview.

The making of Prometheus

Fuel VFX also contributed to the VFX in Prometheus: Find out how Fuel VFX, MPC and Weta Digital helped director Scott Ridley to terrify the world once more in his long-awaited sci-fi blockbuster in the new issue of 3D World.

And for more on the VFX in Avengers Assemble, see the July issue of 3D World – the one with the Hulk on the front cover.

To see more work from Fuel VFX visit its website

Coming soon online: Hoxton Redsox’s Happy Birthday David viral teaser for Prometheus

Cool 3D art: The Hulk from Avengers Assemble

Posted on Thursday, June 21st, 2012 at 12:06 pm under Features, Making of. You can subscribe to comments. You can leave a comment, or trackback from your own site.