Sony Imageworks takes Spider-Man 2 to new heights: the challenge was to make this film's visuals even more believable than the original's

Post, July, 2004 by Daniel Restuccio

CULVER CITY -- Sony Pictures Imageworks (www.imageworks.com) gets to dazzle audiences for a second time with new-and-improved visual effects in Spider-Man 2. The challenge of topping the first film's $820 million world-wide box office is enormous and expectations are high. For the effects team, headed by visual effects designer John Dykstra, the goal is not to pack the movie with more gee-wiz effects but to refine the technique of delivering images that complement the depth and complexity of the story.

Two years have passed since the first Spider-Man movie. One of the major conceits of the first film was being able to fly with Spider-Man through the buildings and skyscrapers of New York. The richness and detail of those buildings, says Dykstra, play a big part in creating that immersive feeling of being there and flying with the web slinger, and that's been taken to a whole new level this time around.

"We improved upon the building texture-mapping technique in the second film," says Dykstra. Imageworks' artists "figured out new ways to stitch the tiles together in a way that made them much more flexible. You are basically shooting a building at one time of day, but that building has to be represented in the film at all times of day and into the night. So the texture maps have to be shot in a certain way to allow you the flexibility to change the apparent lighting on those textures and make it look like the building is real."

"We got a lot more geometric detail with the buildings, which allowed us to get a lot closer," says visual effects supervisor Scott Stokdyk. "We combined that with ambient occlusion lighting in both RenderMan and Mental Ray to get a global illumination look and feel."

They also got 18 new buildings for the New York City digital set, reports Stokdyk. "Our best quality foreground buildings from the last show were considered medium quality for use in Spider-Man 2, and were put behind our newer buildings."

Imageworks designers also improved the look on all their digital set interiors as well as practical set extensions for both day and night by using high dynamic range imaging techniques. Imageworks uses Alias Maya and Side Effects Houdini as the backbone of their CG pipeline and extends their functionality with in-house built toolsets and proprietary compositing software.

DIGITS AND LIGHTING

As in the first film, when Spider-Man swings from building to building he's CG, not a real stunt man. "We did reuse the Spider-Man model," says Stokdyk referring to the digital double created to blend seamlessly with the live-action Spider-Man. "We had one artist on the first show spend eight or nine months with the videotape of the stunt guy, just lining our model up and matching it up, sculpting it, and molding it."

To add to the flexibility of the CG characters in this film, CG heads of actors Toby McGuire (Spider-Man) and Alfred Molina (Spider-Man's nemesis Doctor Otto Octavius, a.k.a. Doc Ock) were built using data from the Light Stage device designed by Paul Debevec at the USC Institute for Creative Technology.

Debevec says, "For Spider-Man 2, the actor is lit from numerous different directions, with light from a special apparatus, and filmed from several viewing angles, which creates a complete digital record of how their face transforms incident illumination into reflected light. All of the face's coloration, shininess, self-shadowing and subsurface translucence is captured in this dataset. The dataset is a bit like a hologram, which records how something looks from many angles--our dataset records how a face looks under any kind of lighting."

By contrast, for The Matrix: Reloaded the universal-capture system used to create virtual heads of Keanu Reeves, Laurence Fishburne and Hugo Weaving works differently, he says. In that case, "the actor is filmed from several directions under one relatively diffuse lighting condition, where they use crossed polarizers to separate out the shininess of the face. The diffuse lighting yields a diffuse texture map for the face, and the way that the face reflects light is then simulated according to analytical models of surface reflectance and subsurface scattering."

The Light Stage/Sony system works differently, explains Debevec. "To create a rendering of the actor in a real or virtual set, the computer program can simply recombine the images of the face to simulate its appearance under any combination of lighting directions--the light from a real set, or the light envisioned by a lighting designer, or a combination of both.

"Algorithms implemented by Mark Sagar and his team at Sony Imageworks made it possible to animate the shape and viewpoint of the face based on several expressions of the actors, creating fully animated photoreal faces," he continues. "A particular advance was a novel rendering pipeline created at Sony Imageworks that allowed the digital lighting team to light the reflectance field faces and traditional CG elements using the same sets of traditional lighting controls."