Riding The Polar Express to a digital pipeline: Sony Pictures Imageworks' first full-length animated feature takes advantage of mocap and IMAX 3D

Post, Nov, 2004 by Daniel Restuccio

CULVER CITY, CA -- There are those who will say that you have not truly seen The Polar Express until you've seen it in IMAX 3D. The Warner Bros./Sony Pictures Imageworks movie, based on Chris Van Allsburg's children's book, has been adapted for the big screen by Oscar-winners Tom Hanks (Best Actor) and Bob Zemeckis (Best Director).

On their journey to realizing the story of a young boy's renewed faith in Christmas, Hanks, Zemeckis, visual effects supervisors Ken Raltson and Jerome Chen, and the team at Imageworks have created an astonishing cinematic hybrid that blends the imaginative beauty of animation with the subtle feel of a live-action film. The startling realism of the movie in 2D achieves an uncanny presence in IMAX 3D. The film opened on November 10.

Hanks and Zemeckis felt a particular affinity to this beloved story that they had read to their own children at Christmas, No stranger to films with complex effects and animation, director Zemeckis envisioned the project as a live-action movie with the look and feel of the original oil pastel illustrations. Zemeckis contacted his Back to the Future colleague Raltson at Sony Imageworks.

The group spent months brainstorming and testing different approaches, including a short-lived, live-action version with a paint filter applied and a version that combined live-action and CG. Eventually they leaned toward a completely computer animated film but using the technique of motion capture.

"My first impression," says Chen, "was that this was going to be an amazing opportunity to create new techniques and technology and not be a traditionally-animated film."

MOTION CAPTURE

The technique of motion capture has evolved over the years as a cost-effective means of applying realistic motion to a CG model versus the labor-intensive approach of keyframing. In basic terms, the actor performs a specific action, in a defined space wearing a special skin-tight suit, with sensors that are read by infrared cameras that capture the positional X, Y and Z information of each sensor in that volume. However, in order to achieve the level of realism Zemeckis wanted, they would have to take that technology to a whole new level.

Enter Demian Gordon, seven-year motion capture veteran and mocap supervisor for The Matrix movies. In The Matrix. Reloaded and The Matrix: Revolutions, Gordon pushed the mocap envelope by presenting truly plausible digital doubles throughout the films. As mocap supervisor for The Polar Express he dropped the jaws of system provider Vicon when he conceptualized a mocap system with 64 cameras, almost three times the 24 he used on The Matrix.

Gordon built three mocap stages but mostly used the high-resolution space, a 10 X 10 cube, that could accurately capture the complex motion of up to four interacting characters. What makes this distinct from The Matrix system, says Gordon, is that they could capture both body motion and facial expression in 360 degrees at the same time.

Zemeckis was so impressed with the first tests, done with Hanks performing one of the characters, that the idea was hatched for the versatile actor to play multiple roles, including the lead role of the eight-year-old boy. Hanks would ultimately play five roles; the boy, the conductor, the boy's father, the hobo and Santa. In scenes where Hanks played the child-size hero, oversize props and set pieces were used.

So began the logistical adventure of shooting the entire feature in a 10-foot cube. As the mocap cameras faithfully rendered the actual movement of the actors. 12 reference video cameras simultaneously recorded all scenes on the stage. Most of those cameras were set as wide shot lock offs. Others were operated to get tight close-ups of the actors' faces. Co-cinematographer Don Burgess worked with Zemeckis on the basic blocking of the scenes, however none of these videos were used to determine the actual shot angles and framing for the movie.

THE INTEGRATORS

The Polar Express did start like a traditional animated movie with a Leica reel, an animated storyboard, cut by Avid Film Composer editors Jeremiah O'Driscoll and R. Orlando Duenas to the voiceover dialogue of the actual cast members back in May of 2002. However, unlike a traditional CG movie, this animatic was not refined into the final CG shots, but used more to budget the picture and as a writing tool for Zemeckis, who penned the script with William Broyles, Jr.

Once the script emerged, scenes were staged on the mocap stage. The editors took the video reference footage and built a "performance assembly," cutting together the actors' best takes into a complete version of the scene. For example, only when Hank's performance as the conductor was cut together with his performance as the boy, did the scene actually exist. Yet once assembled it existed in a 360-degree virtual world. After Zemeckis approved the performance assemblies, they were turned over to "the integrators."

The integrators' job was to find the corresponding mocap data for each performance, in every shot used in the scene, and then build a low-rez CG version of the scene, which was dubbed "Michelin Men" by Zemeckis. These motion-accurate, featureless, low-rez, stick figure models acted out scenes with an inset picture of their detached faces floating above them from the reference video. This is the point where the actual shooting of dailies begins.