LazyTown’s pioneering mix of live action, puppetry and animation has proved to be a major technological challenge. For everyone involved in capturing, managing and editing the HD footage LazyTown has become a by-word for pushing the boundaries.
The workflow has evolved from Series One to Series Two but during the shoot of the first series there were a number of significant bottlenecks to overcome.
The production shot full resolution 10 bit RGB 4:4:4 at 24p (1920x1080, 23.976fps)
on a Grass Valley Viper FilmStream with live keying from an Ultimatte HD engine. Aside from the live keying and virtual set, the production route was conventional. Clips were stored on disc, edited offline before finishing, conforming and output to tape. Yet throughput of each episode’s average of 10 terabytes of data (combined with four or more additional episodes stored in the pipeline at various stages of completion) was causing severe problems. The schedule called for five day shoots, up to 30 daily set-ups and an eight week per episode turnaround.
“It was taking up to 24 hours to conform an episode.” says Executive Producer, Raymond Le Gué. “Then if Magnus Scheving (the creator) wanted to change even one frame of the output we had to re-conform. The pipeline was very shaky. We had the quality but we were pushing way beyond capacity in terms of throughput. We had no flexibility.”
Le Gué, a virtual set pioneer who created the virtual set system Scenario-XR and produced over 2000 hours of virtual set programming for Endemol, joined LazyTown two months after the start of Series One and was immediately tasked with solving the bottlenecks for Series Two’s set of 18 episodes.
The first decision was straightforward – keep the Viper and the Ulimatte keyer. “It was taking 20 minutes to swap between the virtual and the hard sets, so to cut that down we acquired a second Viper,” says Le Gué. “The challenge was to enable Magnus to take creative decisions on the fly and then to manage the throughput of the DPX files while keeping to schedule.”
A Dual Workflow Le Gué explored system scenarios with a number of manufacturers before selecting Avid®. Working together LazyTown and Avid arrived at a solution which called for a dual workflow. This integrated existing key equipment such as the Viper HD cameras and Ultimatte engine while introducing an editing process at the point of acquisition. To do this Avid demonstrated the capabilities of the Avid DNxHD™ codec to Le Gué.
“We tested the quality of uncompressed HD with Avid DNxHD and found it so good that we could finish the material within the edit environment for TV purposes and keep the raw footage as source material for the 400-600 VFX shots.” says Le Gué.
In the revised workflow the Viper’s 4:2:2 stream is sent to an MCR-type room adjacent to the set, where it feeds a Media Composer® Adrenaline™ HD system for immediate logging and editing before recording onto a 15 terabyte Avid Unity™ MediaNetwork system. The other, dual 4:4:4 stream is recorded to a DVS San with 30 terabytes storage but maintained as DPX files. Rushes are backed up on data-tape.
“Pre-editing is one way of limiting the amount of material we’re processing,” says post production supervisor Paul Boots. “Each frame is 10Mb so every frame we trim and don’t store is vital. We’ve probably saved a third of our storage space this way.” It also meant that the director – usually Scheving himself – could immediately see the results of any changes on set. Moreover a rough cut of each episode would be available by the end of the week’s shoot, speeding up the post process.
The live keying not only accommodates on-set interactivity, it also means there is far less matching of shots in post. The thousands of elements comprising the show’s virtual backgrounds are generated live by the XR-Gen4 virtual set system. When Scheving wants to change a graphic, such as recolouring the sky or repositioning a tree, the production can make the adjustment on the fly and on the floor while preparation for the shot continues. Shooting of the adjusted scenes takes place within minutes.
The camera head and crane are motion controlled with each axis encoded so the movement (focal length, tilt and camera height) are matched exactly to the backgrounds. “Whenever the camera and backgrounds match the animatic we accept the shot and send the data to the VFX department,” says Ultimatte Operator Richard Welnowski. Additional code has been written which splits the metadata into chunks for more efficient management.
Online and Offline The offline project from the Adrenaline system is stored on an Avid Unity at HD quality in DNxHD for access by operators using the Symphony™ Nitris® system. “The beauty of the system is that we can work offline or online at any time,” says Boots. “We’ve cut promos on the Symphony Nitris system without touching the DPX.” Colour correction and conform of the graphics files are performed on the DS Nitris which has the ability to manage DPX data. Graphics are also swapped as Avid DNxHD files to the DS Nitris system within a Quicktime wrapper.
“Colour correction itself is fairly simple because we’re dealing with a primary colour palette,” explains Welnowski. “We adjust the contrast and brightness a little in post and tweak the gamma. The main thing during capture is to ensure that the pink, blue and yellow, which are LazyTown’s signature colours, stand out.”
LazyTown’s complex audio requirements also posed significant challenges. As well as blending thousands of sound effects in each episode with dialogue and song, LazyTown also wanted to synchronise audio and video editing so that both could be done at the same time. They chose the Digidesign® Pro Tools| HD® system for the audio post production side of the project, due to its unparalleled integration with the Avid systems like the Avid Unity Media Network.
However simultaneous synching of audio and video editing is no easy matter. The Pro Tools HD systems works with standard definition Avid picture so a transcoding workflow was designed for the audio suites using Virtual Katy’s VK2 product. The tool works by comparing two picture cuts and finding the changes between them, then re-conforming an entire Pro Tools session in accordance with the new cuts, saving huge amounts of production time.
Despite all of these modifications the pipeline is still being pushed to its limits. Avid® Interplay™, Avid’s nonlinear workflow engine which was under development at the time that LazyTown’s decisions were being made, might be integrated for future productions to ensure a smoother tracking of assets, possibly to log clips (currently this is done manually) and recall the vast library of footage. “LazyTown has an extremely luxurious 11:1 shooting ratio, all of which is stored as DPX files,” says Le Gué. “Magnus might want to use a stunt we shot and rejected for Series One, episode 24. Any delay in searching and playing that material is costly to us.”
“It’s the most ambitious project I’ve been involved with,” says Executive Producer Raymond Le Gué. “What we’ve done, I believe, is unprecedented in terms of integrating film craft processes with television. We’ve brought post-production into the pre-production stage so that it’s even more deconstructed than normal TV production.”
| From Director’s Vision to Commerical Reality |