DreamWorks Animation has produced some of the most popular animated movies of the last 25 years, from Shrek and Madagascar to Kung Fu Panda and How to Train Your Dragon. As an early mover in 3D animation and one of the first challengers to Pixar, DreamWorks requires powerful hardware – and lots of it – to stay competitive in terms of the length and quality of its productions. DreamWorks’ latest film, The Croods: A New Age, used nearly 200 million compute hours from start to finish. Recently, the company hosted a talk discussing how DreamWorks worked with Lenovo to upgrade its in-house animation datacenter with new solutions – during the pandemic.

The animators faced a key problem: the space for its datacenter was 22 years old, space-limited, power-limited, and sited on DreamWorks’ centrally located Los Angeles campus, leaving little prospect for any kind of physical expansion. This left DreamWorks looking for density-driven solutions.

“We know we’re not getting more real estate, it’s unlikely we’re going to get more power, so the ability to do more work with the same watts is ever more critical for us going forward,” explained Skottie Miller, vice president of platforms and infrastructure at DreamWorks. “When we started hearing about the liquid-cooled options from Lenovo, we realized we could get more density and a lot more cores without having to do major renovations.”

“The first thing we did was a feasibility study,” Miller said. “We had Lenovo professional services people come out to our studio in Glendale, walk through the datacenter and look at where the plumbing was, look at where we wanted to put the systems. We picked a corner of the datacenter that’s under-cooled for air-cooled systems but has plenty of access to plumbing.”

DreamWorks opted for a dense HPC configuration of Lenovo’s ThinkSystem SD650 servers, which max out at 56 Intel Xeon cores and 1.5 TB of memory per node. 72 of these nodes can fit in each rack, drawing up to 54 kW of power. (Lenovo did not disclose the quantity or specific configuration of the racks installed in DreamWorks’ datacenter.) The SD650 is, crucially, liquid-cooled by Lenovo’s Neptune system, which Lenovo claims reduces energy consumption by up to 40 percent.

Achieving such an ambitious installation in a relatively fixed space during a pandemic was no small feat. 

“As we got closer to making that order, Lenovo did what I thought was a smart thing, given this was during a global pandemic and supply chains were uncertain: Lenovo pre-ordered all the parts,” Miller said. “And that meant that when we actually placed the order for the system, our lead times were cut from multiple months to multiple weeks.”

Even then, the logistics were difficult due to the extensive COVID-19-related restrictions in California at the time. “We couldn’t receive the systems in Glendale,” Miller said. “So Lenovo built them and warehoused them in Europe on our behalf, and then we carefully orchestrated when we could be on-site to receive them, shipping within the pandemic to get them from Europe to southern California.”

“I joke with people that I couldn’t buy a roll of toilet paper during the global pandemic, but I could buy an HPC cluster,” Miller added. “And to me, that was fantastic.”

Once it arrived, installation required careful design of water flows, retrofits to the building to enable those flows, earthquake-proofing for the racks and much more. Despite the complexity – and some hiccups along the way – the entire process of installation and commissioning took just a day and a half of extraordinarily careful work.

“Lenovo and DreamWorks and our mechanical contractors worked together to figure out COVID practices about distancing,” Miller said. “We had a 20-minute on, 20-minute off requirement for people who were working adjacent in the machine room, so we very carefully scheduled who could do what, when.”

A week later, the systems were rendering production jobs for new DreamWorks movies, and DreamWorks now reports that the new systems are delivering a 20 percent increase in performance compared to the previous air-cooled servers. 

The new installations allowed DreamWorks to scale up the amount of power pulled per square foot by more than a hundred times, and the increase in cores allowed DreamWorks’ renderer, MoonRay, to dramatically scale up. “You’re suddenly watching things that would take minutes or even hours render in real time right in front of you,” said Scott Chapin, director of systems engineering for DreamWorks.

Lenovo committed to working with DreamWorks to support innovation in the animators’ workflows. “A key component of our partnership with Lenovo is to be a living laboratory as well as a sounding board for innovations their engineers want to try with a real customer,” Miller said.

“Everyone exceeded my expectations [in their] ability to provide such a complex system in a legacy environment during a pandemic,” Miller concluded. “I couldn’t imagine how much better it might have gone did we not have a pandemic.”