Film Studios Refocus HPC Management Lens

By Nicole Hemsoth

August 25, 2014

With the help of advanced supercomputing technologies, animated films have evolved from simple cycles of sketches to photorealistic, physics-accurate works of art.

As many of us have tracked, from the dawn of Pixar to the latest blockbusters where humans interact with realistic animations, there seems to be no limit to the hyper-realism in everything from a toss of hair, to a simulated breeze blowing snowflakes in real-time across a frozen tundra, to ships tossed by the laws of physics on rough, rendered oceans.

With great realism comes great investment, however. For instance, consider that from the time the Return of the King film emerged in 2003 with its stunning visuals until the more recent release of Dawn of the Planet of the Apes this year, there has been an estimated 10x increase in computational requirements, says David Beer, Senior Software Engineer at Adaptive Computing.

It’s no surprise that the advances in realism and accurate physics require an ever-mounting computational investment. While the media and entertainment segment represents only around an estimated, relatively steady 10% of the overall HPC server business, companies are looking to increase their compute—albeit in some interesting forms that fit the spotty, but demanding nature of the entertainment business.

The issue with rendering for many studios is that while the need for vast computational resources parallels those of large supercomputing sites, having a few thousand servers around permanently for only a relatively short period of work isn’t always efficient. Further, pushing this work out into the public cloud comes with its own concerns in terms of overall cost of provisioning large numbers of servers via special relationships with cloud service providers. Securing the servers is one side—on the cost front, the data movement adds significant weight to the bottom line while at top of mind for many studios are the still-lingering concerns about security.

Adding to all of this is the fact that most studios have conjured custom codes to manage the rendering of complex, character or scene-specific elements, which require custom workflows and management. In the end, says Beer, this means studios require a great deal of personalization, resource usage options, and workflow guidance along the way. While several studios have their own centrally-located datacenters (Pixar, for instance), others, including Weta Digital have their datacenter operations spread out over multiple centers. Still, the resources are nothing to overlook—when combined, these place in the high 100s on the Top 500.

When it comes to production studio rendering environments, the node count is often between 2,000 and 5,000 nodes, but the core counts can vary widely. Beer says that GPUs in particular are standard fare for the studios and while they might purchase a great many of them and take some advantage of mixed environments with boxes containing more memory or different coprocessors, their purchases tend to be more on the “vanilla” HPC side. In other words, studios aren’t experimenting with the latest HPC processor innovations—they’re sticking to what they know and focusing extensively on some other issues, including power and workload management.

While studios might be out front in terms of their use of GPUs and coprocessors, resource management remains a concern. While there are examples of production companies relying solely on both public or internal clouds for their rendering needs, Beer says they’re looking to make sure they’re maximizing investments in expensive HPC gear throughout the demand cycle—not just during times of peak operations. Accordingly, he says that companies like his own are finding ways to accommodate the needs of both larger studios that balance workloads across different sites as well as smaller media companies that can make use of the studio servers elsewhere via a private cloud arrangement. The point for them is that while it’s useful to house their own datacenters, for high demand for massive compute during large bursts of activity the ROI is clear, but during and after that wind-down, resource management becomes trickier.

“In the broader HPC world, they’ve solved a lot of the problems around utilization, efficiency and so forth. However, for the studios to see the utilization and power consumption rates that are found at the big supercomputing sites, there are still a lot of lessons to be learned,” Beer told us.

He also noted that as he talks with HPC community members about the concerns around exascale, the needs of the community devoted to that lofty goal and those that are driven by commercial enterprises, including media and entertainment, aren’t that different. In both realms, it’s about “getting your data where it’s supposed to be, having it ready for compute, and keeping the data pipeline stocked,” explained Beer.

“We are getting to a point when we build massive systems easily enough and while labs might have their concerns, for these people it’s worries about whether the power company will start charging big premiums. Exascale or commercial, it’s a dollars and sense conversation—but solving one solves the other.”

In short, whether it’s finding the right cloud balance, the right management framework, and the most efficient way to move and handle data, the scope of the systems for this industry aren’t out of the range of other HPC arenas. While the demand may be less ongoing, media and entertainment as a whole will be forced to face the need for growing computational requirements in increasingly complex ways.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Live and in Color, Meet the European Student Cluster Teams

November 21, 2017

The SC17 Student Cluster Competition welcomed two teams from Europe, the German team of FAU/TUC and Team Poland, the pride of Warsaw. Let's get to know them better through the miracle of video..... Team FAU/TUC is a c Read more…

By Dan Olds

SC17 Student Cluster Kick Off – Guts, Glory, Grep

November 21, 2017

The SC17 Student Cluster Competition started with a well-orchestrated kick-off emceed by Stephen Harrell, the competition chair. It began with a welcome from SC17 chair Bernd Mohr, where he lauded the competition for Read more…

By Dan Olds

Activist Investor Starboard Buys 10.7% Stake in Mellanox; Sale Possible?

November 20, 2017

Starboard Value has reportedly taken a 10.7 percent stake in interconnect specialist Mellanox Technologies, and according to the Wall Street Journal, has urged the company “to improve its margins and stock and explore Read more…

By John Russell

HPE Extreme Performance Solutions

Harness Scalable Petabyte Storage with HPE Apollo 4510 and HPE StoreEver

As a growing number of connected devices challenges IT departments to rapidly collect, manage, and store troves of data, organizations must adopt a new generation of IT to help them operate quickly and intelligently. Read more…

Installation of Sierra Supercomputer Steams Along at LLNL

November 20, 2017

Sierra, the 125 petaflops (peak) machine based on IBM’s Power9 chip being built at Lawrence Livermore National Laboratory, sometimes takes a back seat to Summit, the ~200 petaflops system being built at Oak Ridge Natio Read more…

By John Russell

Live and in Color, Meet the European Student Cluster Teams

November 21, 2017

The SC17 Student Cluster Competition welcomed two teams from Europe, the German team of FAU/TUC and Team Poland, the pride of Warsaw. Let's get to know them bet Read more…

By Dan Olds

SC17 Student Cluster Kick Off – Guts, Glory, Grep

November 21, 2017

The SC17 Student Cluster Competition started with a well-orchestrated kick-off emceed by Stephen Harrell, the competition chair. It began with a welcome from Read more…

By Dan Olds

SC Bids Farewell to Denver, Heads to Dallas for 30th

November 17, 2017

After a jam-packed four-day expo and intensive six-day technical program, SC17 has wrapped up another successful event that brought together nearly 13,000 visit Read more…

By Tiffany Trader

SC17 Keynote – HPC Powers SKA Efforts to Peer Deep into the Cosmos

November 17, 2017

This week’s SC17 keynote – Life, the Universe and Computing: The Story of the SKA Telescope – was a powerful pitch for the potential of Big Science projects that also showcased the foundational role of high performance computing in modern science. It was also visually stunning. Read more…

By John Russell

How Cities Use HPC at the Edge to Get Smarter

November 17, 2017

Cities are sensoring up, collecting vast troves of data that they’re running through predictive models and using the insights to solve problems that, in some Read more…

By Doug Black

Student Cluster LINPACK Record Shattered! More LINs Packed Than Ever before!

November 16, 2017

Nanyang Technological University, the pride of Singapore, utterly destroyed the Student Cluster Competition LINPACK record by posting a score of 51.77 TFlop/s a Read more…

By Dan Olds

Hyperion Market Update: ‘Decent’ Growth Led by HPE; AI Transparency a Risk Issue

November 15, 2017

The HPC market update from Hyperion Research (formerly IDC) at the annual SC conference is a business and social “must,” and this year’s presentation at S Read more…

By Doug Black

Nvidia Focuses Its Cloud Containers on HPC Applications

November 14, 2017

Having migrated its top-of-the-line datacenter GPU to the largest cloud vendors, Nvidia is touting its Volta architecture for a range of scientific computing ta Read more…

By George Leopold

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

GlobalFoundries Puts Wind in AMD’s Sails with 12nm FinFET

September 24, 2017

From its annual tech conference last week (Sept. 20), where GlobalFoundries welcomed more than 600 semiconductor professionals (reaching the Santa Clara venue Read more…

By Tiffany Trader

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Leading Solution Providers

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Intel Launches Software Tools to Ease FPGA Programming

September 5, 2017

Field Programmable Gate Arrays (FPGAs) have a reputation for being difficult to program, requiring expertise in specialty languages, like Verilog or VHDL. Easin Read more…

By Tiffany Trader

HPC Chips – A Veritable Smorgasbord?

October 10, 2017

For the first time since AMD's ill-fated launch of Bulldozer the answer to the question, 'Which CPU will be in my next HPC system?' doesn't have to be 'Whichever variety of Intel Xeon E5 they are selling when we procure'. Read more…

By Dairsie Latimer

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

IBM Advances Web-based Quantum Programming

September 5, 2017

IBM Research is pairing its Jupyter-based Data Science Experience notebook environment with its cloud-based quantum computer, IBM Q, in hopes of encouraging a new class of entrepreneurial user to solve intractable problems that even exceed the capabilities of the best AI systems. Read more…

By Alex Woodie

Share This