Since 1986 - Covering the Fastest Computers in the World and the People Who Run Them

Language Flags
February 5, 2013

NVIDIA Scores Green Victory with Italian Supercomputer

Nicole Hemsoth

There must have been a few companies that were a bit put off by the announcement of the Beacon computer as the Green500 champ last November. Everyone, that is, except Intel Corp.

Beacon, developed by the National Institute for Computational Sciences (NICS) at the University of Tennessee, started out as a research project to port software code to small clusters of Intel Many Integrated Core (MIC3) chips. When the group started working with the MICs, now better known as the Xeon Phi processor line, it found some compelling energy-saving possibilities. The NICS group decided to use the technology to create a small HPC cluster.

On Friday Eurotech and NVIDIA announced a new HPC system, called Eurora, that the companies say beats Beacon’s energy efficiency by 26% – using NVIDIA GPUs instead of Intel Xeons, of course. With Eurora pumping out 3.15 gigaflops per watt versus Beacon’s 2.49 gigaflops per watt, NVIDIA executives are happy that they can now claim the most energy-efficient HPC system uses their GPUs.

The most powerful supercomputer in the world, the Titan system at Oak Ridge National Lab, uses the same K20 GPUs as Eurora. Titan, however, finished third on the Green500 list. Eurora is NVIDIA’s chance to take the title back from Intel.

Even with the advent of multicore CPUs, accelerators seem to be critical to creating today’s most efficient HPC systems. NVIDIA technology, in particular, has made its presence felt on recent Green500 lists. “GPUs have been the biggest innovation in energy efficiency in the last five years,” says Sumit Gupta, NVIDIA’s general manager for Tesla GPU Accelerated Computing.

Eurora’s efficiency was not just about GPUs, though. Everything about the system was created with the intention of optimizing power draw. The machine is part of Eurotech’s latest Aurora Tigon line of HPC systems. Aurora was designed to decrease energy cost by using Eurotech’s custom-built hot-water cooling system, enabling a datacenter PUE as low as 1.05.

To create Eurora, Eurotech replaced older generation NVIDIA GPUs in its previous Aurora systems with the latest Tesla K20 processors. NVIDIA’s K20 Kepler generation GPU for the HPC market was also designed specifically to increase FLOPS per watt.

Next >>

The Eurora system was delivered to Cineca, an Italian non-profit consortium of universities and a member of the Partnership for Advanced Computing in Europe (PRACE). Cineca provides access to HPC systems for research institutions across Europe and prides itself as usually having one of the most powerful supercomputers in the world. Now, it seems, it can add energy efficiency to its boast. It’s an important consideration in Europe, where energy prices tend to be much higher than those in the U.S., and where governments are increasingly demanding green technology.

Still, Eurora is essentially a prototype upon which to build. NVIDIA says it shipped a special version of its GPU to Eurotech. And Eurora, with a Linpack rating of 110 teraflops, is a small system compared to Titan at 17 petaflops. That makes the Oak Ridge machine 150 times more powerful than Cineca’s.

NVIDIA’s Gupta says it’s not really fair to compare the reported energy efficiency for Beacon with that of other systems like Titan and Eurora. “Beacon was measured only for the Green500,” he points out. “You can always create the ideal situation for that. Cineca is doing what they would do in production. Cineca is government funded and has regulations. They have to be mindful of the data they report.”

Gupta notes that paradoxically, the energy-efficiency gains in HPC have been driven in large part by consumer products, not the need to cut energy costs for supercomputers or datacenters. The GPUs built to run electronic game graphics have to get by on as little energy as possible because of short battery life. Those same GPUs then started going into HPC systems, both to boost performance and to cut down on energy use.

NVIDIA’s Kepler generation, released a couple years ago for the gaming market, is the processor that was really designed to reduce energy consumption as much as possible. The company streamlined the design by eliminating a lot of registers and logic that were not really needed, and increased the number of cores dramatically. With more cores working, the company could cut the clock speed of the processors by almost half. “Frequency has a bigger impact on power efficiency” than almost any other change one can make, says Gupta. “The simple trick is to increase the number of cores and decrease the frequency.”

Still, Eurora is noticeably more energy efficient than the Titan, which uses the same generation of NVIDIA GPUs. So why the difference? A lot of it is due to Eurotech’s on-board cooling system. That system delivers 50 degree C water to a heat exchanger – flat metal plates with water-flow systems attached. The plates sit directly in contact with the components on a circuit board to uniformly draw away heat. That’s why NVIDIA had to provide a special version of its GPUs to Eurotech. They don’t have the usual heat sinks attached, but accommodate a plate instead.

Still, given the efficiency of Beacon, there will be an ongoing contest between NVIDIA’s GPUs and Intel’s Xeon Phi coprocessors for the best performance per watt. The Intel Phi processors are not yet shipping beyond a few test systems such as Beacon, but will be more generally available within a few months. As these technologies move beyond their testing stages and into real production systems, supercomputers and datacenters should see real benefits.

And, of course everyone will be watching to see whose accelerators show up on top of the next Green500 list.

SC14 Virtual Booth Tours

AMD SC14 video AMD Virtual Booth Tour @ SC14
Click to Play Video
Cray SC14 video Cray Virtual Booth Tour @ SC14
Click to Play Video
Datasite SC14 video DataSite and RedLine @ SC14
Click to Play Video
HP SC14 video HP Virtual Booth Tour @ SC14
Click to Play Video
IBM DCS3860 and Elastic Storage @ SC14 video IBM DCS3860 and Elastic Storage @ SC14
Click to Play Video
IBM Flash Storage
@ SC14 video IBM Flash Storage @ SC14  
Click to Play Video
IBM Platform @ SC14 video IBM Platform @ SC14
Click to Play Video
IBM Power Big Data SC14 video IBM Power Big Data @ SC14
Click to Play Video
Intel SC14 video Intel Virtual Booth Tour @ SC14
Click to Play Video
Lenovo SC14 video Lenovo Virtual Booth Tour @ SC14
Click to Play Video
Mellanox SC14 video Mellanox Virtual Booth Tour @ SC14
Click to Play Video
Panasas SC14 video Panasas Virtual Booth Tour @ SC14
Click to Play Video
Quanta SC14 video Quanta Virtual Booth Tour @ SC14
Click to Play Video
Seagate SC14 video Seagate Virtual Booth Tour @ SC14
Click to Play Video
Supermicro SC14 video Supermicro Virtual Booth Tour @ SC14
Click to Play Video