A Slice of Green with Accelerators On Top
On the heels of a successful ISC14 in Leipzig, Germany, last week, the latest Green500 list was just published, and once again the TSUBAME-KFC is the world’s greenest supercomputer with an energy efficiency of 4.4 gigaflops per watt. This system from Tokyo Institute of Technology broke the 4.0 gigaflops per watt mark last November, and it’s still the only supercomputer to do so.
Launched in November 2007, and now in its eighth year, the Green500 helps raise awareness for sustainable supercomputing by ranking the most powerful systems in the world based on energy-efficiency. Where the TOP500 list is a measure of pure performance, i.e., FLOPS, the Green500 list emphasizes energy-efficiency, using the metric of FLOPS-per-watt.
As with the TOP500 list, the Green500 has been moving toward heterogeneity in terms of the highest-ranked systems. On the current Green500 list, the top 17 spots are occupied by accelerator-based systems, which combine graphics processing units (GPUs) or coprocessors (such as Intel Xeon Phi chips) with traditional processors (CPUs). Of these 17, the first fifteen systems employ NVIDIA Kepler GPUs; the 16th system, Shadow, is equipped with Intel Phi coprocessors and Intel Xeon CPUs; and the number 17 system, SANAM, is accelerated with AMD FirePro S10000 GPUs operating in tandem with Intel Xeon CPUs.
In the number one spot for the second time is TSUMAME-KFC installed at the Tokyo Institute of Technology. The supercomputer combines NVIDIA Tesla K20X GPUs with a specialized cooling system that immerses servers in an oil-based liquid bath. At 4.4 gigaflops per watt, Tsubame-KFC is about 20 percent more efficient than the runner-up Cambridge University’s Wilkes supercomputer, which runs at 3.6 gigaflops per watt. In third place is the HA-PACS TCA system at the University of Tsukuba, delivering 3.5 gigaflops per watt.
Up to this point, the listing is the same as the November 2013 iteration with the top three systems unchanged. Unlike the TOP500 list, which only had one new entrant into the top ten, however, the Green500 sees three new systems in this coveted grouping. One of these new entrants is SURFsara’s Cartesius Accelerator Island, comprised of 66 bullx B515 GPGPU accelerated nodes outfitted with Intel Xeon CPUs and Telsa K40m GPUs. The system entered the June 2014 TOP500 at number 421 with a Linpack performance of 153.6 teraflops. It has an energy consumption of 3.459 gigaflops per watt.
In fifth place is the Swiss system, Piz Daint, down one spot from the previous list. The Cray XC30 supercomputer, employed for advanced weather forecasting and other research endeavors, is based on both Xeon CPUs and NVIDIA K20x parts.
Rounding out spots 6-10 are systems from ROMEO HPC Center, CSIRO, a second machine from the Tokyo Institute of Technology, an exploration and production system from energy firm Eni, and an “unknown financial institution.” The CSIRO GPU Cluster, manufactured by Xenon Systems, moved from tenth position up to seventh position as a result of an upgrade that increased its energy-efficiency from 2.358 gigaflops per watt to 3.019 gigaflops per watt.
As was seen with the most recent TOP500, heterogeneity has infiltrated the top end of the list, but the hybrid approach has not yet permeated the rest of the list. Of the 500 systems, only 64 of them employ some kind of accelerator, relatively unchanged from the last list. There is little question about the energy-efficiency of the mixed processor strategy, however. The average energy efficiency of the accelerator-based group is 1.938 gigaflops per watt, compared with 0.743 gigaflops per watt for the homogeneous systems.
Before accelerators infiltrated supercomputing, IBM Blue Gene had a lock on power efficiency. Remember when Big Blue swept the first 20 spots on the June 2012 list? This power (Power) block is still there, it’s just moved down a bit. BlueGene/Q systems dominate 24 out of 25 spots, starting from position 18. Their energy efficiency ranges from 2.177 gigaflops per watt to 2.299 gigaflops per watt.
As for the current TOP500 chart toppers: Titan, the Cray XK7 installed at Oak Ridge National Laboratory and the fastest supercomputer in the US, fell a few pegs on the Green500 from 35th to 43rd place with 2.143 gigaflops per watt. The world’s fastest super, China’s Tianhe-2 is sitting in 49th place on the Green500 with 1.901 gigaflops per watt – down from 40th place on the previous list. The six-petaflopper (Linpack) Piz Daint is the only system to make it into the top ten of both lists: sixth on the TOP500 and fifth on the Green500. With the stringent power requirements of the coming exascale era, one would expect to begin seeing more overlap between the two sets.
While there were a few changes from November, the June Green500 is in somewhat of a holding pattern, much like the TOP500, and this phenomenon prompted no end of speculation at the recent ISC conference. Is this the calm before the storm that’s anticipated for 2015 or is something else afoot?