15 Years Later, the Green500 Continues Its Push for Energy Efficiency as a First-Order Concern in HPC

By Oliver Peckham

July 15, 2021

The Green500 list, which ranks the most energy-efficient supercomputers in the world, has virtually always faced an uphill battle. As Wu Feng – custodian of the Green500 list and an associate professor at Virginia Tech – tells it, “noone” cared about energy efficiency in the early 2000s, when the seeds of the Green500 were being planted. Nevertheless, in 2007, the Green500 launched its first list, ranking the participating supercomputers by their flops per watt. 29 lists and nearly 15 years later, the Green500 organizers (now having joined forces with the Top500) continue to use the list to advocate for a future for supercomputing that treats energy efficiency as a first-order concern, even as organizations hesitate to put their increasingly large systems through the efficiency gauntlet. At ISC 2021, Feng discussed the state of the Green500 and the supercomputers it tracks.

A more ambitious (and more challenging) reporting landscape

Over the years, the Green500 has developed three levels of energy efficiency reporting: levels one, two and three. Level one submissions (which, when the Green500 launched, were all submissions) have the most forgiving requirements, allowing system operators to assess efficiency using a small subset of the system during just its core phase, with further leniency allowed in meter accuracy.

The other two levels are intended to push system operators toward more accurate reporting and more active consideration of energy efficiency. Level two submissions must use a larger portion of the system – including all its subsystems – and measure the average power of a full run with higher accuracy, while level three submissions assess the efficiency of the entire system and its energy use over a full run with exacting precision (<=0.5%).

Image courtesy of Wu Feng.

Feng said that making a level two or three report comes with advantages: system operators will then have access to more accurate information, allowing them to leverage that knowledge and data in system operations (or even procurement). Furthermore, some of them had reported that the barrier to entry for a level three report was relatively low.

Reporting at different levels also has a small correlation with the values reported, Feng said: “Usually, the level one reading for flops per watt will be a smidgen higher than for level two and level three.” Comparing the Green500 reporting to miles-per-gallon ratings for a car, he said that “instead of 29 miles per gallon and 22 miles per gallon highway and city, it might be 28.96 miles per gallon and 21.96 miles per gallon … There’s higher correlation between level two and level three than [there is for] level two and level three relative to level one, but they’re all still pretty close.”

But this push for greater accuracy, of course, has its impediments: for instance, Feng explained, a level three run might indeed pose little difficulty for an organization – “but only if you have the appropriate infrastructure in place – otherwise, it can be very difficult to do.”

“[For a level three run,] you have to have the entire supercomputer instrumented – all the nodes,” Feng told HPCwire. For increasingly massive systems, the number of meters required for a level three report can span into the thousands. If you have a massive system without the appropriate instrumentation, making a more accurate submission to the Green500 can be a very tall order. (Nvidia, for instance, had been unable to make level three submissions.)

As a result, despite level two and three reporting offering more accuracy – and more benefits for system operators – it is unfeasible to require those higher standards, particularly considering that even a level one report is optional for submission to the Top500, despite the lists joining forces five years ago. (Feng did say that one proposal had suggested that the organizers only award Green500 systems if they had made a level two or higher report.) “We started with level one mainly to try and get people to start reporting,” he said. “Because if the barrier to entry is that you have to do a level three measurement, we would have even fewer entries in the Green500.”

But beyond the push for level two and three reporting, the broader Green500 list faces some general resistance. “Just doing a Top500 run is labor-intensive, and then we’re adding onto it the measurement of power,” Feng said. “So it’s hard for a lot of these institutions to be commandeering, if you will, their supercomputers for such a long period of time.”

Furthermore, Feng explained that many Top500 systems choose not to report to the Green500 because they know or suspect that their energy efficiency rating will not be competitive. “By and large, most of the measurements [on the Green500 list] are of the supercomputers that are more energy efficient,” he said. “You see a smattering of ones farther down … but that’s probably because they were higher up earlier on and then they just kept reporting the number.”

These tendencies have created a dichotomous Green500 list that exhibits both promising and worrying reporting trends. “We had a high of approximately 350 submissions early on when everything was level one,” Feng said. “[But] the level of reporting has come down somewhat.” Indeed, the most recent edition had 181 submissions – a far cry from that high water mark, which Feng said was largely fueled by IBM’s strong interest and investment in energy efficiency.

On the other hand, the average reporting quality of the respondents is going up, as are the raw numbers of respondents choosing level two and three reporting. The 29 level two submissions – more than 16 percent of overall submissions – are at least a four-year high, as are the 14 level three submissions. The higher-tier levels are also overrepresented among the most efficient supercomputers, with only five of the Green500’s top ten using level one reporting.

The top ten most energy-efficient systems on the June 2021 Green500 list. The “power quality level” column on the right denotes the reporting level: level one, two or three: Image courtesy of Wu Feng.

Increasing efficiency, especially at the high end

Trends in the systems themselves are promising, as well: systems are broadly approaching 30-gigaflops-per-watt efficiency, with heterogeneous systems “really delivering and driving on the energy efficiency front” and accelerators dominating the list. These trends are accentuated in the higher echelons of system efficiency: “The median has been improving steadily,” Feng said, “[but] it’s the top end of the list that’s been improving much more rapidly.” The Green500’s top ten also showed a strong 60 percent turnover from November’s list, drawing a sharp contrast to the relatively stable Top500 list it accompanied.

Despite that strong turnover, Feng remains bearish about the ambitious 20MW energy efficiency goal that has long loomed over U.S. plans for exascale supercomputing. Throughout his talk, Feng extrapolated from various trends to an exaflops: the imminent 30-gigaflops-per-watt efficiency of the Green500, for instance, “linearly extrapolates to an exascale computer in approximately 33MW”; based on the top system on the Top500, the extrapolation was closer to 68MW. Whether any of the three planned U.S. exascale systems will hit the 20MW (50-gigaflops-per-watt) target remains a subject of debate in the broader community, of course – and for more on that, read HPCwire’s feature on the Frontier system.

Image courtesy of Wu Feng.

Now, Feng and the Green500 press on, working to push the envelope of energy efficiency in a supercomputing status quo that regularly pushes into tens of megawatts per system.

“The goal of the list is we seek to raise awareness and encourage energy efficiency of supercomputers, and we seek to drive energy efficiency as a first-order design constraint that is on par with performance,” Feng said. Later, he added: “The problem is that … frankly, here in the United States, we’re in a society that is all about performance.”

To view the full June 2021 Green500 list, click here.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire