Sept. 16, 2024 — How will the nation’s power grid keep up with AI data centers’ soaring demand for electricity? As a longtime innovator in energy-efficient supercomputing, the Oak Ridge Leadership Computing Facility (OLCF) is investigating new ways for minimizing power consumption while maximizing performance

As high-tech companies ramp up construction of massive data centers to meet the business boom in artificial intelligence, one component is becoming an increasingly rare commodity: electricity.

Commercial demand for electricity has been growing sharply in recent years and is projected to increase by 3% in 2024 alone, according to the U.S. Energy Information Administration. But that growth has been driven by just a few states — the ones that are rapidly becoming hubs for large-scale computing facilities, such as Virginia and Texas.

The inventory of North American data centers grew 24.4% year over year in the first quarter of 2024, as the real-estate services firm CBRE reports in its “Global Data Center Trends 2024” study. These new centers are being built with capacities of 100 to 1,000 megawatts, or about the same loads that can power from 80,000 to 800,000 homes, notes the Electric Power Research Institute in a 2024 white paper. In this paper, EPRI analyzes AI and data-center energy consumption and predicts that if a projected high growth rate of 10% per year continues, data centers will annually consume up to 6.8% of total U.S. electricity generation by 2030 — versus an estimated 4% today.

To satisfy that soaring demand, Goldman Sachs Research estimates that U.S. utilities will need to invest around $50 billion in new electrical generation capacity. Meanwhile, community opposition to data center construction in some areas is also growing, as grassroots groups protest the potential local impacts of more and more data centers and their increasing demands for electricity for AI and water for cooling.

Whether the nation’s private enterprises can pull off the daunting challenge of powering an AI “revolution” may depend less on money and more on ingenuity. That CBRE study concludes with a helpful, or perhaps hopeful, recommendation: “High-performance computing [or HPC] will require rapid innovation in data center design and technology to manage rising power density needs.”

At the Oak Ridge Leadership Computing Facility, a Department of Energy Office of Science user facility located at Oak Ridge National Laboratory, investigating new approaches to energy-efficient supercomputing has always been part of its mission. Since its formation in 2004, the OLCF has fielded five generations of world-class supercomputing systems that have produced a nearly 2,000 times increase in energy efficiency per floating point operation per second, or flops. Frontier, the OLCF’s latest supercomputer, currently ranks first in the TOP500 list of the world’s most powerful computers, and in 2022, it debuted at the top of the Green500 list of the world’s most energy-efficient computers.

Keeping the electricity bill affordable goes hand in hand with being a government-funded facility. But constructing and maintaining leadership supercomputers are no longer just the domain of government. Major tech companies have entered HPC in a big way but are only just now starting to worry about how much power these mega systems consume.

“Our machines were always the biggest ones on the planet, but that is no longer true. Private companies are now deploying machines that are several times larger than Frontier. Today, they essentially have unlimited deep pockets, so it’s easy for them to stand up a data center without concern for efficiency,” said Scott Atchley, chief technology officer of the National Center for Computational Sciences, or NCCS, at ORNL. “That will change once they become more power constrained, and they will want to get the most bang for their buck.”

With decades of experience in making HPC more energy efficient, the OLCF may serve as a resource for best “bang for the buck” practices in a suddenly burgeoning industry.

“We are uniquely positioned to influence the full energy-efficiency ecosystem of HPC, from the applications to the hardware to the facilities. And you need efficiency gains in all three of those areas to attack the problem,” said Ashley Barker, OLCF program director. “Striving for improvements in energy efficiency comes into play in every aspect of our facility. What is the most energy-efficient hardware we can buy? What is the most energy-efficient way we can run that hardware? And what are the most energy-efficient ways that we can tweak the applications that run on the hardware?”

As the OLCF plans its successor to Frontier – called Discovery – those questions are asked daily as different teams work together to deliver a new supercomputer by 2028 that will also demonstrate next-generation energy efficiencies in HPC.

System Hardware

One of the most significant computational efficiency advancements of the past 30 years originated from an unlikely source: video games.

More specifically, the innovation came from chip makers competing to fulfill the video game industry’s need for increasingly sophisticated in-game graphics. To achieve the realistic visuals that drew in gamers, personal computers and game consoles required dedicated chips — also known as the graphics processing unit, or GPU — to render detailed moving images.

Today, GPUs are an indispensable part of most supercomputers, especially ones used for training artificial intelligence models. In 2012, when the OLCF pioneered the use of GPUs in leadership-scale HPC with its Titan supercomputer, the design was considered a bold departure from traditional systems that rely only on central processing units, or CPUs. It required computational scientists to adapt their codes to fully exploit the GPU’s ability to churn through simple calculations and speed up the time to solution. The less time it takes a computer to solve a particular problem, the more problems it can solve in a given time frame.

“A GPU is, by design, more energy efficient than a CPU. Why is it more efficient? If you’re going to run electricity into a computer and you want it to do calculations very efficiently, then you want almost all the electricity powering floating point operations. You want as much silicon area to just be floating point units, not all the other stuff that’s on every CPU chip. A GPU is almost pure floating point units. When you run electricity into a machine with GPUs, it takes roughly about a tenth the amount of energy as a machine that just has CPUs,” said ORNL’s Al Geist, director of the Frontier project.

The OLCF’s gamble on GPUs in 2012 paid off over the next decade with progressively more energy-efficient systems as each generation of OLCF supercomputer increased its number of speedier GPUs. This evolution culminated in the architecture of Frontier, launched in 2022 as the world’s first exascale supercomputer, capable of more than 1 quintillion calculations per second and consisting of 9,408 compute nodes.

However, when exascale discussions began in 2008, the Exascale Study Group issued a report outlining its four biggest challenges, foremost of which was power consumption. It foresaw an electric bill of potentially $500 million a year. Even accounting for the projected technological advances of 2015, the report predicted that a stripped-down 1-exaflop system would use 150 megawatts of electricity.

“DOE said, ‘That’s a non-starter.’ Well, we asked, what would be acceptable? And the answer that came back was, ‘We don’t want you to spend more money on electricity than the cost of the machine,’” Geist said. “In the 2009 time frame, supercomputers cost about $100 million. They have a lifetime of about five years. What you end up with is about $20 million per year that we could spend on electricity. How many megawatts can I get out of $20 million? It turns out that 1 megawatt here in East Tennessee is $1 million a year, roughly. So that was the number we set as our target: a 20-megawatt per exaflop system.”

There wasn’t a clear path to achieving that energy consumption goal. So, in 2012, the DOE Office of Science launched the FastForward and DesignForward programs to work with vendors to advance new technologies. FastForward initially focused on the processor, memory and storage vendors to address performance, power-consumption and resiliency issues. It later moved its focus to node design (i.e., the individual compute server). DesignForward initially focused on scaling networks to the anticipated system sizes and later focused on whole-system packaging, integration and engineering.

As a result of the FastForward investment, semiconductor chip vendor AMD developed a faster, more powerful compute node for Frontier — consisting of a 64-core 3rd Gen EPYC CPU and four Instinct MI250X GPUs — and figured out a way to make the GPUs more efficient by turning off sections of the chips that are not being used and then turning them back on when needed in just a few milliseconds.

“In the old days, the entire system would light up and sit there idle, still burning electricity. Now we can turn off everything that’s not being used — and not just a whole GPU. On Frontier, about 50 different areas on each GPU can be turned off individually if they’re not being used. Now, not only is the silicon area mostly devoted to floating point operations, but in fact I’m not going to waste any energy on anything I’m not using,” Geist said.

However, with the next generation of supercomputers, simply continuing to add more GPUs to achieve more calculations per watt may have reached its point of diminishing returns, even with newer and more advanced architectures.

“The processor vendors will really have to reach into their bag of tricks to come up with techniques that will give them just small, incremental improvements. And that’s not only true for energy efficiency, but it’s also true for performance. They’re getting about as much performance out of the silicon as they can,” Atchley said. “We’ve been benefiting from Moore’s Law: transistors got smaller, they got cheaper and they got faster. Our applications ran faster, and the price point was the same or less. That world is over. There are some possible technologies out there that might give us some jumps, but the biggest thing that will help us is a more integrated, holistic approach to energy efficiency.”

System Operations

Feiyi Wang — leader of the OLCF’s Analytics and AI Methods at Scale, or AAIMS, group — has been spending much of his time pondering an elusive goal: how to operate a supercomputer so that it uses less energy. Tackling this problem first required the assembly of a massive amount of HPC operational data.

Long before Frontier was built, he and the AAIMS group collected over one year’s worth of power profiling data from Summit, the OLCF’s 200-petaflop supercomputer launched in 2018. Summit’s 4,608 nodes each have over 100 sensors that report metrics at 1 hertz, meaning that for every second, the system reports over 460,000 metrics.

Using this 10-terabyte dataset, Wang’s team analyzed Summit’s entire system from end to end, including its central energy plant, which contains all its cooling machinery. They overlaid the system’s job allocation history on the telemetry data to construct per-job, fine-grained power-consumption profiles for over 840,000 jobs. This work earned them the Best Paper Award at the 2021 International Conference for High Performance Computing, Networking, Storage, and Analysis, or SC21.

To continue reading OLCF’s article, click here.

---

Source: Coury Turczyn, ORNL