The GenAI Data Center Squeeze Is Here

By Doug Eadline

February 1, 2024

The immediate effect of the GenAI GPU Squeeze was to reduce availability, either direct purchase or cloud access, increase cost, and push demand through the roof.

A secondary issue has been developing over the last several years. Even though your organization secured several racks with GPUs, how will you power them, and where will you put them?

For instance, many universities have previously placed new equipment in their campus data center. Many of these data centers are now “tapped out” and have no more space or power. Estimates for the current “GPU rack” range from 50 to 100 KW (kilowatts) per rack (previous estimations for “CPU racks” are 10-17 KW per rack). If you want to co-locate 4 GPU racks, a data center that can provide 400 KW of power may be difficult to find.

The same applies to the smaller companies that provide HPC colocation services. They are finding that current data centers are tapped out and must search for space and power. These companies are the non-Hyperscalers that can’t stand up an entire data center campus.

A recent report from JLL, a real estate investment and management firm, Data Centers 2024 Global Outlook  (The report can be obtained by providing an email address), shares some interesting trends. The report explores how data centers need to be designed, operated, and sourced to meet the evolving needs of the global economy, in particular, the power increase required for GPU-heavy GenAI clusters.

The AI-fueled growth is expected to continue into the near future. Consumers and businesses are anticipated to generate twice as much data in the next five years as all the data created over the past ten years.

In addition to the GPU computing needs, the growing demands of GenAI, data center storage capacity is expected to grow from 10.1 zettabytes (ZB) in 2023 to 21.0 ZB in 2027, for a five-year compound annual growth rate of 18.5%[1]. This increased storage will generate a need for more data centers, and generative AI’s greater energy requirements – ranging from 300 to 500+ megawatts per campus – will also require more energy-efficient designs and locations. The need for more power will require data center operators to increase efficiency and work with local governments to find sustainable energy sources to support data center needs.

According to the report, AI-specialized data centers look different than conventional facilities and may require operators to plan, design, and allocate power resources based on the type of data processed or stage of GenAI development. Regarding heat removal, the huge increase in GPUs will surpass current standards. In a typical data center, air-cooling typically accounts for roughly 40% of an average data center’s electricity use. Users, particularly hyperscalers and operators, are shifting from traditional air-based cooling methods to liquid cooling and rear-door heat exchangers. Case studies have shown that liquid cooling offers significant power reductions, as high as 90%, while improving capability and allowing more densely packed systems that, in turn, increase rack power usage.

Hyperscalers, which have been at the forefront of adopting AI and high-performance computing (HPC), have the greatest need for high-density infrastructure (See Figure). Currently, their large facilities have an estimated average density of 36kW per rack, with increased liquid cooling density and GPU hardware IDC estimates will grow at a 7.8% CAGR in the coming years to approach 50kW per rack by 2027. [2]

(Source JLL: Data Centers 2024 Global Outlook)

In an interview with Andy Cvengros, managing director of U.S. Data Center Markets for JLL, HPCwire learned many issues facing data center deployments.

The first and most important issue he suggests is planning. For instance, Cvengros mentioned that with power grids becoming effectively tapped out and transformers having more than three-year lead times, operators will need to innovate. 

The GPU squeeze is taking pace at the data center level, where a small colocation deployment of 4-5 racks will have a harder time finding a data center because the hyperscalers are requesting entire data center campuses.

According to Cvengros, all major metro areas are basically tapped out, and secondary areas, like Reno, NV, or Columbus, OH, are now prime locations for new data center construction. The demand is expected to continue, and new data centers are 3.5 years out. He reiterated, “Planning is key.”

He recommends working with a company that specializes in high-performance systems for smaller HPC GPU-Cluster colocation (i.e., the University that is completely tapped out of power and space). In his experience, data center providers must actively track usage and availability for global opportunities to deliver any near-term colocation capabilities.

The Data Center Squeeze is a Global Problem

The JLL report also lists the critical changes needed across the globe to address increased power usage.

  • In Europe, one-third of the grid infrastructure is over 40 years old, requiring an estimated €584 billion of investment by 2030 to meet the European Union’s green goals.
  • In the United States, meeting energy transition goals to upgrade the grid and feed more renewable energy into the power supply will require an estimated $2 trillion.
  • The rapid growth of data centers is also putting pressure on limited energy resources in many countries. In Singapore, for example, the government enacted a moratorium to temporarily halt construction in certain regions to carefully review new data center proposals and ensure alignment with the country’s sustainability goals.

The global GenAI energy “request” (or demand) presents opportunities and challenges to the data center sector. GenAI needs power on a scale not seen in the past. Finding GPUs for HPC is only half the problem; where to plug them in may become a bigger challenge.

 

[1] IDC, Revelations in the Global StorageSphere, July 2023

[2] July 2023, IDC #AP50326223, Asia/Pacific (Excluding Japan) DC Deployment Model and Spend Forecast, 2H22: 2022–20a27

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket for an optional 8087 math coprocessor. The math coprocessor ma Read more…

IonQ Reports Advance on Path to Networked Quantum Computing

February 22, 2024

IonQ reported reaching a milestone in its efforts to use entangled photon-ion connectivity to scale its quantum computers. IonQ’s quantum computers are based on trapped ions which feature long coherence times and qubit Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Apple. Today the consumer electronics giant started rolling Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to derive any substantial value from it. However, the GenAI hyp Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks – that builds on earlier work and is an eff Read more…

AWS Solution Channel

Shutterstock 2283618597

Deep-dive into Ansys Fluent performance on Ansys Gateway powered by AWS

Today, we’re going to deep-dive into the performance and associated cost of running computational fluid dynamics (CFD) simulations on AWS using Ansys Fluent through the Ansys Gateway powered by AWS (or just “Ansys Gateway” for the rest of this post). Read more…

Atom Computing Reports Advance in Scaling Up Neutral Atom Qubit Arrays

February 15, 2024

The scale-up challenge facing quantum computing (QC) is daunting and varied. It’s commonly held that 1 million qubits (or more) will be needed to deliver practical fault tolerant QC. It’s also a varied challenge beca Read more…

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket fo Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performa Read more…

The Pulse of HPC: Tracking 4.5 Million Heartbeats of 3D Coronary Flow

February 15, 2024

Working in Duke University's Randles Lab, Cyrus Tanade, a National Science Foundation graduate student fellow and Ph.D. candidate in biomedical engineering, is Read more…

It Doesn’t Get Much SWEETER: The Winter HPC Computing Festival in Corpus Christi

February 14, 2024

(Main Photo by Visit Corpus Christi CrowdRiff) Texas A&M University's High-Performance Research Computing (HPRC) team hosted the "SWEETER Winter Comput Read more…

Q-Roundup: Diraq’s War Chest, DARPA’s Bet on Topological Qubits, Citi/Classiq Explore Optimization, WEF’s Quantum Blueprint

February 13, 2024

Yesterday, Australian start-up Diraq added $15 million to its war chest (now $120 million) to build a fault tolerant computer based on quantum dots. Last week D Read more…

2024 Winter Classic: Razor Thin Margins in HPL/HPCG

February 12, 2024

The first task for the 11 teams in the 2024 Winter Classic student cluster competition was to run and optimize the LINPACK and HPCG benchmarks. As usual, the Read more…

2024 Winter Classic: We’re Back!

February 9, 2024

The fourth edition of the Winter Classic Invitational Student Cluster Competition is up and running. This year, we have 11 teams of eager students representin Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire