Annual GPU Upgrades: Nvidia’s Plan for Faster Chips 

By Agam Shah

October 16, 2023

If you are waiting in a giant line for Nvidia’s H100 GPUs, be advised that the next-generation H200 chip is already on its way.  The GPU maker earlier this month released its product roadmap of AI GPUs leading into 2025. The new GPUs are intended to continue Nvidia’s dominance in artificial intelligence, a market that took off after ChatGPT’s magic shook the world. 

The most surprising takeaway from the roadmap is Nvidia’s plan to release GPUs every year. Nvidia’s H100 was released in 2023, two years after its predecessor, A100.   

Nvidia has no one seriously challenging its competition, and revving up to yearly GPU upgrades puts pressure on rivals to keep up, analysts said. 

Intel is already behind with its Falcon Shores GPU coming out in 2025, and AMD’s MI300 is due out by the end of this year. Even Nvidia’s customers, such as OpenAI, are considering developing their own AI chips as the cost of running AI becomes untenable. 

“Nvidia may be speeding up its roadmap because it wants to put more distance between itself and the competition, and even its customers, many of whom are now designing their own AI chips,” said Kevin Krewell, principal analyst at Tirias Research. 

Nvidia is already getting a premium for its H100 and can charge even more for next-generation GPUs. Customers deploying AI workloads, like gamers, will shell out the cash for the latest and greatest hardware.  

Chip development is also at an inflection point, making it possible for Nvidia to upgrade GPUs yearly. From an architectural standpoint, Nvidia has many variables and options to play with when integrating processing, IO, memory, and communications and packaging them vertically. 

“The semiconductor market is also entering a new era with chiplets, which could be changing how Nvidia designs its chips,” Krewell said.  

The yearly upgrades also include CPUs and networking products. That cadence for upgrades may not apply to AI chip competitors, who are still trying to find chip customers. 

The new roadmap was mentioned in an investor presentation published earlier this month but could easily change.

The New Roadmap

Nvidia’s new roadmap lists yearly products related to compute and communications. It breaks down GPUs used with x86 chips and GPUs and CPUs based on its own ARM processor designs. 

For x86-related GPUs, Nvidia’s successor to the H100 GPU will be the H200, which will be released in 2024. In the same year, Nvidia will also release the B100 GPU and, finally, the X100 GPU in 2025. 

The roadmap also has a lineup of successors for the L40s, which is based on the Ada Lovelace architecture. The L40s is a poor man’s version of the H100 but is faster than previous-generation A100 GPUs in AI training and inference. Nvidia is redirecting H100 customers in urgent need of GPUs to buy the L40S GPU. 

The successor to L40S will be B40 in 2024 and the X40 in 2025. The roadmap shows the L40S-B40-X40 lineup for “X86 enterprise and inferencing,” meaning it is optimized for inferencing.  

Nvidia’s CPU roadmap provides yearly upgrades on its ARM processors, which can be paired with the GPUs mentioned above.  

 

Nvidia roadmap 2021-2025
Nvidia Roadmap GPU Progression 2021-2025 (Source Nvidia)

For inferencing, the current GH200, which has HBM3E memory, will ship next year and is tied to the H200 GPUs. The GB200, also due next year, is designed to be used with the B100 GPU. The GX200, coming in 2025, is designed for use with the X100 GPU. 

Nvidia will add NVLink interconnects for AI training to provide high-speed links between ARM CPUs and related GPUs. The GH200NVL (H200 GPU) and GB200NVL (B100 GPU) will ship in 2024, and the GX200NVL (X100 GPU) will ship in 2025. 

CPUs are much less relevant for large-scale models, but Nvidia’s ARM CPU and GPU package is a great combination for AI training, said Naveen Rao, vice president for generative AI at Databricks. Rao previously was CEO of MosaicML, an AI startup sold to Databricks for $1.3 billion earlier this year. 

“The CPU could become relevant as a programming interface, though…GH200 essentially uses ARM as the programming interface with a fast GPU sitting very close. This design could be an awesome combo,” Rao said.   

Technologies like AMX in Intel CPUs could also be super relevant, but they need to go much bigger and enable multi-chip scaling in a big way, Rao added. 

The new roadmap also speeds up the networking bandwidth from 400G in 2024 to 1,600G in 2026 in its Quantum product for Infiniband interfaces and Spectrum-X for Ethernet and hyperscale infrastructures. Nvidia’s DGX systems use both Infiniband and Ethernet network technologies. 

How Chiplets Could Define the Roadmap

Historically, Intel upgraded chips yearly with either new manufacturing technologies or new features on the same manufacturing processes. But that slowed down as scaling manufacturing became challenging. 

Chip design is now at an inflection point with conventional technologies like FINFET running out of steam, said David Kantor, principal analyst at Real World Technologies.  

Conventional chip designs focused on integrating all components into single chips. An emerging trend is to decompose SOCs and chips into chiplets, or smaller compute, I/O, and communications units, which can be assembled in 2.5D or 3D packages, Kantor said.  

Nvidia declined to comment on its next-generation GPUs. But industry experts said Nvidia has many options it can explore on GPUs, DRAM, I/O, and SRAM integration, and suggested many possibilities of what chips may look like. Nvidia may also have the option of Intel as a manufacturing partner by 2025 for X100. 

The H200 chip upgrade includes a new type of HBM3E memory. Nvidia earlier this year announced the GH200 chip, connecting it in the roadmap to the H200 GPU.  

After H200, Nvidia could start using chiplets, modularizing the GPU into blocks with a choice of CPUs, accelerators, and interconnects in a package. The blocks, also called tiles, could be manufactured using various processes. 

This method would allow Nvidia to conceive heterogeneous chips with separate tiles for compute, I/O, and SRAM, made on different manufacturing processes.  

For example, the compute tiles and memory of B100 and B40 chips could be made on TSMC’s N3 process and SRAM tiles on an older process. N3 does not scale well for SRAM and analog portions of GPUs, so the possibility of integrating these modules made on older manufacturing processes will be an advantage.  

By 2025, Intel’s 18A process will come online and possibly surpass TSMC, and Nvidia could turn to Intel’s fabs to manufacture X100. Nvidia has manufactured test chips on Intel’s next-generation processes, and CEO Jensen Huang was happy with the outcome. 

TSMC is expected to shift from N3 to the two-nanometer N2 by 2025. Both Intel and TSMC will have gate-all-around technologies on those nodes. 

 

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket for an optional 8087 math coprocessor. The math coprocessor ma Read more…

IonQ Reports Advance on Path to Networked Quantum Computing

February 22, 2024

IonQ reported reaching a milestone in its efforts to use entangled photon-ion connectivity to scale its quantum computers. IonQ’s quantum computers are based on trapped ions which feature long coherence times and qubit Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Apple. Today the consumer electronics giant started rolling Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to derive any substantial value from it. However, the GenAI hyp Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks – that builds on earlier work and is an eff Read more…

AWS Solution Channel

Shutterstock 2283618597

Deep-dive into Ansys Fluent performance on Ansys Gateway powered by AWS

Today, we’re going to deep-dive into the performance and associated cost of running computational fluid dynamics (CFD) simulations on AWS using Ansys Fluent through the Ansys Gateway powered by AWS (or just “Ansys Gateway” for the rest of this post). Read more…

Atom Computing Reports Advance in Scaling Up Neutral Atom Qubit Arrays

February 15, 2024

The scale-up challenge facing quantum computing (QC) is daunting and varied. It’s commonly held that 1 million qubits (or more) will be needed to deliver practical fault tolerant QC. It’s also a varied challenge beca Read more…

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket fo Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performa Read more…

The Pulse of HPC: Tracking 4.5 Million Heartbeats of 3D Coronary Flow

February 15, 2024

Working in Duke University's Randles Lab, Cyrus Tanade, a National Science Foundation graduate student fellow and Ph.D. candidate in biomedical engineering, is Read more…

It Doesn’t Get Much SWEETER: The Winter HPC Computing Festival in Corpus Christi

February 14, 2024

(Main Photo by Visit Corpus Christi CrowdRiff) Texas A&M University's High-Performance Research Computing (HPRC) team hosted the "SWEETER Winter Comput Read more…

Q-Roundup: Diraq’s War Chest, DARPA’s Bet on Topological Qubits, Citi/Classiq Explore Optimization, WEF’s Quantum Blueprint

February 13, 2024

Yesterday, Australian start-up Diraq added $15 million to its war chest (now $120 million) to build a fault tolerant computer based on quantum dots. Last week D Read more…

2024 Winter Classic: Razor Thin Margins in HPL/HPCG

February 12, 2024

The first task for the 11 teams in the 2024 Winter Classic student cluster competition was to run and optimize the LINPACK and HPCG benchmarks. As usual, the Read more…

2024 Winter Classic: We’re Back!

February 9, 2024

The fourth edition of the Winter Classic Invitational Student Cluster Competition is up and running. This year, we have 11 teams of eager students representin Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire