Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

By Agam Shah

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn’t an apt description, so it settled on Hypercomputer.

The AI training system is home to Google’s new AI chip, the Cloud TPU v5p, released alongside the Gemini large-language model.

Gemini is Google’s latest attempt to take on Microsoft, which uses GPT-4 to provide AI techniques to answer questions, create documents and images, and much more.

Google timed the release of the TPU V5p chip with the new generative model. The Hypercomputer was also announced to host the new TPU, which is targeted to run the AI workloads.

The system seems to be an example of the hardware-software co-design concept, which all major chip makers are chasing to speed up the time needed to generate AI results. Nvidia typically releases new CUDA releases in conjunction with high-performance GPUs — CUDA 12 was released with the Hopper GPUs.

In a blog entry, Google said its Hypercomputer resulted from “decades of research and development” to develop a system for modern AI workloads. Google claims the chip provides better AI hardware utilization rates, reducing power consumption.

AI applications require vast hardware resources, and cloud providers are developing systems to provide faster results while reducing the cost per transaction. Hyperscalers are investing billions to build data centers loaded with GPUs, AI chips, memory, and high-bandwidth interconnects to process AI queries.

Microsoft has its own AI supercomputer called Eagle, which is now the third-fastest supercomputer in the world, according to the Top500 list. The system delivers 561 petaflops of performance and has 1.12 million computing cores. The horsepower is provided by Nvidia’s H100 GPU and Intel’s Sapphire Rapids CPUs.

Google’s Hypercomputer is built specifically for AI applications, as TPUs aren’t made for scientific computing. The Google system includes a network of pods with Cloud TPU v5p chips. Each pod includes 8,960 chips, which are interconnected via proprietary optical circuit switches (OCS) and provide a throughput of 4,800 Gbps.

The TPU v5p pods are significantly larger in processing scope compared to pods with the older TPUv4 chips, which were introduced in 2020. The TPUv4 pods had 4,096 chips, which are also interconnected via OCS.

Google’s new AI chip, the Cloud TPU v5p, released alongside the Gemini large-language model. (Source: Google)

IMAGE – TPU v5p, our most powerful and scalable TPU to date.

“Compared to TPU v4, TPU v5p features more than 2X greater FLOPS and 3X more high-bandwidth memory (HBM),” Google said.

The TPU v5p pods have 95GB of high-bandwidth memory, while the TPU v4 pods have 32GB of HBM. The HBM bandwidth of a TPU v5p pod is 2,765GBps, while the TPU v4 bandwidth was 1228GBps.

The company claimed that TPU v5p can train large-language models 2.8 times faster than TPU v4. However, the data is misleading; the chart showed TPUv5 relative performance measured on the INT8 data type and the TPUv4 based on the BF16 data type.

The TPU v4 did not support INT8, and Google shared a more realistic number — the TPU v5p being 67% faster than TPUv4 on BFloat16.

With TPU v5p, Google now has a chip that can be compared to Nvidia’s H100. Google researchers previously compared the TPUv4 to Nvidia’s A100, in line with the relative product release dates, and claimed the TPU v4 is 1.2x-1.7x faster and used 1.3x-1.9x less power on similarly sized systems.

Earlier this year, Google released another TPU v5 variant, the TPUv5e, which is a power-efficient variant designed for inferencing. Google used TPU v4 and V5e to train the Gemini transformer model. Each TPU v5e chip has four matrix multiplication units, a vector, and a scalar processing unit, all connected to HBM2 memory.

The Hypercomputer with TPU v5p can flip the switch for different types of users and their commitment. The system is flexible enough to assign dedicated resources to big-buck customers with long-term commitments and has more flexible systems for on-demand users. That has helped Google create a more efficient way for customers to consume cloud services.

The consumption system is driven by a feature called the Dynamic Work Scheduler, which is designed so users do not have to wait to execute workloads or see results. Google said the Flex Start mode is “for higher resource obtainability and optimized economics,” and the Calendar mode “targets workloads with higher predictability on job-start times.”

Behind the consumer-facing consumption model is the internal system of software and chips that run the AI models.

The software layer includes an open-source software framework that provides model flexibility with support for JAX, PyTorch, and TensorFlow. Cloud services rely on virtualization technology for provisioning, orchestration, and management of complex AI workloads, which includes slicing up training tasks to virtual machines and coordinating the processing and delivery of results. The Hypercomputer also supports the Google Kubernetes Engine and Compute Engine for container management.

The Hypercomputer software layer also includes support for XLA, which compiles for AI models to run on any hardware. To use AI models, customers can choose between GPUs, CPUs, or Google’s TPUs.

“It optimizes distributed architectures across a wide range of hardware platforms, ensuring easy-to-use and efficient model development for diverse AI use cases,” Google said.

Google already offers supercomputers, such as A3, that hosts 26,000 Nvidia GPUs, but it remains unclear how the Hypercomputer will link up to GPUs. The AI infrastructure typically dedicates specific hardware configurations with forks of generative AI models. Google points to a range of models.

The Hypercomputer is glued together by the Jupiter interconnect, which includes the optical switching system. The 4,800Gbps bandwidth of TPU v5p is two times that of TPUv4.

The optical infrastructure is fitted in a swap-in, swap-out infrastructure that links up AI chips and clusters. Each rack operates independently and can scale up based on computing requirements. The ability to scale computing on the fly and reconfigure the network topology depending on the application and how it fits into Google’s dynamic cloud consumption model.

Google didn’t provide specific details on the number of cores or other specifications in the TPU v5p. But Google introduced many new technologies with the v5e, including Multislice, which can network hundreds of thousands more AI chips together in a cluster. The technology is now supported by TPU v5p.

The TPU v5 first surfaced when Google’s researchers talked about the chip’s development in June 2021. At the time, Google researchers said AI was used to design the chip. The researchers claimed AI floor-planned the chip significantly faster than human experts. Google fired one researcher ahead of the paper’s appearance in Nature magazine, and the company’s claims are now being questioned. It’s unclear if the new chip is based on the paper published by researchers.

Google is making its Hypercomputer available to customers via its cloud service. Pricing is still not available.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket for an optional 8087 math coprocessor. The math coprocessor ma Read more…

IonQ Reports Advance on Path to Networked Quantum Computing

February 22, 2024

IonQ reported reaching a milestone in its efforts to use entangled photon-ion connectivity to scale its quantum computers. IonQ’s quantum computers are based on trapped ions which feature long coherence times and qubit Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Apple. Today the consumer electronics giant started rolling Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to derive any substantial value from it. However, the GenAI hyp Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks – that builds on earlier work and is an eff Read more…

AWS Solution Channel

Shutterstock 2283618597

Deep-dive into Ansys Fluent performance on Ansys Gateway powered by AWS

Today, we’re going to deep-dive into the performance and associated cost of running computational fluid dynamics (CFD) simulations on AWS using Ansys Fluent through the Ansys Gateway powered by AWS (or just “Ansys Gateway” for the rest of this post). Read more…

Atom Computing Reports Advance in Scaling Up Neutral Atom Qubit Arrays

February 15, 2024

The scale-up challenge facing quantum computing (QC) is daunting and varied. It’s commonly held that 1 million qubits (or more) will be needed to deliver practical fault tolerant QC. It’s also a varied challenge beca Read more…

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket fo Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performa Read more…

The Pulse of HPC: Tracking 4.5 Million Heartbeats of 3D Coronary Flow

February 15, 2024

Working in Duke University's Randles Lab, Cyrus Tanade, a National Science Foundation graduate student fellow and Ph.D. candidate in biomedical engineering, is Read more…

It Doesn’t Get Much SWEETER: The Winter HPC Computing Festival in Corpus Christi

February 14, 2024

(Main Photo by Visit Corpus Christi CrowdRiff) Texas A&M University's High-Performance Research Computing (HPRC) team hosted the "SWEETER Winter Comput Read more…

Q-Roundup: Diraq’s War Chest, DARPA’s Bet on Topological Qubits, Citi/Classiq Explore Optimization, WEF’s Quantum Blueprint

February 13, 2024

Yesterday, Australian start-up Diraq added $15 million to its war chest (now $120 million) to build a fault tolerant computer based on quantum dots. Last week D Read more…

2024 Winter Classic: Razor Thin Margins in HPL/HPCG

February 12, 2024

The first task for the 11 teams in the 2024 Winter Classic student cluster competition was to run and optimize the LINPACK and HPCG benchmarks. As usual, the Read more…

2024 Winter Classic: We’re Back!

February 9, 2024

The fourth edition of the Winter Classic Invitational Student Cluster Competition is up and running. This year, we have 11 teams of eager students representin Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire