Top500: Aurora Is in the House, But It Is Still Frontier’s Party

By Doug Eadline

November 13, 2023

The fall 2023 TOP500 list is out and Frontier retains its top spot and is still the only exascale machine. However, five new or upgraded systems have shaken up the Top 10.

Located at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, Frontier leads the pack with an HPL score of 1.194 EFlop/s – unchanged from the June 2023 list. Frontier utilizes AMD EPYC 64C 2GHz processors based on the latest HPE Cray EX235a architecture. The system has a total of 8,699,904 combined CPU and GPU cores. Additionally, Frontier has an impressive power efficiency rating of 52.59 GFlops/watt and relies on Cray’s Slingshot 11 network for data transfer.

The Frontier Exascale system, located at Oak Ridge National Laboratory (ORNL), Tennessee, USA.

The “new” Aurora system at the Argonne Leadership Computing Facility in Illinois, USA, entered the list at the No. 2 spot – previously held by Fugaku, with an HPL score of 585.34 PFlop/s. Argonne may want a footnote next to Aurora numbers indicating the submission was for half of the planned final system. However, you go with what you got in this case. Aurora is being commissioned and will reportedly exceed Frontier with a peak performance of 2 EFlop/s when finished.

Aurora is built by Intel and is based on the HPE Cray EX – Intel Exascale Compute Blade, which uses Intel Xeon CPU Max Series processors and Intel Data Center GPU Max Series accelerators. These communicate through Cray’s Slingshot-11 network interconnect.

In the list, 20 new systems now use Intel Sapphire Rapids CPUs. Bringing the total number of systems using this CPU to 25, the Intel Sapphire Rapids CPU is currently leading the new CPU among new systems. However, of the 45 new systems on the list, only four use the corresponding Intel GPU, with Aurora being the largest by far.

Another new system named Eagle, installed in the Microsoft Azure Cloud in the USA, has taken the No. 3 spot. This system is the highest rank a cloud system has ever achieved on the TOP500. Only two years ago, a previous Azure system was the first cloud system ever to enter the TOP10 at spot No. 10. This Microsoft NDv5 system has an HPL score of 561.2 PFlop/s. It is based on Intel Xeon Platinum 8480C processors and Nvidia H100 accelerators. 

Fugaku has moved to its current ranking of No. 4 after achieving No. 2 in the June 2023 list and holding the No. 1 spot from June 2020 until November 2021. This system is based in Kobe, Japan, and has an HPL score of 442.01 PFlop/s. It remains the highest-ranked system outside the USA.

The LUMI system based at Euro HPC/CSC in Kajaani, Finland, achieved the No. 5 spot with an HPL score of 379.70 PFlop/s. This system is the largest in Europe and has seen multiple upgrades that keep it near the top of the list, this time improving from an HPL score of 309.10 PFlop/sec on the last list.

Other TOP500 Highlights

The 62nd edition of the TOP500 shows that Intel, AMD, and IBM processors are the preferred choice for HPC systems. Out of the TOP10, five systems use Intel Xeon processors (Aurora, Eagle, Leonardo, MareNostrum 5 ACC, and EOS Nvidia DGX SuperPod), two systems use AMD processors (Frontier and LUMI), and two systems use IBM processors (Summit and Sierra.)

Much like it has been on many previous lists, China and the United States earned the most entries on the entire TOP500 list. The United States increased its lead from 150 machines on the last list to 161 on the current list, while China dropped from 134 to 104.

In terms of entire continents, North America improved from 160 machines on the previous list to 171 on this one, Asia decreased from 192 machines to 169, and Europe increased from 133 systems to 143.

Ethernet is still the most used interconnect on the list, although it decreased its share from 227 machines on the previous list to 209. Infiniband came in second with 219 machines, which increased over the prior count of 200 machines. Omnipath came in third with 33 machines on this list, and custom interconnects came in fourth with 30 machines.

Top 10 Summary

  • Frontier remains the No. 1 system in the TOP500. This HPE Cray EX system is the first US system with a performance exceeding one Exaflop/s. It is installed at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, operated by the Department of Energy (DOE). It currently has achieved 1.194 Exaflop/s using 8,699,904 cores. The HPE Cray EX architecture combines 3rd Gen AMD EPYC™ CPUs optimized for HPC and AI with AMD Instinct™ 250X accelerators and a Slingshot-11 interconnect.
  • Aurora achieved the No. 2 spot by submitting an HPL score of 585 Pflop/s measured on half of the complete system. It is installed at the Argonne Leadership Computing Facility, Illinois, USA, where it is also operated by the Department of Energy (DOE). This new Intel system is based on HPE Cray EX – Intel Exascale Compute Blades. It uses Intel Xeon CPU Max Series processors, Intel Data Center GPU Max Series accelerators, and a Slingshot-11 interconnect.
  • Eagle, the new No. 3 system, is installed by Microsoft in its Azure cloud. This Microsoft NDv5 system is based on Xeon Platinum 8480C processors and Nvidia H100 accelerators and achieved an HPL score of 561 Pflop/s.
  • Fugaku, the No. 4 system, is installed at the RIKEN Center for Computational Science (R-CCS) in Kobe, Japan. It has 7,630,848 cores, which allowed it to achieve an HPL benchmark score of 442 Pflop/s. 
  • The (again) upgraded LUMI system, another HPE Cray EX system installed at the EuroHPC center at CSC in Finland, is now the No. 5 with a performance of 380 Pflop/s. The European High-Performance Computing Joint Undertaking (EuroHPC JU) is pooling European resources to develop top-of-the-range Exascale supercomputers for processing big data. One pan-European pre-Exascale supercomputer, LUMI, is in CSC’s data center in Kajaani, Finland.
  • The No. 6 system Leonardo is installed at a different EuroHPC site in CINECA, Italy. It is an Atos BullSequana XH2000 system with Xeon Platinum 8358 32C 2.6GHz as main processors, Nvidia A100 SXM4 40 GB as accelerators, and Quad-rail Nvidia HDR100 Infiniband as interconnect. It achieved a Linpack performance of 238.7 Pflop/s.
  • Summit, an IBM-built system at the Oak Ridge National Laboratory (ORNL) in Tennessee, USA, is now listed at the No. 7 spot worldwide with a performance of 148.8 Pflop/s on the HPL benchmark, which is used to rank the TOP500 list. Summit has 4,356 nodes, each housing two POWER9 CPUs with 22 cores and six Nvidia Tesla V100 GPUs, each with 80 streaming multiprocessors (SM). The nodes are linked together with a Mellanox dual-rail EDR InfiniBand network.
  • The MareNostrum 5 ACC system is new at No. 8 and installed at the EuroHPC/Barcelona Supercomputing Center in Spain. This BullSequana XH3000 system uses Xeon Platinum 8460Y processors with NVIDIA H100 and Infiniband NDR200. It achieved 183.2 Pflop/s HPL performance.
  • The new Eos system listed at No. 9 is an Nvidia DGX SuperPOD-based system at Nvidia, USA. It is based on the Nvidia DGX H100 with Xeon Platinum 8480C processors, Nvidia H100 accelerators, and Infiniband NDR400 and it achieves 121.4 Pflop/s.
  • Sierra, a system at the Lawrence Livermore National Laboratory, CA, USA, is at No. 10. Its architecture resembles the #7 system’s Summit. It is built with 4,320 nodes, each with two POWER9 CPUs and four Nvidia Tesla V100 GPUs. Sierra achieved 94.6 Pflop/s.

GREEN500 Results

The No. 1 spot on the GREEN500 was claimed by Henri, located at the Flatiron Institute in New York, USA. The system achieved an energy efficiency rating of 65.40 GFlops/Watt while producing an HPL score of 2.88 PFlop/s. Henri is a Lenovo ThinkSystem SR670 with Intel Xeon Platinum and Nvidia H100. It has 8,288 total cores, and it ranked No. 293 on the TOP500 list.

The Frontier Test & Development (TDS) system at ORNL in Tennessee, USA, claims the No. 2 spot with an energy efficiency rating of 62.68 GFlops/Watt and an HPL score of 19.2 PFlop/s. The TDS is just one rack identical to the Frontier system and utilizes 120,832 cores.

The No. 3 spot was taken by the Adastra system, which is housed at GENCI-CINES in France. The system achieved an energy efficiency rating of 58.02 GFlops/Watt and an impressive HPL score of 46.1 PFlop/s. Adastra has 319,072 total cores.

Additionally, just like on the last list, the actual Frontier system at No. 1 on the TOP500 deserves an honorable mention here for its achievements in energy efficiency. Despite more than doubling the HPL score of the Aurora system at the No. 2 spot on the TOP500, Frontier took the No. 8 spot on the GREEN500 with an energy efficiency of 52.59 GFlops/Watts.

Considering this system was the first machine to achieve exascale, Frontier proves that power does not need to be sacrificed to achieve an impressive energy efficiency rating.

HPCG Results

The TOP500 list has incorporated the High-Performance Conjugate Gradient (HPCG) benchmark results, which provide an alternative metric for assessing supercomputer performance. This score is meant to complement the HPL measurement to give a fuller understanding of the machine.

Like on the June 2023 list, Fugaku remains the leader with an HPCG benchmark of 16 PFlop/s. Frontier claimed the No. 2 spot with 14.05 HPCG-PFlop/s, and No. 3 was taken by LUMI with 4.59 HPCG-PFlop/s.

HPL-MxP Results (Formerly HPL-AI)

The HPL-MxP benchmark seeks to highlight the use of mixed precision computations. Traditional HPC uses 64-bit floating point computations. Today, we see hardware with various levels of floating-point precisions – 32-bit, 16-bit, and even 8-bit. The HPL-MxP benchmark demonstrates that much higher performance is possible by using mixed precision during computation. By using mathematical techniques, the same accuracy can be computed with a mixed-precision technique when compared with straight 64-bit precision.

Frontier was the clear winner here with an HPL-MxP score of 9.95 EFlop/s. LUMI claimed the No. 2 spot with a score of 2.35 EFlop/s, and Fugaku took the No. 3 spot with a score of 2.0 EFlop/s.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket for an optional 8087 math coprocessor. The math coprocessor ma Read more…

IonQ Reports Advance on Path to Networked Quantum Computing

February 22, 2024

IonQ reported reaching a milestone in its efforts to use entangled photon-ion connectivity to scale its quantum computers. IonQ’s quantum computers are based on trapped ions which feature long coherence times and qubit Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Apple. Today the consumer electronics giant started rolling Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to derive any substantial value from it. However, the GenAI hyp Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks – that builds on earlier work and is an eff Read more…

AWS Solution Channel

Shutterstock 2283618597

Deep-dive into Ansys Fluent performance on Ansys Gateway powered by AWS

Today, we’re going to deep-dive into the performance and associated cost of running computational fluid dynamics (CFD) simulations on AWS using Ansys Fluent through the Ansys Gateway powered by AWS (or just “Ansys Gateway” for the rest of this post). Read more…

Atom Computing Reports Advance in Scaling Up Neutral Atom Qubit Arrays

February 15, 2024

The scale-up challenge facing quantum computing (QC) is daunting and varied. It’s commonly held that 1 million qubits (or more) will be needed to deliver practical fault tolerant QC. It’s also a varied challenge beca Read more…

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket fo Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performa Read more…

The Pulse of HPC: Tracking 4.5 Million Heartbeats of 3D Coronary Flow

February 15, 2024

Working in Duke University's Randles Lab, Cyrus Tanade, a National Science Foundation graduate student fellow and Ph.D. candidate in biomedical engineering, is Read more…

It Doesn’t Get Much SWEETER: The Winter HPC Computing Festival in Corpus Christi

February 14, 2024

(Main Photo by Visit Corpus Christi CrowdRiff) Texas A&M University's High-Performance Research Computing (HPRC) team hosted the "SWEETER Winter Comput Read more…

Q-Roundup: Diraq’s War Chest, DARPA’s Bet on Topological Qubits, Citi/Classiq Explore Optimization, WEF’s Quantum Blueprint

February 13, 2024

Yesterday, Australian start-up Diraq added $15 million to its war chest (now $120 million) to build a fault tolerant computer based on quantum dots. Last week D Read more…

2024 Winter Classic: Razor Thin Margins in HPL/HPCG

February 12, 2024

The first task for the 11 teams in the 2024 Winter Classic student cluster competition was to run and optimize the LINPACK and HPCG benchmarks. As usual, the Read more…

2024 Winter Classic: We’re Back!

February 9, 2024

The fourth edition of the Winter Classic Invitational Student Cluster Competition is up and running. This year, we have 11 teams of eager students representin Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire