The long-troubled, hotly anticipated MareNostrum 5 supercomputer finally has a vendor: Atos, which will be supplying a system that includes both Nvidia and Intel CPUs and GPUs across multiple partitions. The newly reimagined system—which had its tender reissued in late 2021 after it was canceled earlier in the year—will deliver 314 aggregated peak petaflops. The system is expected to deploy next year.
Previously
Some brief history: back in June 2019, BSC was one of eight sites selected for EuroHPC’s inaugural octet of supercomputers—specifically, one of the three selected to host a “pre-exascale” system measuring in the hundreds of petaflops. News of procurements—and even installations—of its siblings came and went over the ensuing years until, abruptly, the procurement for MareNostrum 5 was canceled in May 2021.
Politico reported that competing bids from Atos and IBM/Lenovo had brought the decision-makers to a deadlock, with the European Commission representatives favoring the bid from the European company, Atos, and the Spanish representatives favoring the IBM/Lenovo bid. Other accounts attributed the canceled procurement to a need to update the system’s specifications.
In late December 2021, the procurement for MareNostrum 5 was reissued, soliciting a system for €151,410,000 with a close date of January 31, 2022. (A couple of weeks later, the close date was amended to February 21, 2022.) That €151,410,000 cost—around $160 million USD—was designed to be borne equally by the EuroHPC Joint Undertaking and a consortium consisting of Spain, Portugal and Turkey.
Specifications
Now, Atos has been selected as the vendor for MareNostrum 5—an easy bet, perhaps, in the wake of louder calls and stronger efforts to reinforce European supercomputing sovereignty amid the Covid-19 pandemic, supply chain disruptions and geopolitical tension.
Nvidia told HPCwire that MareNostrum 5 will feature Nvidia’s H100 Tensor Core GPUs and its Quantum-2 (aka NDR) InfiniBand networking. The system will also include Nvidia’s Arm-based Grace CPU Superchips, which feature tightly paired CPUs and which are slated to debut on the Venado system at Los Alamos National Laboratory. (More on Nvidia’s Grace and Hopper tech here.)
That was confirmed at the 79th HPC User Forum Forum at Oak Ridge National Laboratory, where Leonardo Flores, senior HPC expert for the European Commission, shared a more specific breakdown of the system.
MareNostrum 5, which will combine Atos’ BullSequana XH3000 architecture and Lenovo’s ThinkSystem architecture, will have an aggregated peak performance of 314 petaflops and aggregated Linpack performance of 205 petaflops. It will contain four partitions:
- One accelerated partition powered by Nvidia Hopper GPUs and Intel Sapphire Rapids CPUs and weighing in at 163 Linpack petaflops.
- A second accelerated partition powered by Intel Rialto Bridge GPUs, the successor to the Ponte Vecchio GPU that is slated to debut in the Aurora exascale supercomputer, and next-gen Intel Emerald Rapids CPUs. This partition will weigh in at 4 Linpack petaflops.
- A CPU partition powered by Intel Sapphire Rapids CPUs and weighing in at 36 Linpack petaflops.
- Finally, a second CPU partition powered by Nvidia Grace Superchips and weighing in at 2 Linpack petaflops.
EuroHPC’s details on the system share that it will have 248PB of net capacity, and Nvidia adds that the system is expected to deliver 18 exaflops of AI performance, making it the fastest AI supercomputer in the European Union. BSC elaborated that the system will be “fully powered with green energy, and will utilize heat reuse technology.”
“We hope that we could see [the] first installation capable of ranking in June 2023,” Flores said, “with, hopefully, a full installation and ranking in November 2023.” The new datacenter for MareNostrum 5 (which is housed in BSC’s new facilities adjacent to the chapel that hosts MareNostrum 4) was already well underway last summer when the previous procurement was canceled. EuroHPC says that the datacenter has a total power capacity of 20MW, a cooling capacity of 17MW and a PUE below 1.08.
Applications
MareNostrum 5 is specifically targeted for medical research and drug development—one of the major adjustments in the specifications of the system following its reissued procurement. The system will also support applications like climate science and engineering. Nvidia Omniverse will power the development of digital twins for these applications.
“The acquisition of MareNostrum 5 will enable world-changing scientific breakthroughs such as the creation of digital twins to help solve global challenges like climate change and the advancement of precision medicine,” said Mateo Valero, director of BSC. “In addition, [BSC] is committed to developing European hardware to be used in future generations of supercomputers and helping to achieve technological sovereignty for the EU’s member states.”
EuroHPC
This is only the latest announcement for EuroHPC, which has had an extraordinarily busy few weeks amid the launch of its first pre-exascale system (LUMI) and the site selection for five new supercomputers (including EuroHPC’s first exascale system, to be hosted by Germany). To learn more about the state of EuroHPC, read HPCwire‘s additional coverage here.
Header image: BSC’s new corporate headquarters. Image courtesy of BSC.