The three national laboratories (Lawrence Livermore, Los Alamos and Sandia) that support the National Nuclear Security Administration (NNSA) occupy a strange role in the landscape of government-funded research and supercomputing. The NNSA manages the military applications of nuclear science (i.e. the country’s nuclear weapons) – a demanding task that requires a large amount of relatively opaque supercomputing to effectively model the behavior and effects of such powerful, secretive, and rarely tested devices. Now, HPE has announced that it is adding another weapon to the NNSA’s supercomputing stockpile: a new system named “Crossroads.”
Crossroads will be powered by the new HPE Cray EX supercomputer, a liquid-cooled, blade-based, high-density cluster system that will use forthcoming Intel Xeon processors (10nm “Sapphire Rapids”) with Advanced Matrix Extensions (for AI acceleration) and “an advanced memory architecture.” The system is also intended to be optimized for 3D modeling at high resolutions: a major priority for nuclear weapons modeling. HPE plans to deliver Crossroads to Los Alamos National Laboratory (LANL) in the spring of 2022, allowing for a useful lifetime from 2022 through 2026. The system will be provided through a $105 million deal between the Department of Energy and HPE.
“HPE has a long history in designing a number of HPC systems for the National Nuclear Security Administration, and today’s announcement further builds on our commitment to deliver powerful and diverse architectures required for critical research missions in maintaining the reliability and security of the nation’s nuclear stockpile,” said Bill Mannel, vice president and general manager for HPC at HPE. “In this collaboration with Los Alamos National Laboratory and Sandia National Laboratories, we are significantly advancing existing resources for NNSA research with the HPE Cray EX supercomputer, which combines the Cray Shasta architecture and industry-leading HPC software and networking interconnect. We are also quadrupling performance with combined technologies featuring advanced compute, I/O, memory, networking and software to enable faster and more efficient performance for current and future projects.”
This “quadrupled performance” is likely a comparison to the system Crossroads will be succeeding: Trinity, a Cray system that was partially deployed at LANL in 2015 and fully deployed by 2016. According to LANL, Trinity will be reaching the end of its useful lifetime in 2021. Trinity’s two partitions – one Intel Haswell-based, one Intel Knights Landing-based – deliver 20.2 Linpack petaflops. The system debuted on the Top500 list in 2015 in sixth place and, over the years, fell to eleventh place, where it currently stands.
Like other NNSA supercomputers, Crossroads will be used to help ensure the nation’s nuclear weapon stockpile is “safe, secure and reliable.” LANL cites uses in supporting stockpile stewardship certifications and assessments; quantifying and aggregating uncertainties in simulation tools; and providing mission-responsible computational environments. These uses are compounded by general fidelity improvements, including increased fidelity in physical, numerical and geometric modeling. “Ultimately,” LANL wrote, “Crossroads will accelerate industry’s research and development (R&D) technology roadmap for the ASC program’s future exascale-class computational needs.”
“This machine will advance our ability to study the most complex physical systems for science and national security. We look forward to its arrival and deployment,” said Jason Pruet, program director for the Advanced Simulating and Computing Program at LANL.
Crossroads also operates under the Alliance for Computing at Extreme Scale (ACES), a partnership program between LANL and Sandia National Laboratories (SNL). “Sandia is pleased to be part of the ACES team,” said S. Scott Collis, director of the Center for Computing Research at Sandia National Laboratories, “and we anticipate that the Crossroads supercomputer will be highly efficient at running the largest physics and engineering calculations in support of our nation’s nuclear deterrent.”
Crossroads is the latest in a series of wins for the HPE Cray EX supercomputer (formerly known as Shasta), which is the basis for Perlmutter (NERSC) as well as future Frontier, El Capitan and Aurora systems (at ORNL, LLNL and ANL, respectively). Unlike those systems, which all employ heterogeneous CPU-GPU designs, Crossroads is a straight x86 CPU system.