Over the last nine months, innumerable HPC systems around the world have turned their computing power toward the coronavirus – however, relatively few have been purpose-delivered with COVID-19 in mind. Lawrence Livermore National Laboratory (LLNL), one of the national laboratories under the umbrella of the National Nuclear Security Administration (NNSA), is joining that short list with a new, high-memory cluster called “Mammoth,” which LLNL is already putting to work on COVID-19 research.
Beyond being immediately purposed for COVID-19 research, the cluster was actually procured using funds from the Coronavirus Aid, Relief and Economic Security (CARES) Act. The CARES Act, which became a law in late March, offers funding to Department of Energy (DOE) laboratories for COVID-19 research and associated infrastructure. Lawrence Livermore was already the recipient of another CARES Act grant less than a month ago, receiving funding for a major upgrade of its Corona system that will more than double its peak computing power. Los Alamos National Laboratory – another NNSA lab – similarly received additional funding for its new Chicoma system to enhance its capabilities for COVID-19 research.
Mammoth will consist of 64 nodes, each equipped with twin second-generation AMD Epyc CPUs and two terabytes of memory (for a total of 128 TB) and almost four terabytes of non-volatile memory (for a total of around 256 TB). The system, which will be provided by Supermicro, will use Omni-Path networking from Cornelis Networks (a spin-out of Intel that launched just two months ago) that is purpose-built for high-performance data analytics and AI – the first production AMD-based cluster to implement this technology. LLNL estimates that Mammoth will deliver 294 peak teraflops of computing power.
This power, LLNL says, will be perfect for COVID-19 research – in particular, genomics analysis, non-traditional HPC simulations and graph analytics, all of which are common use cases for COVID-19 researchers. The memory, meanwhile, is described as “crucial” for handling the enormous datasets often in play in COVID-19 studies and simulations.
“The ability of large-memory systems to integrate genomic analysis with large-scale machine learning for predictive modeling of therapeutic response will be important for accelerating the development of effective new therapeutics,” said Jim Brase, LLNL’s deputy associate director for Computing. “Mammoth will be integral for developing new tools to combat COVID-19, but also for fast response in a future pandemic.”
Indeed, this competency is not merely theoretical: Mammoth has already been yoked to active COVID-19 research at LLNL. Researchers at the lab are using Mammoth to run molecular dynamics simulations, structural modeling and more to analyze SARS-CoV-2’s genomic evolution and assess how the virus’ structure changes when mutations are introduced.
“This new system makes a big difference in how we prepare our jobs for calculations and in their performance,” said Adam Zemla, one of the researchers using Mammoth for COVID-19 studies. “With Mammoth I can easily process COVID-19 genomes without the need to split datasets into smaller chunks, like I have had to do on previous machines.”
By way of example, the researchers described how they were able to execute “many more” calculations using Rosetta (code for computing binding free energies) than they were on previous systems, which they say were too memory-limited to run more than 12 or 16 calculations per node at a time. Mammoth, they report, allows them to run 128 simultaneous Rosetta calculations on a single node.