The University of Michigan (U-M) today announced Dell EMC is the lead vendor for U-M’s $4.8 million Great Lakes HPC cluster scheduled for deployment in first half of 2019. The new machine is intended to replace the eight-year-old Flux cluster. Noteworthy, Great Lakes will be the first system in the world to use Mellanox’s HDR 200 Gigabit InfiniBand networking solution. DDN will provide the storage solution.

“High-performance research computing is a critical component of the rich computing ecosystem that supports the university’s core mission,” said Ravi Pendse, U-M’s VP for information technology and chief information officer. “With Great Lakes, researchers in emerging fields like machine learning and precision health will have access to a higher level of computational power. We’re thrilled to be working with Dell EMC, Mellanox, and DDN; the end result will be improved performance, flexibility, and reliability for U-M researchers.”

As noted in the release, the main components of the new cluster are:

• Dell EMC PowerEdge C6420 compute nodes, PowerEdge R640 high memory nodes, and PowerEdge R740 GPU nodes
• Mellanox HDR 200Gb/s InfiniBand ConnectX-6 adapters, Quantum switches and LinkX cables, and InfiniBand gateway platforms
• DDN Gridscaler 14kx and 100 TB of usable IME (Infinite Memory Engine) memory

Further details of Great Lakes’ specs were not released. According to U-M, the existing Flux system consists of approximately 27,000 cores – including 1,372 compute nodes composed of multiple CPU cores, with at least 4 GB of RAM per core, interconnected with InfiniBand networking.

“While Flux, our current cluster, has been serving the university research community quite well, and we have been refreshing cores on a rotating basis over time, a number of nodes are several years old,” said Brock Palen, director of Advanced Research Computing – Technology Services (ARC-TS) at U-M.

“One of the benefits of creating an entirely new cluster will be to create a more homogeneous platform. We have an increasing number of researchers needing accelerated capabilities like machine learning and molecular dynamics, for example, but we are still primarily a general-purpose provider. So we aimed at further optimizing our cluster to support a wide variety of workloads, whether accelerated, CPU, or I/O bound,” Palen said.

ARC-TS supports a variety of systems (see below) including, for example, ConFlux, an IBM Power8-based cluster intended for data-driven science applications, a Hadoop cluster, a high-end storage system, and U-M is also a Globus provider. The IBM-based system was built in 2015 with an NSF grant aimed specifically at physics-constrained data-driven modeling.

Palen noted, “As a public university, we have relatively strict procurement requirements, which resulted in our selection of Dell. That selection does not reflect negatively on any other company with which we have partnered.”

Great Lakes is expected to provide significant improvement in computing performance over Flux. “[E]ach compute node will have more cores, higher maximum speed capabilities, and increased memory. The cluster will also have improved internet connectivity and file system performance, as well as “Nvidia Tensor GPU cores, which are very powerful for machine learning compared to prior generations of GPUs,” according to the announcement.

“Users of Great Lakes will have access to more cores, faster cores, faster memory, faster storage, and a more balanced network,” said Palen.

Not surprisingly the three lead contractors were happy for the win.

“Dell EMC is thrilled to collaborate with the University of Michigan and our technology partners to bring this innovative and powerful system to such a strong community of researchers,” said Thierry Pellegrino, vice president, Dell EMC High Performance Computing. “This Great Lakes cluster will offer an exceptional boost in performance, throughput and response to reduce the time needed for UM researches to make the next big discovery in a range of disciplines from artificial intelligence to genomics and bioscience.”

“HDR 200G InfiniBand provides the highest data speed and smart In-Network Computing acceleration engines, delivering HPC and AI applications with the best performance, scalability and efficiency,” said Gilad Shainer, vice president of marketing at Mellanox Technologies. “We are excited to collaborate with the University of Michigan, Dell EMC and DataDirect Networks, in building a leading HDR 200G InfiniBand-based supercomputer, serving the growing demands of UM researchers.”

“DDN has a long history of working with Dell EMC and Mellanox to deliver optimized solutions for our customers. We are happy to be a part of the new Great Lakes cluster, supporting its mission of advanced research and computing. Partnering with forward-looking thought leaders as these is always enlightening and enriching,” said James Coomer, SVP product marketing and benchmarks at DDN.

Link to announcement: https://arc-ts.umich.edu/great-lakes-announcement/