The University of Bath is upgrading its HPC infrastructure, which it says “supports a growing and wide range of research activities across the University.” The previous system, Balena, is being replaced by a new, Microsoft Azure-based cloud HPC environment called Janus. Janus will be complemented by a “bespoke on-campus [high-throughput computing] cluster running software applications that can’t be moved off campus.”
Janus, the University said, makes it “the first university worldwide to move almost all its HPC research to Microsoft Azure platform[.]” The new cloud environment contains an unspecified smattering of HB-series, HBv2-series, HBv3-series nodes, which contain between 16 and 120 AMD CPUs per node; HC-series nodes, which contain 44 Intel CPUs per node; Fsv2-series nodes, which contain between two and 72 Intel CPUs per node; and NCv3-series and NDv2-series nodes, which contain between one and eight Nvidia Tesla V100 GPUs per node.
These nodes are supported by an aggregate of 34TB of dedicated storage (including 16TB for storing data and 16TB of “scratch space”) and Mellanox EDR InfiniBand networking, all of which is connected via VPN to Bath through a 1GB/s connection. Janus uses Azure CycleCloud for automated configuration and management of the environment and Slurm for cluster job scheduling.
Meanwhile, the new HTC cluster, named Anatra, includes eight compute nodes, each equipped with dual AMD Milan CPUs, 256GB of memory and 960GB of NVMe storage. Anatra is based on RedHat Enterprise Linux v8 and uses a Slurm scheduler.
The University says that Janus will enable a myriad of benefits for its students and researchers, including “significantly reduced” compute times and “a large diversity of cutting edge compute options, regularly updated to meet evolving research needs.” Indeed, the Janus announcement follows the University’s two years of exploration into the suitability of cloud infrastructure, including an HPC cloud service pilot project that it ran last summer with two dozen researchers.
“We used Azure as part of the pilot project for running our computational fluid dynamics simulations with the program OpenFoam,” said Katharine Fraser, a lecturer in the Department of Mechanical Engineering at the University of Bath. “It was great to be able to run these large simulations just when we wanted, rather than waiting in the queue, and transitioning from Balena was straightforward with Azure having a user-friendly interface. … In future, the wide variety of nodes will be beneficial for running different types of simulations. The flexibility that cloud computing promises for engineering research is really exciting!”
Both Janus and Anatra are still under development and “will gradually be made available to researchers and postgraduate students at the University.” The University is accepting applications for early-access user groups.
Balena, which Janus and Anatra will eventually replace, is pictured in the header image and includes 196 Intel Ivy Bridge nodes and 17 Intel Skylake nodes and delivers 63.2 peak teraflops of compute power.