Dell EMC announced yesterday it is building a new supercomputer – the OzStar – for the Swinburne University of Technology (Australia) in support the ARC Centre of Excellence for Gravitational Wave Discovery (OzGrav). The OzGrav project was first announced last September. The OzStar supercomputer will be based on Dell EMC PowerEdge R740 nodes, have more than one petaflops capability, and is expected to be completed in September.

OzGrav will use the new machine in support of efforts to understand the extreme physics of black holes and warped space-time. Among other projects, OzGrav will process data from LIGO (Laser Interferometer Gravitational Wave Observatory) gravitational wave detectors and the Square Kilometre Array (SKA) radio telescope project with facilities built in Australia and South Africa.

The OzStar’s architecture will leverage advanced Intel (Xeon V5) and Nvidia (P100) technology and feature three building blocks: Dell EMC 14th Generation PowerEdge R740 Servers; Dell EMC H-Series Networking Fabric; and Dell EMC HPC Storage with Intel Lustre filesystem. The networking fabric is Intel Omni Path Architecture and will provide “86.4 Terabits per second of aggregate network bandwidth at 0.9 µs latency” according to Dell EMC. As is typical in such contracts, Dell EMC will provide support.

Here’s snapshot of OzStar’s specs as provided by Dell EMC:

• 4,410 x86 cores at 2.3Ghz across 107 standard compute and eight data crunching nodes
• 230 x NVIDIA Tesla P100 12GB GPUs (one per CPU socket)
• 272 Intel Xeon Phi cores at 1.6Ghz across 4 C6320p KNL nodes
• A high-speed, low latency network fabric able to move data across each building block at over 100Gbps with various features to ensure reliability and traffic flow (9 Dell EMC H1048 switches), as well as GPU-Direct capability
• 5PiB of usable storage via the Lustre file system at 60GB/s throughput, on the Dell EMC PowerVault MD3060e storage with R740 controllers (10 Dell EMC PowerEdge R740 + 16 MD3060e and 1 MD3240)

“While Einstein predicted the existence of gravitational waves, it took one hundred years for technology to advance to the point they could be detected,” said Professor Matthew Bailes, director of OzGrav, Swinburne University of Technology. “Discoveries this significant don’t occur every day and we have now opened a new window on the Universe. This machine will be a tremendous boost to our brand-new field of science and will be used by astrophysicists at our partner nodes as well as internationally.”

“This combination of Dell EMC technologies will deliver the incredibly high computing power required to move and analyze data sets that are literally astronomical in size,” said Andrew Underwood, Dell EMC’s ANZ high performance computing lead, who collaborated with Swinburne on the supercomputer design.

The NSF-funded LIGO project first successfully detected gravitational waves in 2015. Those waves were caused by the collision of two modest size black holes spiraling into one another (see HPCwire article, Gravitational Waves Detected! Historic LIGO Success Strikes Chord with Larry Smarr). LIGO has since detected two more events opening up a whole new way to examine the universe.

According to today’s announcement, up to 35% of the supercomputer’s time will be spent on OzGrav research related to gravitational waves. The supercomputer will also continue to incorporate the GPU Supercomputer for Theoretical Astrophysics Research (gSTAR), operating as a national facility for the astronomy community funded under the federal National Collaborative Research Infrastructure Scheme (NCRIS) in cooperation with Astronomy Australia Limited (AAL). In addition, the supercomputer will underpin the research goals of Swinburne staff and students across multiple disciplines, including molecular dynamics, nanophotonics, advanced chemistry and atomic optics.

OzStar replaces the “green” machines that have served Swinburne for the last decade and seeks to further reduce Swinburne’s carbon footprint by minimizing CO2 emissions by carefully considering heating, cooling and a very high performance per watt ration of power consumption.

OzGrav is funded by the Australian Government through the Australian Research Council Centres of Excellence funding scheme and is a partnership between Swinburne University (host of OzGrav headquarters), the Australian National University, Monash University, University of Adelaide, University of Melbourne, and University of Western Australia, along with other collaborating organisations in Australia and overseas.