June 22, 2020 — Australia’s newest and fastest supercomputer is a game-changer for the country’s leading scientific institutions. Gadi, meaning ‘to search for’ in the language of the Ngunnawal people, will strengthen Australia’s capacity to perform world-leading scientific research, and will directly contribute to the safety and wellbeing of the nation.

The supercomputer, based at the National Computational Infrastructure (NCI Australia), has placed at 25 on the TOP500 list, a global ranking of supercomputers released overnight at the ISC High Performance Digital Conference. Gadi is the most powerful supercomputer in the Southern Hemisphere.

ANU Vice-Chancellor Professor Brian Schmidt said the new supercomputer will help Australian researchers tackle and solve some the most profound and pressing issues of the coming decade.

“Gadi will give researchers the tools to unlock the mysteries of the universe, predict and manage natural disasters, advance cancer research and design new materials for future technologies”, said Professor Schmidt.

Professor Sean Smith, Director of NCI, sees Gadi as a significant and important evolution of high-performance computing for Australia.

“The computational performance available on Gadi is unprecedented in Australia, enabling rapid response to national emergencies such as COVID-19 and bushfires that we could not have approached before.” explains Professor Smith.

“We have over five thousand users that rely on NCI’s unparalleled high-performance compute and data services to produce world-leading scientific research. With Gadi now fully operational, these users will be able to perform their work faster, and with greater precision. Gadi is, quite simply, a critical foundational infrastructure for Australia and Australian research.”

The Bureau of Meteorology, CSIRO, Geoscience Australia and The Australian National University are all major collaborators of NCI and are set to utilise Gadi to its fullest potential in the coming years.

“National high-performance computing facilities are critical for Australia’s future,” says Dr Dave Williams, Executive Director of CSIRO – Digital, National Facilities and Collections.

“At CSIRO, we solve the greatest challenges using innovative science and technology, and Gadi will enable our work in Earth sciences including climate, land, water and fire modelling. The data that Gadi will process and analyse is collected here on Earth as well as by an increasing number of satellites orbiting our planet.”

These computational models will contribute to the CSIRO Decadal Climate Forecasting Project (DCFP), which is already producing near-term climate forecasts with ever-increasing accuracy. Gadi will assist this project by facilitating a move to even higher model resolutions of the atmosphere and ocean, with fast data processing and increased model accuracy.

The Bureau of Meteorology is leveraging the new computational power of Gadi to produce sub-kilometre coupled fire-atmospheric models. Known as ACCESS-Fire, this model represents the complex interactions between the heat of the fire, the release of moisture, and atmospheric circulations. Such models can predict the path of a fire as well as the spread of smoke. These predictions have the potential to reduce the risk to communities and firefighters now and into the future.

“The next generation of seamless prediction systems will make weather forecasts more locally relevant, more accurate and more useful for longer periods of time” says Dr Gilbert Brunet, Chief Scientist and Group Executive, Science and Innovation at the Bureau of Meteorology.

“The Gadi supercomputer will be used to support transitioning the experimental models to routine operation. Improvements such as those from the fire model and specific event case studies are used to better plan fire suppression operations, reduce risk to firefighters and assist the community in affected areas.”

Similarly, Geoscience Australia will use Gadi to run sophisticated models generating large swathes of data to estimate the risks associated with other potential natural disasters.

“Geoscience Australia, aided by the National Computational Infrastructure, is improving the safety and resilience of communities around Australia” explains Dr James Johnson, CEO of Geoscience Australia.

“Without the NCI, we would not be able to compute the huge data needed to assess the risks associated with hazards such as earthquakes, tsunamis and tropical cyclones. I’m looking forward to seeing how this new supercomputer will enable further work at Geoscience Australia to ensure a better informed nation.”

The Probabilistic Tsunami Hazard Assessment provides a framework for estimating the tsunami hazard anywhere in Australia due to distant earthquakes. Likewise, the National Seismic Hazard Assessment predicts the likely ground motions that may occur during an earthquake. While earthquakes are rare in Australia, they have the potential to be devastating should they occur near a large population centre.

Geoscience Australia also develops a national assessment for the hazard of tropical cyclones. The Tropical Cyclone Hazard Assessment is the basis for collaborative projects with Queensland and Western Australian emergency services where the potential impacts of future possible tropical cyclone scenarios is being investigated so that communities can be better prepared for these kinds of events.

These and many other critical environmental assessments will be improved with the technology now available through Gadi.  With an increasing availability of computational power, as well as significant data storage and management services, researchers at Geoscience Australia and other institutions can perform more precise simulations, producing more accurate results.

In addition to these major collaborators, NCI also has agreements with dozens of Australian universities, ARC Centres of Excellence, and medical research groups such as the Garvan Institute of Medical Research and Victor Chang Cardiac Research Institute, representing thousands of Australian researchers across the spectrum of scientific disciplines.

Technical specs

Gadi contains a total of 145,152 CPU cores, 567 Terabytes of memory and 640 GPUs.

• 3,024 nodes each containing two 24-core Intel Xeon Scalable ‘Cascade Lake’ processors and 192 Gigabytes of memory.
  • including 50 nodes each offering 1.5 Terabytes of Intel Optane DC Persistent memory.
• 160 nodes each containing four Nvidia V100 GPUs and two 24-core Intel Xeon Scalable ‘Cascade Lake’ processors.
• Linking the storage systems and Gadi is Mellanox Technologies’ latest generation HDR InfiniBand technology in a Dragonfly+ topology, capable of transferring data at up to 200 Gb/s.
• The storage sub-systems are NetApp enterprise class storage arrays, linked together in a DDN Lustre parallel file system.
• Altair’s PBSPro software optimises job scheduling and workload management.
• Gadi uses the latest version of the CentOS 8 operating system.

About NCI

NCI Australia is the nation’s most highly-integrated, high-performance research computing and data science environment. Based at The Australian National University, it brings together the Australian Government, universities, national science agencies and industry. Major Collaborators include ANU, CSIRO, the Bureau of Meteorology and Geoscience Australia.

As well as the unique $70 million investment to upgrade infrastructure which enabled NCI to build Gadi, NCI currently receives approximately $6 million per annum from NCRIS to support its operating costs. In total, NCI has been allocated more than $210 million through NCRIS since 2006.

---

Source: NCI