“While the need for supercomputing is great, we have, in my view, reached a tipping point,” said Mark Stickells, executive director of Australia’s Pawsey Supercomputing Centre, as he opened his keynote (“Energy Efficient Supercomputing in Australia: Strategies Towards Net Zero”) at Supercomputing Asia 2023 (SCA23) in Singapore.

Stickells immediately acknowledged the massive contributions of supercomputing: “The world could not have got through the global pandemic without supercomputers,” he said, adding that “HPC really is essential for climate modeling and responding to climate change, playing a critical role in our transition to an energy future that is sustainable.”

But then, that pesky crossroads. “We’ve reached an energy, ethical and enterprise-level tipping point,” Stickells said. “Our compute power requires considerable energy, thanks to the requirements of its incredibly powerful processors and the powerful cooling that we need to keep them operating.” Stickells cited numbers he had recently heard: that information technology was using 8% of the world’s electricity – a figure doubling every ten years, with much of it attributable to datacenters and (as with most exponential trajectories) only a short time before it becomes truly unmanageable.

And siting your supercomputers in Australia doesn’t make it much easier. Stickells, characterizing the country as “a land of droughts and flooding rains,” explained that the “vast nation” has a sprawling, but relatively low-density energy transmission system, with an order of magnitude more transmission cabling than Japan but five times less capacity.

Australia has two distinct energy markets: the National Energy Market (NEM) on the east coast and the Southwest Interconnected System (SWIS) for Western Australia, where Pawsey is located. While the NEM has ~27% renewable energy (wind, hydro, solar) and even a region entirely powered by renewable energy (which contains Pawsey’s fellow supercomputing center NCI), Western Australia has no such region, a much smaller grid and a somewhat lower proportion of renewable resources (~21%, mostly wind) – though Stickells expects change on that front. “Global energy markets are in transition,” he said, “and Australia is rapidly transitioning to a greater reliance on renewable energies.”

Pawsey’s estimated carbon footprint was over 7,000 tons of carbon dioxide equivalent last year, the result of using over 11,000MWh on that SWIS grid – and Stickells said that Pawsey’s community was starting to take notice. A recent study from Australia’s prominent astronomy community – one of Pawsey’s biggest stakeholders – had shown that the average Australian astronomer was responsible for 37 tons of carbon dioxide equivalent a year, with the bulk of that footprint attributable to the computing performed on their data.

“Our stakeholders, our employees and our community place ever more weight on ESG credentials,” Stickells said, adding that, while energy in Western Australia is cheap (for now), Pawsey still incurs an annual energy cost in the millions of dollars. Despite the “challenging network and grid,” Stickells argued, Pawsey needs to move further toward sustainability to meet the needs of the changing world and changing attitudes.

Pawsey does have green credentials to its name: its newest system, Setonix, ranks both 15th on the Top500 and 4th on the Green500 list of the world’s most energy-efficient supercomputers. While Stickells indicated that Green500 rankings are not the be-all and end-all of sustainability, Setonix’s GPU partition does represent an order of magnitude improvement in energy efficiency over its predecessor, Magnus, and actually requires less total energy than Magnus.

Stickells also highlighted a long-running geothermal project that uses water from an aquifer beneath Pawsey to cool its systems. That closed-loop system, which is powered by a 120kW solar array at the center, “has resulted in the saving of approximately 70 million liters of water since its inception,” and commercial datacenters in Perth are now looking to deploy similar cooling systems.

Pawsey has also been looking into future technologies, like hydrogen power (not currently feasible, per Stickells) and commercial-scale battery technologies (somewhat more feasible, but not yet at the scale and/or reliability that Pawsey would require). The center has been exploring the use of backup power systems and energy storage to shift to a hybrid model that could take advantage of lower grid prices – and avoid peak pricing – to reduce the center’s energy expenditures. In the interim, Stickells said, purchasing green power certificates was a viable option.

Perhaps the boldest strategy discussed in the talk, though, was a new approach to how computing is allocated at Pawsey. “Our focus has been on optimizing allocation,” Stickells said. “Future allocation models at Pawsey will reward efficient code. … We will look to allocate computing not on classical cycles, but … on energy units.” While the core-hour model will likely be retained for Setonix’s CPU partition, the GPU partition will be allocated based on relative energy consumption, motivating users to optimize the energy cost of their jobs. A similar strategy is underway at RIKEN, operator of the Fugaku supercomputer – a similarity that Stickells noted in his talk.

“Sustainability in supercomputing is a global challenge,” Stickells said. “With the strength of our global community, we have the capacity to respond.”