In partnership with Sandia National Laboratories, the National Renewable Energy Laboratory (NREL) recently completed a nearly year-long energy performance evaluation of Aquila’s Aquarius fixed cold plate cooling system, which we first covered in 2016.
Aquila – a New Mexico-based technology firm – makes its Aquarius warm-water server racks using patented fixed cold plate liquid cooling technology, designed by Clustered Systems. The technology employs cold plates to cool every component over two watts; the heat from the lower-power components is collected by the cold plates through radiation, convection and conduction. The system uses no fans, eliminating the interference posed by fan vibrations and reducing the number of components that can fail.
In 2018, Sandia and NREL announced their intention to study the energy and cost savings of this cooling system in a real-world environment by installing a test system (“Yacumama”) at the National Renewable Energy Laboratory’s (NREL’s) Energy Systems Integration Facility (ESIF).
Now, nearly a year later, the partners have revealed the results of the test.
Yacumama operated independently from all other HPC systems in NREL’s Energy Systems Integration Facility (ESIF). It was equipped with 36 nodes, each with an Intel S2600KP motherboard configured with dual Xeon CPUs, 128GB of RAM, a 128GB SSD, and an Omni-Path adapter. In total, it was capable of delivering over 40 Linpack teraflops on less than 15 kW of power.
In building its ESIF datacenter, NREL set out to focus on three elements of sustainability:
- Efficiently cooling the equipment using direct, component-level liquid cooling with high power usage effectiveness
- Capturing and reusing any waste heat, and
- Minimizing the water use.
In their final report, “Energy Performance Evaluation of Aquila’s Aquarius Fixed Cold Plate Cooling System at NREL’s High Performance Computing Center,” the authors first highlight the seamless installation of the Yacumama cluster. They go on to highlight the results of the Linpack tests, run at 100 percent duty cycle for 48 hours. “[The] key takeaway,” they write, “is that this fixed cold plate design provides a very high percentage of heat capture direct to water—up to 98.3% when evaluating compute nodes only[.]”
The other results are similarly promising. “This cluster has been in operation for nearly 10 months, requiring zero maintenance,” they write. No water leaks occurred during that ten-month period, and the report calls the cold plate design “observably robust in nature.”
The Yacumama system will soon return to Sandia’s warm-water-cooled HPC datacenter.
The full energy performance evaluation of the Yacumama system can be found on NREL’s website at this link.