At the level of leadership supercomputing, system planning and procurement is generally a multi-year process. The excitement that surrounded the initial announcement can wane, but it always comes back when it’s time to start putting the system together. For the Trinity supercomputer, which has been on the Department of Energy’s roadmap since 2013, the proof of this progress is in the pictures.
At Los Alamos’s Strategic Computing Center, work is underway to accomplish the facility overhaul that is required to support the 112 next-generation Cray XC40 racks that make up Trinity, the first of the NNSA’s Advanced Simulation and Computing program’s advanced technology systems. “Once installed, Trinity will be the first platform large and fast enough to begin to accommodate finely resolved 3D calculations for full-scale, end-to-end weapons calculations,” observes the laboratory.
The supercomputing room itself takes up 43,500 square feet, almost an acre, and nearly one-quarter of that space (about 10,000 square feet) is dedicated to the warm water cooling infrastructure needed to keep the machine’s Haswell and Knights Landing-based nodes humming.
The enormous scale of this venture comes to life in an image released by Los Alamos illustrating elements of the warm water gear as they are pieced together below the the raised floor.
“In order to accommodate Trinity, the SCC first had to undergo a series of major mechanical and electrical infrastructure upgrades,” states the lab. “Because energy conservation is a priority at Los Alamos, these upgrades included a shift to warm water cooling technology (which will result in a major energy savings), as well as a decrease in the use of city/well water for cooling towers.”
By using water sourced from LANL’s Sanitary Effluent Reclamation Facility (SERF), instead of city water, the new cooling plant is expected to save millions of gallons of well water per year.
Program staff explored facilities around the country to develop a plan that would maximize energy conservation. Such site visits informed the LANL design strategy; for example, a simple valve added to a cooling tower will save money by enabling cooling without recirculation during months when the ambient outside air is at its coolest.
Trinity will be deployed by the Alliance for Computing at Extreme Scale (ACES), a collaboration between Los Alamos National Laboratory and Sandia National Laboratory. The single system will be comprised of a Haswell partition, scheduled to be up and running by the end of 2015 and a Knights Landing partition to be completed in 2016. In addition to the latest Xeon Phi processors, new architectural features include Burst Buffer storage nodes and advanced power management system software enhancements.
As Trinity’s timeline progresses, we keep learning more about this exascale stepping stone. The following details were shared by Trinity Project Directory Manuel Vigil at the 2014 HPC User Forum in Seattle.