It’s been almost six months since the National Energy Research Scientific Computing Center (NERSC) formally announced Cori, the new supercomputer set to be installed at the lab in the mid-2016 timeframe. Originally known as NERSC-8, the new machine will sport over 9,300 nodes, equipped with next-generation Knights Landing architecture housed within a Cray XC environment. Each of these chips is capable of delivering more than three teraflops of double precision performance, which altogether is enabling a ten-fold application performance over Hopper (aka NERSC-6), for an estimated peak system performance in in the neighborhood of 30 petaflops.
The DOE’s Advanced Scientific Computing Research (ASCR), which funds and manages the NERSC facility, recently published an article exploring how Cori will support research into exascale concepts.
Performance enhancement is only one part of the story. Equally important to this goal is energy efficiency. This means doing more computing with less energy – and it’s one of the essential design elements in the development of Cori.
“Our science users are telling us they need to get to the level of hundreds of petaflops (quadrillion calculations per second) within this decade, and the only way we can get to that level of computing is to make this transition to energy-efficient architectures,” says NERSC Director Sudip Dosanjh.
“With Cori we will begin transitioning the broad range of (DOE) Office of Science codes that run at NERSC to energy-efficient many-core computer architectures.”
The Knights Landing coprocessor, with its emphasis on optimizing the FLOPS-per-watt equation, is at the heart of this strategy.
Knights Landing is expected to provide between 14 and 16 gigaflops per watt of energy. Compare this to the reining Green500 champ, Tsubame-KFC. So far this system from the Tokyo Institute of Technology is the only one to break the 4 gigaflops-per-watt mark.
Going forward, revving clock speed will no longer be an effective way to wring out more performance because all those clock ticks draw energy and release heat meaning additional energy is needed to power and cool the machine, adding up to as much as tens of millions of dollars each year.
Intel’s approach for Knights Landing is to put lots of energy-efficient cores (up to 72) on the same chip. Dosanjh says this translates to “a lot more computing using just a little more power.”
Memory was another design focus. Dosanjh explains that NERSC applications are hindered more by memory bandwidth and data movement than by floating point capabilities.
Cori will provide over 400 gigabits per second of I/O bandwidth and 28 petabytes of disk space. The upcoming Knights Landing chip employs three-dimensional on-package memory developed by Intel and Micron, that is projected to deliver five times more bandwidth than DDR4 memory, helping to alleviate I/O bottlenecks. The system will also include a layer of Non-Volatile Random-Access Memory (NVRAM), called a Burst Buffer, that will move data more quickly between processors and disk, a boon for data-intensive science.
All told Cori will provide NERSC’s more than 5,000 users with approximately 30 petaflops of peak performance to support an estimated 650 applications geared to a wide-range of science and engineering disciplines. The NERSC Exascale Science Applications Program (NESAP) will be instrumental in getting these codes to take advantage of the manycore architecture and the additional memory layers, says Dosanjh.
When Cori arrives in 2016, it will have be housed inside Lawrence Berkeley National Laboratory’s Computational Research and Theory building. This energy-efficient center, which relies on San Francisco Bay air for free cooling, is currently being constructed on the main Berkeley Lab campus.