The Texas Advanced Computing Center (TACC) has won the next NSF-funded big supercomputer beating out rivals including the National Center for Supercomputing Applications (NCSA)/University of Illinois at Urbana-Champaign, the San Diego Supercomputer Center, and the Pittsburgh Supercomputing Center among others. The NSF award – Towards a Leadership-Class Computing Facility – Phase 1 – is worth roughly $60 million for the system and another $60 million for five years of operations. A Phase 2 effort for building a true Leadership Class system is called out in the solicitation but with few details and no specific funding. The list of submitters is not made public.

Details of the planned TACC system or winning proposal were not immediately available. TACC is delaying public announcements until the “final contracts are signed” according to a spokesperson. However, a publicly-available memorandum from the National Science Board (NSF’s governing body) reported that NSB passed a resolution last Tuesday “authorizing the Director to make an award to the Texas Advanced Computing Center at the University of Texas at Austin for the acquisition of the system described in the [TACC] proposal Computation for the Endless Frontier.”

HPCwire will report the details of the new machine when they are available. In describing the project’s requirements the NSF solicitation used the Blue Waters (NCSA) supercomputer as a benchmark saying the Phase 1 system should have “least two to three-fold time-to-solution performance improvement over the UIUC’s Blue Waters system for a broad range of computational and data-intensive workflows that require the highest capabilities in terms of scale, throughput, and data analytics.”

It will be fascinating to see who the prime vendors are and what architecture is planned. Blue Waters, headquartered at NCSA, is a 13 petaflops (peak) system built by Cray and powered by AMD Opteron processors and Nvidia K20 GPUs. By comparison to the just-announced award, Blue Waters cost about $190 million capital and roughly $150 million for operations for five years.

Reached for comment on the NSF award, Bill Gropp, director of NCSA, congratulated TACC and expressed disappointment for NCSA. He declined to offer details of the NCSA proposal as there has been speculation regarding potential NSF support for an additional system; NCSA would be interested in competing for it if indeed such an opportunity arises and is keeping its cards close to its vest for the moment.

Said Gropp, “The call for the Blue Waters replacement emphasized performance across a wide range of applications. That’s a very challenging target. We worked hard to propose a system that would deliver sustained performance in the requested quantity for a variety of applications. Going forward it’s going to get harder and harder to do that because the architectures become more specialized in order to do performance targets.”

The new award is sure to spark debate over NSF’s selection process and its overall strategy for growing the U.S. HPC infrastructure serving academic science. While the Department of Energy (DoE) has a clear exascale computing program to serve DoE missions, NSF does not.

Gropp says, “NSF essentially doesn’t like using the word exascale. To some extent I am ok with that because it is an arbitrary performance number and we should be focused on enabling science.” Indeed DoE has likewise de-emphasized exaflops capability as the goal and instead has set 50x performance above existing leadership class systems as the benchmark.

Many in the HPC community say NSF plans for building extreme scale computing infrastructure in support of science are inadequate. While acknowledging the many pressures NSF faces, Gropp is nonetheless firmly in the camp that believes NSF’s plans should be more aggressive.

“The current NSF situation is disappointing. The system that TACC won, if you look at the title of this [NSF solicitation], it is ‘Towards Leadership Computing,’ which is really truth in advertising, because that’s not going to be a leadership system unless TACC got a really great deal from somebody. It’s going to be not in the same ballpark as leadership systems either at DoE or elsewhere in the world,” said Gropp, who was co-chairman of the National Academies Report, Future Directions for NSF Advanced Computing Infrastructure to Support U.S. Science and Engineering in 2017-2020, which supported more aggressive expansion of high end HPC resources.

“The challenge that we see going forward is the level of funding and level of resources provided are declining both pretty much in absolute terms and certainly relative to our international competitors. If you look at the plans that Japan has for what I would consider their Track 2 systems, I think that right now there are 11 systems on that roadmap. That is a real roadmap with specific kinds of systems, performance targets, centers of operations and timelines. Now it’s a roadmap; it is not a guarantee, but it shows thinking that allows science teams to plan on what sort of systems are going to be available and what they need to do to adapt to use those systems,” said Gropp.

Gropp noted a roadmap slide presented by NSF at SC16 included second system. At SC17, a similar slide lacked the second system. (see slides below)

“If you read the RFP for this solicitation, it is a two-phase solicitation, but only the first phase is funded. So part of what you had to respond to in the proposal was how you were going to plan for phase two. So another question is how much money they are going to set aside for phase two. Are they prepared to set aside $200- or $300 million dollars for capital and something on the same order for operations, which is what a leadership system costs these days.”

No doubt there will be heightened discussion of NSF’s strategy following the TACC award as well as excitement around learning the details of TACC’s planned new machine.

Link to NSB memorandum: https://www.nsf.gov/nsb/meetings/2018/0717/major-actions.pdf

Link to full NSF solicitation: https://www.nsf.gov/pubs/2017/nsf17558/nsf17558.htm

Link to National Academies Report: https://www.nap.edu/catalog/21886/future-directions-for-nsf-advanced-computing-infrastructure-to-support-us-science-and-engineering-in-2017-2020