The last time we left the Department of Energy’s exascale computing program in July, things were looking very positive. Both the U.S. House and Senate had passed solid Fiscal Year 2018 (FY-18) appropriations for the exascale activities for both the National Nuclear Security Administration (NNSA) and the Office of Science (SC). However, it also looked like there would be potential major challenges with other parts of the DOE’s budget. These included significant differences with some programs, such as ARPA-E that was over $300 million apart between the House and Senate appropriation bills.

After its August recess, Congress was expected to have some major budget fights in September. This not only included reconciling the difference between the versions of House and Senate appropriations, but also the question of raising the U.S. debt ceiling. Then on September 6^th, those potential fights came to a sudden end when President Trump reached an agreement with the House and Senate Democratic leaders for a FY-18 Continuing Budget Resolution (CR) and to raise the government debt ceiling until early December 2017. That effectively maintains the Exascale FY-17 status quo in the short term. From a funding perspective, things for the exascale program continue to look very good.

On September 26^th and 27^th, some more clarity about the technical aspects of the program was provided during the public SC Advanced Scientific Computing Advisory Committee (ASCAC) meeting. The ASCAC is a regular meeting of the officially endorsed Federal Advisory Committee Act (FACA) group that provides advice to Advanced Scientific Computing Research (ASCR) program. During the meeting, Barb Helland, the associate director for the ASCR office, provided a presentation about the status of the their activities (link). The presentation included some very interesting information about the status of the SC Exascale program.

On slide number 7, she told the ASCAC that there had been a shift in the delivery of the Argonne National Laboratory (ANL) Aurora system. That computer had originally been scheduled to be delivered in 2018 with a performance of 180 petaflops. However, the revised plan for the system is for a 1,000 petaflops (or 1 exaflops) computer to be delivered in 2021. The machine would use “novel technology choices” and would focus on the three pillars of simulation, big data, and machine learning. This shift in the program seems to explain the House’s concern “that the deployment plan for an exascale machine has undergone major changes without an appropriately defined cost and performance baseline.”

Ms. Helland reported that the shift in the machine architecture had been subject to a review in September of 2017 and had received very favorable comments. These included, “The system as presented is exciting with many novel technology choices that can change the way computing is done. The committee supports the bold strategy and innovation, which is required to meet the targets of exascale computing. The committee sees a credible path to success.” Another comment was, “The hardware choices/design within the node is extremely well thought through. Early projections suggest that the system will support a broad workload.” She also reported that a Rebaseline Independent Project Review was scheduled for November 7^th to 9^th.

Another important piece of news was about the status of the installation of the Oak Ridge National Laboratory (ORNL) Leadership Computing Facility’s Summit computer. This is expected to be a 150 petaflops computer based on the IBM Power9 processors with Nvidia Volta graphic processing units (GPUs). During the meeting, it was reported that the system cabinets had been installed along with the interconnection switches. The computer node boards are expected to arrive sometime towards the end of October and that acceptance testing would start soon after that. It was also reported, that installation of the NNSA’s Lawrence Livermore National Laboratory (LLNL) Sierra computer (similar to Summit) was also underway. One interesting feature of the ORNL computers is that they are installed on a concrete slab with all of the supporting wiring and cooling coming from overhead.

During her presentation, Barb Helland made the point that ASCR would soon be releasing information about the procurement of additional exascale systems to be delivered in the 2022 timeframe. No details were provided, but she explained that these systems would be follow-on systems to the ones delivered as part of the CORAL procurement.

Finally, there were two other interesting exascale revelations during the ASCAC meeting. One was the clarification of the differences between the acronyms of ECI and ECP that appeared in the President’s budget request. Slide number 5 provides the definitions of the terms and states that the ECI (Exascale Computing Initiative) is the partnership between the NNSA and SC. On the other hand, ECP (Exascale Computing Project) is a subprogram within ASCR (SC-ECP) and includes only support for research and development activities in applications, and in partnership with NNSA, investments in software and hardware technology and co- design required for the design of capable exascale computers. The other revelation is that Paul Messina of ANL, the founding director of ECP, is stepping down and will be replaced as of October 1^st by Doug Kothe of ORNL. The ASCAC thanked Paul for his service to the country in establishing the foundations for the ECP.

All in all, the most recent ASCAC meeting provided some valuable insights into the U.S. exascale program. Certainly not all of the questions have been answered, but the information provided at the meeting helps to clarify the Department of Energy cutting edge computing program. Perhaps the best news is that the program is still receiving strong Presidential and Congressional support. However, the new December 2017 budget deadline continues to lurk in the background. Once again, more to come.

About the Author

Alex Larzelere is a senior fellow at the U.S. Council on Competitiveness, the president of Larzelere & Associates Consulting and HPCwire’s policy editor. He is currently a technologist, speaker and author on a number of disruptive technologies that include: advanced modeling and simulation; high performance computing; artificial intelligence; the Internet of Things; and additive manufacturing. Alex’s career has included time in federal service (working closely with DOE national labs), private industry, and as founder of a small business. Throughout that time, he led programs that implemented the use of cutting edge advanced computing technologies to enable high resolution, multi-physics simulations of complex physical systems. Alex is the author of “Delivering Insight: The History of the Accelerated Strategic Computing Initiative (ASCI).”