Exascale machines (of at least a 1 exaflops peak) are anticipated to arrive by around 2020, a few years behind original predictions; and given extreme-scale performance challenges are not getting any easier, it makes sense that researchers are already looking ahead to the next big 1,000x performance goal post: zettascale computing. In a recently published paper, a team from the National University of Defense Technology in China, responsible for the Tianhe series of supercomputers, suggests that it will be possible to build a zettascale machine by 2035. The paper outlines six major challenges with respect to hardware and software, concluding with recommendations to support zettascale computing.

The perspective piece gives an interesting peek into China’s post-exascale intentions (the project is supported by the National Key Technology R&D Program of China), but the challenges presented will be familiar to anyone engaged in pushing the boundary on leadership supercomputing.

The article “Moving from exascale to zettascale computing: challenges and techniques,” published in Frontiers of Information Technology & Electronic Engineering, (as part of a special issue organized by the Chinese Academy of Engineering on post-exascale computing) works as high-level survey of focus areas for breaching the next big performance horizon. And when might that be? The research team, even while pointing to slowdowns in performance gains, has set an ambitious goal: 2035. For the purposes of having a consistent metric, they’ve defined zettascale as a system capable of 10^21 double-precision 64-bit floating-point operations per second peak performance.

The potential impact of mixed-precision arithmetic and AI-type algorithms on performance metrics (already in motion) was not a focus topic, but the authors did note, “With the continuous expansion of application types and scales, we expect that the conventional scientific computing and the new intelligent computing will further enrich the application layer. Techniques (such as machine learning) will be used to auto-tune various workloads during runtime (Zhang et al., 2018).”

The likely impact on architectures was also noted:

“[S]ince conventional HPC applications and emerging intelligent computing applications (such as deep learning) will both exist in the future, the processor design should take mixed precision arithmetic into consideration to support a large variety of application workloads.”

The paper is organized thusly:

1 Introduction
2 Future technical challenges in high performance computing
2.2 Challenges in power consumption
2.3 Challenges in interconnection
2.4 Challenges in the storage system
2.5 Challenges in reliability
2.6 Challenges in programming
3 Future high-performance computing technology evolution and revolution
3.1 Architecture
3.2 High-performance interconnecting technology
3.3 Emerging storage technology
3.4 New manufacturing process
3.5 Programming models and environments
4 Suggestions for zettascale computing

The 9-page paper is accessible and best read in full. This excerpt from the final section gives a sense of the directions under consideration:

“To realize these metrics, micro-architectures will evolve to consist of more diverse and heterogeneous components. Many forms of specialized accelerators (including new computing paradigms like quantum computing) are likely to co-exist to boost high performance computing in a joint effort. Enabled by new interconnect materials such as photonic crystals, fully optical-interconnecting systems may come into use, leading to more scalable, high-speed, and low-cost interconnection.

“The storage system will be more hierarchical to increase data access bandwidth and to reduce latency. The 2.5D/3D stack memory and the NVM technology will be more mature. With the development of material science, the memristor may be put into practice to close the gap between storage and computing, and the traditional DRAM may end life. To reduce power consumption, cooling will be achieved at multiple levels, from the cabinet/board level to the chip level.

“The programming model and software stack will also evolve to suit the new hardware models. Except for the MPI+X programming model, new programming models for new computing paradigms and new computing devices will be developed, with the balance of performance, portability, and productivity in mind. Conventional HPC applications and emerging intelligent computing applications will co-exist in the future, and both hardware and software layers need to adapt to this application workload evolution (Asch et al., 2018).”

Link to Journal article: https://link.springer.com/article/10.1631/FITEE.1800494

Link to paper: https://link.springer.com/content/pdf/10.1631%2FFITEE.1800494.pdf

Special issue on post-exascale supercomputing: https://link.springer.com/journal/11714/19/10/page/1