As reported in the South China Morning Post HPC pioneer Jack Dongarra mentioned the lack of benchmarks from recent HPC systems built by China.

“It’s a well known situation that China has these computers, and they have been operating for a while. They have not run the benchmarks, but [the community has] a general idea of their architectures and capabilities based on research papers published to describe the science coming out of those machines,” he said.

Dongarra was recently (August-2023) in China to attend a workshop on Exascale computing software and algorithms in Beijing. His comments were part of an an interview with the South China Morning Post after his return to the US.

Official specifications and benchmarks for new systems do not seem to be reported or at least hard to find. All performance numbers for new systems at this point are estimates based on prior systems. As reported by Tom’s Hardware the new Sunway Oceanlite system built by the National Supercomputing Center in Wuxi is estimated to use 19.2 million cores across 49,230 compute nodes. This information was found searching a report on the SC 23 Gordon Bell Finalist.

All Top500 machines must describe their architecture and component processors, GPUs, memory, and interconnects. This information is important from both a historical and performance perspective. The current Top500 system (as measured by the HPL benchmark) is Frontier at Oak Ridge with 1.194 exaFLOPS.

The last number one system China submitted to the Top500 was the 93.01 petaFLOPS Sunway Taihulight computer, which maintained the No.1 Top500 position from June 2013 until Nov 2017. As of September 2023, the 40,960 Sunway SW26010 processor-based system is ranked at number 7 on the current Top500 list.

Both the recent Sunway Oceanlite (equipped with 49,230 SW26010-Pro processors) and Tianhe-3 (equipped with Arm Phytium’s FT-2000 CPUs and Matrix 2000 DSP accelerators) systems are presumed to be Exascale capable systems. However, they have not been submitted to the Top500 list.

There could be a variety of reasons. Dongara suggests that “Maybe having the No.1 computer would make news and put China under the spotlight. It can cause the US to take actions against China that would further restrict technologies from flowing into China.” Dongarra continued, “However, China is still the country which produces the most supercomputers. With domestic and Western-designed chips, supercomputers assembled in China are sold all over the world, including the US.”

The US embargoes certainly could be causing a slowdown with these systems, but much of the processors and GPUs used by China on their leading edge systems are designed domestically. There is still the need for foreign fabricators like TSMC or Samsung Electronics.

Finally, running and reporting performance on the Top500 list is often considered more public relations than a general performance measure. From a historical context, the Top500 is very valuable. In terms of systems benchmarking, it can be considered one data point in a large ecosystem of modern HPC performance metrics. It also takes a non-trial amount of time and effort to get a good benchmark run . As Dongarra mentions they have not run (reported) benchmarks for their new systems, but information is available trough research papers and application performance–which is what really counts in any case.

It should also be mentioned that many of the new massive systems aimed at the exploding GenAI market do not run (or at least publish) a Top500 HPC benchmark. And there is no requirement that any supercomputer class systems run the HPC benchmark and report their performance to the Top500 list. There are many capable machines used by private companies and 3-letter government entities that never publish any type of benchmark.

Certainly the threat of further embargoes, looms over China’s HPC systems and keeping quiet about progress may be way to stay out of the limelight. Supercomputing has always been viewed as a strategic capability by many counties and the need to have home grown components ensures that global disruption will not diminish your ability to compute at the highest levels.

In some respects, the Top500 makes it “too easy” to compare systems. There are more balanced benchmarks (HPCG) consisting of several applications, including the HPL benchmark used to rank the Top500. Perhaps China’s reluctance to run or share these benchmarks will invite users to listen more carefully to system design and application performance results.