TOP500 Reanalysis Shows ‘Nothing Wrong with Moore’s Law’

By Tiffany Trader

November 20, 2015

In Tuesday night’s TOP500 session, list co-creator and co-author Erich Strohmaier brought perspective to what could at first appear to be a land grab of unprecedented scale by China, when he shared that many of these new entrants were mid-lifecycle systems that were just now being benchmarked. But what is likely to be even more revealing is his reanalysis of what the TOP500 says about the apparent health of Moore’s law. Could Intel be right about this after all? And that’s not the only common wisdom that got trounced. Accelerator growth also came under scrutiny. Let’s dive in.

Joined onstage by his co-authors Horst Simon, Jack Dongarra and Martin Meuer, as well as HLRS research scientist Vladimir Marjanovic who would also present, Dr. Strohmaier, head of the Future Technologies Group at Berkeley Lab, began with a review of the top ten, which taken as a set comprise the most mature crop of elite iron in the list’s history. There were two new entrants to that camp, both Crays: the first-part of the Trinity install for Los Alamos and Sandia national laboratories with 8.1 petaflops LINPACK; and the Hazel-Hen system, installed at the HLRS in Germany, the most powerful PRACE machine with 5.6 petaflops LINPACK at number eight.

The biggest change on this November’s list was the number of systems from China — 109 installed systems up from 37 in July — cementing’s China’s number two in system share behind the United States, which only managed a list market share of 40 percent, down from a typical 50-60 percent footprint.

TOP500 SC15 Performance of Countries

But what Strohmaier said he likes to look at more than pure system share is aggregate installed performance of systems, which provides a ranking of peak systems by size, filtering out the effect you might have from a lot of small systems.

“If you look instead at the development of installed performance over time, you see the last ten years that China had had a tremendous increase in terms of installed performance,” Strohmaier remarked. “It is just ahead of Japan now — clearly the second most important geographic region in terms of installed capability, but it’s not nearly as close to US [as when looking at number of systems].”

Going one step further, the list author clarified that the systems installed in China are actually on the small size, excepting their flagship Tianhe-2, the 33.86 petaflops supercomputer, developed by China’s National University of Defense Technology, which has been sitting in the number one spot for six list iterations.

“China has had a tremendous run in the last decade,” Strohmaier observed, “and it’s continuing, but it’s not as dramatic as a simple system count would suggest.”

By number of systems, HP clearly is dominant in terms of market share. Cray is number two, and third is Sugon, the surprise company on the list. Sugon has 49 systems on the latest TOP500 system, a 9.8 percent system share. As the TOP500 list co-author discussed during the TOP500 BoF, Sugon’s story from a performance perspective looks a little different. The company captured over 21 petaflops for a 5 percent market share, which positions them in seventh place, below Cray (25 percent), IBM (15 percent), HP (13 percent), NUDT (9 percent), SGI (7 percent), and Fujitsu (also rounded to 5 percent – but with a slightly higher 22 petaflops).

Strohmaier went on to make the point that Sugon is new to the TOP500 and had to learn how to run the LINPACK benchmark and submit to the list. The company increased its list share from 5 on the previous listing to 49 systems – since one fell off, that means 45 systems were added.

“Sugon really took the effort, and the energy and the work and ran the benchmark on all their installations, regardless of how well or badly they performed and gave us the number,” said Strohmaier, “They went to great length to figure out where they are in terms of supercomputing, in terms of what the systems can do and in terms of where they’ll be in the statistics.”

He gave due to the company and the individuals within it that made this possible. “Sugon is now number three, while before it had very little list presence,” he added.

The kicker here, however, is that these are not new systems, which really would be an extraordinary feat if they were. “Many of the additions are … two to three years old, which had never been measured or submitted until now,” Strohmaier clarified.

The list also reflects the shake-up from the IBM x86 offload to Lenovo, leaving a rather confusing four-way division represented by the following categories: IBM, Lenovo, Lenovo/IBM and IBM/Lenovo. These “artifacts” will disappear over time, but right now this arrangement that was worked out between the vendors and customers dilutes the original IBM and Lenovo categories.

Lenovo is of course a Chinese company with a mix of systems that they built and sold as well as previous IBM systems that they now hold title to. Then there’s Inspur with 15 systems, another Chinese vendor. In all, there are three Chinese companies which are now prominent in the TOP500 and that produced an influx of Chinese systems, said Strohmaier.

He went on to examine a vendor’s total FLOPS as a percentage of list share, which shows that “HP traditionally installs small systems, Cray installs large systems, and then there is Sugon, which is an exception, because they have smaller systems, thus their share of performance is much smaller.” IBM, which is closest in system share with 45 systems (mostly leftover BlueGenes), has a large market share in terms of performance because they kept custody of the large Blue Gene systems. Inspur and Lenovo both have below average list share, while Fujitsu and NUDT have much larger shares which are of course reflecting their flagship systems, K computer and Tianhe-2 respectively.

Switching back to looking at the list in general, Strohmaier addressed the low turnover of the last couple of years. Before 2008 the average system age was 1.27 years, now it’s a tick below three years, marginally better than the June list. The TOP500 author attributed this to the bolstering influence from Sugon and from the IBM-Lenovo offload. “Customers keep their systems longer than they used to; this has not changed other than that small upturn [which can be explained].”

Moore’s law is fine!

The classic slide from each list iteration is the one that shows how performance grows over time with the performance of the first, the last and the sum of the TOP500, which Strohmaier thinks of as “500 times the average.”

TOP500 SC15 Performance Development

There has been impressive growth and for many years it was very accurate for predicting future growth, but in the last few years, the inflection points have appeared, 2008 and another in 2014, where the trajectory reduces.

This raises two important questions, says Strohmaier: why is there an inflection point and why is the inflection point in two timestamps?

“The nice thing is that the old growth rate before the inflection points were the same on both lines and the new growth rates are again the same on both lines. So the one effect is clearly technology, the other, in my opinion, is financial,” noted Strohmaier.

TOP500 SC15 Projected Performance Development

In the slide that shows the projections to the end of the decade before and after the inflection point, it can be seen that a seemingly small variance results in a significant 10X differential by the end of the decade.

“So instead of having an exascale computer by 2019 as we may have predicted ten years ago; we now think it’s going to be more like the middle of the next decade,” Strohmaier stated.

“Looking into what actually happened, we have to be more careful in how we construct the basis for our statistics,” he continued. “The TOP500 is an inventory based list, new and old technology are all mixed up. If you really want to see the changes in technology on the list, you have to apply filters filter out all the new systems with new technology coming into the list and analyze that subset.”

Strohmaier filtered out all the new systems and further filtered out all systems which only use traditional superscalar processes, so Nvidia chips and no Intel Phis. The point of this exercise was to tease out the track of traditional processor technology.

This is the result:

TOP500 SC15 Tech Trends Scalar Processors - Moore's Law is fine

Strohmaier:

What you see is that the performance per core has taken a dramatic hit around 2005-2006, but it was compensated by our ability to put more and more cores on a single chip, which is the red curve, and if you multiple that out as in performance per socket per actual chip, you get to the blue curve, which is actually pretty much Moore’s law. So what you see is the sample there is no clear indication that there is anything wrong with Moore’s law.

So what caused the slow-down in the performance curve?

The other thing is over the decades we put more and more components into our very large systems. I tried to approximate that by looking at the number of chip sockets per scalar process we have on these large systems — that’s what you see on the red curve – while the average performance follows Moore’s law, the red line does not follow a clear exponential growth rate after about 2005-2006.

TOP500 SC15 Tech Trends Scalar Systems

At that point, we seem to run out of steam and all out of money in our ability to put more components in the very large systems and the very large systems are not growing overall in size anymore as they have before. That is my interpretation of the data – that is why we have an inflection that is why the overall performance growth in the TOP500 has been reduced from its previous levels.

Right now supercomputing grows with Moore’s law, just like when supercomputing began and it does not which we had seen before.

So it’s clearly a technological reason, but it’s not a reason on a chip, it’s actually a reason on the facility and system level that is most likely related to either power or money or both.

Accelerator stagnation

Strohmaier, who has been one of the more ardent defenders of the benefits of LINPACK as a unified benchmark, went on to explore accelerator trends, acknowledging that they are responsible for a considerable share of petaflops. “But if you look at what fraction of the overall list those accelerators contribute,” he went on, “and if you focus on the last two years, their share has actually stagnated if not fallen.”

“That means there is a hurdle that is linked to market penetration of those accelerators that have not been able to penetrate the markets beyond scientific computing. They have not gotten into the mainstream of HPC computing,” he added.

Power efficiency is another metric covered in the BoF. Looking at the top ten in terms of average power efficiency, the course is uneven, but it’s growing. Highest power efficiency is much better, however. These power winners tend to either use accelerators or be BlueGeneQ systems, which are engineered to power efficiency.

TOP500 SC15 Most Power Efficient Architectures

The chart above shows the standouts for highest power efficiency, with new machines highlighted in yellow. TSUBAME KFC, installed at the Tokyo Institute of Technology and upgraded to NVIDIA K80s from K20xs for the latest benchmarking, came in first. What surprised Strohmaier were the number two and three machines terms of power – Sugon and Inspur, respectively. And once again, every system on this list ranked for megaflops-per-watt has an accelerator on it. (More on the greenest systems will be forthcoming in a future piece.)

The final slide presented by Dr. Strohmaier plots the best application performance from the Gordon Bell prize that is awarded each year at SC with TOP500 to show correlation. Since these are different applications with potentially different systems, a close tracking between these two trends over time could be taken to suggest that the LINPACK is still a useful reflection of real world performance. This is something to dive deeper into another time, but for now, here is that slide:

TOP500 SC15_TOP500 vs Gordon Bell

 

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire