Now that this year’s Top500 has been released and analyzed, we wanted to take a step back to look at a few emerging trends. One of the elements of the last couple of lists that caught our eye is that despite the availability of new accelerator/co-processor choices, there is a noticeable leveling off in overall use following a sharp increase that began in 2010 with the full arrival of GPU computing.

This data is rather out of step with what the analysts are finding in terms of accelerator adoption. For instance, both IDC’s high performance computing group and Intersect360 see strong growth for the accelerator/co-processor market now and in years ahead. IDC found that the number of sites using these technologies jumped from 28.2% in 2011 to an incredible 76.9% in 2013. They also noted that NVIDIA GPUs and the Xeon Phi were “neck in neck” in the race for HPC customers, noting the “use of co-processors and accelerators is still wider than it is, meaning that these newer devices have entered many more sites, but are often still used for exploratory purposes rather than production computing.” IDC also highlighted that industrial users were less likely to buy large number of accelerators, but were more likely to use them in production. Intersect360 Research had a more modest estimate, finding that accelerators were being used on 21% of the installed base, although they agreed with IDC that evaluation was playing a large role in the adoption at this point.

So if this massive uptick in accelerator and co-processor adoption is finding its way into analyst research with such striking figures, how is it that only a tick over 12% of the Top500 list of supercomputers is making use of them? These are the most experimental environments and while, as you can see in the charts coming soon, there was a spike in 2010-2011 signalling the full arrival of GPUs in particular, there’s been no rapid increase. Just an even line.

Of all the machines on this year’s Top500, only 62 are using accelerator/co-processor technology, which is only a slight increase from the November ranking, which showed a total of 53 systems. Of thos, 44 are using NVIDIA GPUs (see the generation in the graphic below), 17 have implemented Xeon Phi as the co-processor/acceleration option, and two machines are leveraging the ATI Radeon cards. While there are not necessarily surprises in these vendor breakdowns, we wanted to highlight the flat line that extends across these accelerated machines. On the one hand, it would seem that given the options available with the addition of Intel’s Xeon Phi into the HPC market and the ever-richer ecosystem around CUDA and OpenCL, why aren’t more supercomputing sites choosing to push their machines with accelerators?

There are a few reasons for this tapering off, says Top 500 list curator, Erich Strohmaier, but make no mistake, none of them spell a dire future for GPUs or co-processors like the Xeon Phi.

The culprit for this even keel story for what’s remained one of the most exciting technology areas in HPC has nothing to do with interest, it’s a matter of procurement cycles aligning with product cycles. He said that some years ago, the list could see more dramatic swings with new technologies because procurements were secured with upgrade agreements so users could confidently grab a system instead of delaying procurement to wait for the latest and greatest part.

It is probably coming as no surprise then that we’re going to see a marked uptick in accelerator adoption in 2015 (or that’s the plan, according to what publicly stated roadmap details we have suggest) when Knight’s Landing comes into the market, bringing with it a slew of systems that are literally waiting on the right parts. As we noted during the NERSC-8 “Cori” system announcement, the users there were most interested in the on-package memory because of their workloads. It wouldn’t make sense to buy a system now and retrofit–and chances are, there will be other announcements around pending product launches for big systems (CORAL, etc.–just a guess).

But here’s the interesting thing. The Top500 list isn’t just made up of the academic/national lab supercomputers that are more capable of tying their procurements to the products they’re anticipating. What about industry HPC users who have different processes for securing their machines and, arguably, a more mission-focused (read as monetary) incentive to do what works now and buy what’s coming when it’s ready? It would be one thing to look at the analyst data on the sharp rise in accelerator/co-processor use and say that it doesn’t affect the Top500 because of the influence of commercial systems but guess what? Well over half the Top500 is made up of industrial machines.

This could mean a few things. First, we might be wrong in the assumption that industrial users are less likely to wait around and tie their procurement processes to product cycles. Perhaps everyone does now–feel free to comment on this. But even still, there’s no real increase in actual Linpack-ready implementation, so that might suggest that if they had already evaluated accelerators/coprocessors and didn’t find them of immediate value, they may have moved on from the experiment. So perhaps the “honeymoon” phase for these technologies is over, only for the passion to be reignited again with the arrival of Knight’s Landing and the upcoming technologies NVIDIA has on deck (and let’s not forget what will happen with FPGAs and OpenPower–didn’t meant to be exclusionary there).

Either way, there seem to be some conflict fact points about what’s really happening with the accelerator adoption curve. The other argument to all of this, of course, is that the Top500, even with its industry users, isn’t counting a huge number of systems that could run Linpack and rank highly if only they chose to do so. It might be that we have a range of data about adoption that is incomplete on all sides. Analyst figures vary widely between research groups, the Top500 has a leveled-off showing, and as one might imagine, if you ask the vendors how their accelerator/co-processor business is doing, it’s all sunshine and roses.

On the bright side, there are two closing figures from this ISC’s Top500 that seemed worth pointing out. While they’re not related to adoption, they do make the case for the value for these technologies on both a performance and efficiency front. The first graphic, courtesy of Erich Strohmaier, shows the performance share of accelerated systems matched with the top ten supercomputer rankings that shows quite clearly that these are the key piece for high performance. 9 out of 10 of the systems are outfitted with GPUs or Xeon Phi.

The second graphic shows the green performance of these GPU and Xeon Phi-enabled systems.

We would love to hear from you on this point. Is it a matter of waiting until 2015 for things to pick up again following the product cycle that so many seem to be holding out for? Or is it that the peak interest and experimentation has yielded results that are expected to be stagnant? And further, how should the upcomign Knight’s Landing processor and anything integrated that NVIDIA, IBM, AMD and others do be classified if the accelerator is inside the chip–won’t that become the norm?

Here is more text.