Borne on the rising wave of emerging high performance data analytics (HPDA) and machine learning use cases, along with global government spending on development of exascale-class system the HPC server market grew at a healthy year-over-year rate of just under 10 percent in 2017, according to industry watcher Hyperion Research (formerly IDC’s HPC analyst group).

At its mid-year HPC market update breakfast at ISC18 in Frankfurt, Hyperion CEO Earl Joseph said the industry in Q1 of this year continued the momentum achieved 2017, and he expects the industry to grow at a healthy 10 percent CAGR clip through 2022. For 2017, industry server revenues totaled $12.3 billion (up from $11.2B in 2016), with the supercomputer sector leading the way at $4.8 billion, followed by divisional and departmental systems at $2.3 billion and $3.4 billion, respectively.

Beyond the spurs of AI and HPDA, Joseph also pointed to HPC offerings from public cloud service providers are a source of growth.

“The cloud portion is lifting the market, too,” Joseph said. “A big question was: Will people stop spending on-prem and spend more on the cloud. Across the market, that doesn’t happen. It’s like the cluster revolution of 15-20 years ago. When you can do more HPC, which really is doing more R&D in general, you’re more successful. And organizations and governments tend to give you more money. We generally are seeing the cloud lifting the market as a whole and bringing more money and expenditures into organizations.”

As for server vendor market share, HPE continued its big lead with $4.2 billion in HPC systems sales last year, nearly twice those of Dell EMC at $2.3 billion, followed by Lenovo at $870 million and Inspur at $740 million.  IBM HPC systems revenues were $575 million, up from $492 reported a year ago.

For the broader market, accounting for all HPC-related technologies, high performance storage revenues continued to grow faster than the server market, showing YOY expansion of 10.3 percent, totaling $4.7 billion in 2017 sales, while HPC software applications sales totaled $4 billion. The total HPC market (including servers) came to $24.3 billion last year, according to Hyperion, and is expected to reach $38.4 billion by 2022.

The government, defense and university sectors purchased $5.7 billion of HPC servers last year, with industrial sectors spending $6.5 billion, led by CAE, bio-sciences, geosciences DCC and distribution and financial services.

Hyperion predicts that by 2021, the HPDA market will reach $6.6 billion in technology revenues ($4 billion for servers, $2.6 billion for storage). Hyperion COO and SVP of Research Steve Conway said HPDA growth is pushed by new industrial use cases in fraud detection, BI, affinity marketing and personalized medicine.

“New commercial users (are coming into the market), and we don’t mean automotive, pharmaceuticals, and oil and gas,” Conway said. “We’re talking about moving over to the enterprise data center and plugging HPC into the live workflow there.”

As HPDA evolves, so will its ability to take on increasingly sophisticated problems.

“Today, we’re definitely in the era of machine learning – but for bounded problems,” Conway said, “including relatively less complex workloads for image and voice recognition… These are problems where lots and lots of observations (is required), but not a lot of choices have to be made in the end. Even with automated driving as it’s envisioned right now…, ultimately there are not a lot of choices but a heck of a lot of observations.”

Unbounded problems, on the other hand, “are the money makers,” Conway said, “in precision medicine, fully automated driving. There you have many observations but also many, many choices to make, so they are much more difficult.”

Transparency into ML decision making remains a major potential problem, Conway said, citing a fatal accident earlier this year when an Uber self-driving car hit a pedestrian.

“There’s great concern among the buyers in this market – the automotive companies, the healthcare companies, and so forth – that it won’t take too many accidents like that for society to react and possibly slow down progress,” he said.

Another challenge is for organizations to access enough data required to train machine learning systems. Conway cited a statement from Marti Head, senior director at GlaxoSmithKline, who said “The amount of data available today is miniscule compared to what we need for deep learning.”

“The early ecosystem that we’re in was first formed around social media and the web giants because they have lots of data to work with,” said Conway. “That ecosystem is not particularly applicable to lots of other markets, like healthcare, so that needs to evolve… But (organizations) can still do a lot of activity. People know they’ve got to get there, so they’re spending a lot of money.”

While HPC in the cloud is helping to expand the HPC market, Conway said it’s a “phenomenon (that’s) still much wider than it is deep.” He said the percentage of HPC centers running some jobs in public clouds has grown quickly, “the percentage of all of their HPC jobs they’re sending to HPC clouds is not growing very fast. It’s still just under 10 percent at the moment.”

The constraint, he said, “is on the side of the cloud services providers, not so much the users. As cloud services providers make the cloud architectures friendlier toward a broader spectrum of HPC job types, then more jobs will go to those clouds. And it is happening.” Conway said Hyperion expects total HPC jobs running on public clouds to grow to 15-20 percent over the next few years.

He also said that during the second half of 2017, Hyperion detected growing popularity of “virtualized private clouds” from several public cloud service providers offering isolated, on-demand pools of shared computing resources.