The HPC sector is on the cusp of yet another change in the way it does business, except this time the pendulum is swinging away from HPC vendors having the upper hand in choosing the products they deliver towards HPC users who define their workloads and then assemble the vendor-agnostic hardware and software they need to get the job done. As a result, in the near future and more so as time progresses, considerations of brand loyalty, vendor lock in, legacy hardware, and proprietary software will become less important, perhaps even fading away. Instead, users will increasingly look to a collection of vendors to supply the targeted solutions they need–without having to worry too much about the specific hardware and software used to deliver those solutions.
The analogy with the evolution of commercial television is simply too good to resist. In the early days of broadcast television, there were only three networks (CBS, NBC, and ABC), and everyone watched whatever and whenever those networks chose to broadcast. Some upstarts came along, like the Fox network and some local stations, but they basically followed the same model: give them what we want. Eventually, cable appeared and changed the model somewhat: premium networks like HBO and Showtime offered new options but defined themselves essentially as being not like the networks. And people subscribed because it was HBO and Showtime, but content was still a push rather than pull business.
And then came streaming, which redefined choice by giving viewers the upper hand in determining exactly what they could watch. Moreover, it soon became apparent that within the streaming world, content was king and that the actual content provider mattered little, if not at all. Viewers were drawn not to the provider but instead to the content, regardless of where it was available and who delivered it, and they could access it whenever they wanted.
Today, when someone wants to watch Ted Lasso, it’s not because it’s on Apple TV but because it’s Ted Lasso. Apple TV is merely the conduit. Here, there is no brand loyalty to Apple TV, or HBO, Hulu, Amazon Prime or whoever. The allegiance lies with the content and not the provider, and the underlying technology makes it easy to support that model. All viewers have to do is find the show, subscribe to the specific service, and the hardware and software deliver, albeit for a small monthly subscription fee. Once TV viewers got a taste of this flexibility, there was no going back.
So, like the major TV networks of old, the major HPC hardware and software vendors will need to address this new steaming HPC model if they are to stay relevant. This is critical as streaming in the HPC world, like that in the commercial TV world, is well suited to support a spate of new entrants who can take advantage of the low barriers to entry to provide new, innovative, or even experimental content that users may want, or at least want to try out.
Support for this new streaming model is being driven by a number of related developments within the HPC sector. Virtualization software and associated container technology is enabling flexibility in hardware choices and decoupling hardware from software. This opens new opportunities for users to select the appropriate vendors and their associated products with limited penalty for moving away from existing legacy or brand commitments. Hardware vendors have already started to respond to this shift by adding more options to their offerings: increasingly HPC users can pick from a menu of hardware options from progressive HPC vendors including a range of processor choices, such as x86 or Arm, GPU, or related FPGA accelerators, like those offered by Nvidia, AMD, or Intel, as well as diverse memory, storage, interconnect, power/cooling options, and, the near future, even quantum-based accelerators.
Likewise, compute locations are becoming more fluid on the edge, in the datacenter, in the cloud or some mix of any of the above, increasingly defying the traditional concept of a centrally located data center – be it physically on-premises or virtually in the cloud. The growing spate of IoT data collectors and sensors will only add another dimension to those looking to wring critical insights out of massive data sets, often in real-time. Likewise, within the cloud world, options for direct user engagement in the HPC computing experience is proliferating with choices including Infrastructure as a Service, Platform as a Service, Software as a Service, and even Everything as a Service.
Perhaps most importantly, end user workload variety is expanding. Traditional modeling/simulation is increasingly sharing space with big data analytics and AI-based jobs, each with their own unique and sometime orthogonal hardware and software requirements, except when they are blended, such as AI-accelerated simulation jobs. In all cases, workload-specific architectures are becoming an increasingly popular option for many users, replacing the one-size fits all HPC architectural paradigm that currently dominates. And in the near-future, new workloads will almost certainly emerge, shaking up the sector with new use cases, engendering heretofore unanticipated hardware and software requirements, similar to the way AI and GPUs fueled an upheaval in HPC architectural requirements and programming models.
How this all may work out: HPC users will seek out vendors that support flexibility to move applications to the computing environment that best suits performance and price requirements while avoiding issues of technology lock-ins, with the same ease as canceling one’s cable channel subscription when a better one comes along. This gives those users significant insurance for future proofing their computing ecosystem should any new technology developments arise. The most successful HPC vendors will understand this and will work hard to configure their product lines for flexibility and portability, looking to offer the most inviting configurations of hardware and software best suited to for the broadest base of use case scenarios. Winners here will understand that loyalty, like that once enjoyed by the big three television networks, is a thing of the past, and that it’s time to move to a development and product offering philosophy that reflects exactly what each HPC user wants, lest they decide to see what’s on another channel.
Bob Sorensen is Senior Vice President of Research, Hyperion Research.