Meet the new HPC, where Productivity is our middle name. High Productivity Computing is not a whim; it is a new set of market dynamics that is more relevant to today's global economy. For a growing number of users and vendors, HPC refers not to cores, cycles, or flops but to discovery, efficiency, or time to market.

DARPA notably used the new P for the High Productivity Computing Systems (HPCS) program, stating that “value will be determined by assessing many additional factors beyond just theoretical peak flops.” There is also a proliferation of vendors in the market now selling pieces of the HPC solution. Tabor Communications is not pushing the industry in a new direction; we are updating the definitions to head in the same direction the industry is already going.

It's not as if none of us cared about productivity before. Of course we did. Productivity has always been the main intended benefit of HPC, and we never suffered from abbreviation-induced heartburn. So why does P no longer stand for performance?

Performance (defined roughly as what the server could accomplish with a Utopian application load) and productivity (defined roughly as the advantage gained by the user) were at one time tightly correlated, such that an increase in raw performance would boost the speed of generating results. Thanks to a variety of immutable technical and market forces — clustering, data explosion, and multi-core come to mind — the correlation between raw performance and productivity has broken down.

Over time the industry has moved from uni-processor to SMP to distributed-memory clusters. Each step toward higher absolute performance has been taken at the cost of additional architectural intricacy. The newest challenge is the transition to dual-core, quad-core, and future higher-level multi-core chips. The introduction of a new level of complexity at the socket level drives another wedge in the gap between theoretical and realized performance. Most applications will not take full advantage of additional cores without changes to the codes themselves or to the productivity tools surrounding them.

We have reached a point where there are too may bottlenecks. On systems with the same processor counts and types, productivity can vary widely depending on the workload manager, file system, interconnect, or scheduler, just to name a few.

The challenge is not unique to HPC. It applies equally well to business computing and to PCs. This is both a comfort and a curse. It increases the odds that someone will develop compelling solutions for multi-core parallelism, but it decreases the odds that these solutions will be designed and built with HPC in mind.

All is not lost. There are technologies in development — schedulers, workload managers, accelerators, tools — that have promise in easing the multi-core productivity-performance gap. We need to give these technologies a fair chance in the market. We need constructs for analyzing the efficacy of these tools. As an industry, we need to measure and reward productivity, not performance.

Tabor Research has an inclusive view that defines High Productivity Computing as follows:

High Productivity Computing (HPC) is the use of servers, clusters, and supercomputers — plus associated operational components such as software tools, networks, storage subsystems, and services — for scientific, engineering, or analytical tasks that are particularly intensive in computation, memory usage, or data management. HPC is used by scientists and engineers both in research and in production across industry, government, and academia. Within industry, HPC can frequently be distinguished from general business computing in that companies generally will use HPC applications to gain advantage in their core endeavors — e.g., finding oil, designing automobile parts, or protecting clients' investments — as opposed to non-core endeavors such as payroll management or resource planning.

This updated view of the market requires a change in market research, because it adds users, vendors, and applications to the industry. Forecasting and analysis are the cornerstones of the business, and now have more things to count. Tabor Research is also incorporating demand-side research to gain a view toward how users spend their money — including hardware, software, facilities, and staff — to achieve real productivity with today's systems.

From a vendor perspective, this definition requires us to push research beyond big server vendors' main product lines to include their partners and solutions for all forms of workflow optimization. Second-tier vendors play a major role here and become an important part of the market census.

From a user perspective, there are categories of HPC applications that the old definition failed to cover. HPC technologies have become more accessible and are now being adopted more rapidly in commercial markets. Online gaming, for example, has been using HPC system configurations to meet the real-time I/O requirements of hosting massive multi-user domains. Non-traditional application areas across the spectrum of system sizes and configurations are becoming the face of the new HPC. (For more on our broadened application scope, register for free to read “Five Important Predictions for HPC in 2007” at www.taborresearch.com.)

This approach to gathering market intelligence requires more work, both from the analysts and from the community, but it is ultimately more rewarding for all of us. For the cost of answering a survey now and then, you not only gain a better understanding of where you stand in the community, but you also make your opinions heard and thus steer the course of HPC development.

Tabor Research isn't inventing the new HPC. It already exists. Our goal is to count it and forecast it. And if the industry gains a better understanding of the ways in which Productivity, not merely Performance, is at the heart of HPC, we'll all be better off.

—–

Addison Snell is the VP/GM of Tabor Research, providing actionable market intelligence for High Productivity Computing. You can register for free to download exclusive premium content and record your opinions on the market at the new Tabor Research website, debuting today at www.taborresearch.com.