If one were to categorize the enterprise software market as mature, robust, innovative — and definitely 21st century — as it races headstrong into the cloud — how would one categorize the HPC software market? Not to throw stones, but you could easily put it in the circa 1980 timeframe and use terms like immature, cottage-like and definitely lacking investment. When you mention HPC to any of the venture guys, they run for the hills.

I can rewind 20+ years to when I first entered this market and the conversation has not changed. No software, no money for software, and a continuous “humm” over the impracticalities of building software for an ever increasingly complex set of platforms. The dialogue never seems to evolve beyond parallel programming, languages (Fortran and C — of course), open source and the vertical application specialists who seem to own the scarcest commodity.

Now mind you, there is an implicit expectation that the government should invest in and drive the initiative. Not only is the conversation focused in the wrong direction, but we are not even asking the right question! In my opinion, the issue is much more complex. Don’t get me wrong. These are not absolutes, and we certainly need to get very real and focused on solving the challenges associated with programming models, as well as building robust middleware and tools. The big question, however, is around productivity, not platforms; and how do we make high performance systems fit seamlessly into an overall IT environment? For me the question is “where is SAP for the rest of us?”

If productivity is the “uber-trend,” are we focusing on the right issues? A quick analysis of the numbers tells a very interesting story. The majority of the market is comprised of smaller clusters — not extreme levels of parallelism. Yes, multicore will make things more complicated. But the heart of the market (the sweet spot) is in the midrange and in the industrial sector — broadly defined. This is where the growth is, and this is where customers need help.

What is interesting to me is that after all of these years, we have not found the “common thread” that links all of these segments. An HPC “ERP” equivalent, if you will. Maybe, we haven’t looked! Workflows in product development are pretty similar. Supply chains are complex and growing more complex; and again, have common attributes. Data volumes and the management and use of that data are becoming gigantically difficult. I can’t begin to count the number of users who ask why we don’t have some sort of application framework that enables applications to “speak to one another.” I dug into my personal archives to find some anecdotal statements from discussions I have had recently.

• “We’re knee deep in silos of data; there is no unified view.”

• “My applications don’t talk to one another.”

• “What data management?? We use sneaker-net!”

• “We have a massive data redundancy problem, but we lack the tools to fix it.”

• “We’re desperate for collaboration tools.”

The list goes on….

My apologies to those who have innovated on this front. PTC (Parametric Technology Corp.) and Accelrys immediately come to mind — but these attempts are highly verticalized. If you are in the manufacturing segment, you are in luck. Between PTC and Dassault there are solutions. PTC has found the “killer app” in the manufacturing space — PLM. But why hasn’t that translated to other segments?

I’ve heard all the arguments, ranging from HPC applications are too niche; there is no demand; to the lack of common horizontal applications. But again they miss the point that there is a great deal of commonality within and across engineering and scientific workflows. Common requirements exist around data transparency, data analysis, and data management. There are also requirements for integrated applications that provide consistency and efficiency between elements of a workflow, across an organization or beyond the borders of a corporation.

Maybe it is a matter of timing. HPC is hot right now and getting hotter every day. From my vantage point, this screams of opportunity. Someone, please jump in!