IDC painted a rosy picture for the HPC server market in the third quarter of 2007. According to the analyst firm, revenue for the HPC market grew 8.8 percent from Q2 and 18 percent compared to the same period last year. The total HPC server revenue in Q3 worked out to $3.0 billion, which represents an astounding 22.9 percent of the $13.1 billion worldwide server revenue in Q3, a 0.5 percent increase from Q2 and a 6.3 percent increase from Q3 2006.

Another way to look at it is that non-HPC server revenue declined slightly in the third quarter and only managed anemic year over year growth. Without HPC to buoy it up, enterprise server sales are flagging. It's pretty clear that virtualization is asserting downward pressure on the overall server market, allowing users to get more application mileage out of less hardware. But at this point, high performance computing has resisted the effects of virtualization. The typical HPC application demands more compute cycles than are typically available in a single server, so sharing that server would defeat the purpose. Today, HPC application virtualization only makes sense over an entire server grid, but that model doesn't explicitly constrain server purchases.

Embedded in the surging HPC revenue are the processor sales. In 2006, HPC systems accounted for 26 percent of all processors sold in the server market. Intel's latest push to position its new "Harpertown" and "Wolfdale" Penryn server chips as HPC processors and AMD's struggles to feed the demand for its new quad-core Opterons are two indications of how important the HPC space has become to the chipmakers. While both Intel and AMD are busily adding virtualization support in their hardware, at some level they probably wish virtualization would just go away, since it reduces demand for their server chips. In HPC though, they see the opportunity to feed a beast with an insatiable appetite for compute cycles.

IDC attributes much of the HPC revenue growth to lower entry prices. The fastest growing segment is at the low end -- systems priced under $50,000. With more powerful processors available, workgroups, and even individual engineers and researchers can purchase HPC systems for as little as $10,000 dollars. IDC projects the sub-$50,000 segment to have an 11.4 percent CAGR through 2011.

The other big driver IDC identified is the growing trend of replacing physical R&D with computer simulation and modeling. Even if HPC system prices were stagnant, there is a cost benefit of using computing to replace physical experimentation, since the latter is almost always labor and capital intensive.

Looking at the yearly HPC revenue, IDC reported server revenue topped $10 billion in 2006. According to them, adding in other elements of the ecosystem like storage and services pushes the total to $16.3 billion. Even this figure may be conservative. Tabor Research, using an end-user research approach, estimates HPC server revenue represents only 43 percent of the total spending, even excluding things like staff, facilities and power/cooling costs. Using IDC's $10 billion server figure, the Tabor Research model would yield something closer to $23.5 billion in total spending. Now you're talking real money.

Of course, all of this growth is riding on the popularity of cluster computing systems, which in the third quarter represented 68 percent of all HPC server revenue. With no competing architecture on the horizon offering comparable price/performance for the majority of applications, clusters are destined to maintain their dominance in HPC for the foreseeable future.

Which brings us to something of a paradox. If clusters are such a growth industry, why aren't there more publicly traded cluster computing system vendors? The big vendors, like IBM, HP, Dell and Sun Microsystems, are public, but the HPC server business is just a slice of a much larger set of offerings. All of the HPC-only cluster system vendors, such as Appro, Linux Networx, Penguin Computing, etc., are privately held.

Supermicro Inc., a company that sells a range of all-purpose x86 servers, some of which make it into the HPC market, launched its IPO in March 2007. Although the company is profitable, the current stock is sitting at about $8.50/share (the initial offering was $8.00/share). With double-digit HPC cluster growth, one might have predicted better results. Time will tell.

The closest thing we have to a pure HPC cluster company that's publicly owned is SGI, although their globally shared memory Itanium-based Altix line technically disqualifies them. Plus, the company offers storage and visualization products and also generates significant revenue from services. In any case, as I reported last week, SGI's year-old cluster business is just now attempting to reach escape velocity and turn a profitable quarter. Their stock is currently just above $17/share, following an April high of $30.

Rackable Systems is another publicly traded cluster vendor, but, like Supermicro, it targets a much wider audience than HPC. The company also offers a number of storage products. Despite some innovative engineering in power and cooling, Rackable is finding it tough going in the cluster marketplace after a banner year in 2006. The company's stock is in no better shape than SGI's -- worse actually. It's now hovering at around $10/share, down from a January high of over $30.

Theoretically, the tier two players who specialize in cluster systems should be able to compete effectively against the tier one vendors, by either adding value, undercutting prices, or both. But the commodity nature of cluster computing cuts both ways. It makes entering the market easy, but establishing long-term differentiation hard. Almost everyone is building these machines from the same commodity parts: x86 processors, standard memory chips, Linux (or Windows) software, and Ethernet or InfiniBand gear. Higher value features that address usability, ease of deployment, and manageability are only now starting to be perceived as equally important as raw performance. With thin profit margins on the hardware, vendors are using these higher value features as their "secret sauce."

The tough competition won't dissuade vendors, though. With IDC projecting $15 billion in yearly HPC server revenue by 2011, there will be plenty of companies satisfied to get a slice of the market. The fun is just beginning.

-----

As always, comments about HPCwire are welcomed and encouraged. Write to me, Michael Feldman, at editor@hpcwire.com.