Just a few years ago it seemed likely — perhaps inevitable — that we were headed for the Henry Ford business model in HPC: you would be able to get any kind of machine you wanted, as long as it was an x86-based cluster. Having grown up in HPC in the ’90s I found this, frankly, depressing. Relative to the diversity I knew at the start of my career, this seemed like a pretty sparse landscape.

Happily things are turning out to be much more interesting than that. GPGPUs, FGPAs, accelerator boards, Cell processors, and hybrid architectures are being put through their paces as they try to find, and hold on to, a niche in the market. All this diversity and competition is, as Martha Stewart would say, a Good Thing.

Most of this variety comes from a whole host of companies, each specializing on a piece of the puzzle. Very few companies are bucking this trend, offering products across the full breadth of the HPC market. IBM is one of those companies, with a very broad product offering that ranges from x86-based systems to custom big iron. The past few months have been busy for Big Blue, with announcements across much of their product line ranging from product refreshes — like the latest Intel chips plugged into the cluster offering — to re-engineered offerings like the new QS22 Cell-based blade.

According to Herb Schultz, marketing manager for Deep Computing at IBM, all of this change is customer-initiated, and indicates a strong demand in all aspects of IBM’s HPC customer base. “A few years ago everyone thought the market would shrink down to basically a single offering,” says Schultz. “What we’ve seen is an expansion of demand from a variety of customers all with different needs, and a growth in the hardware diversity that IBM needs to offer to meet those needs.”

IBM’s foothold in the traditional HPTC market is driven by its Power-based servers, including ownership of four of the top ten slots on November’s Top500 list by the Blue Gene. Last month IBM announced some love for its HPC bread and butter, the p series of machine, by moving its new Power6 processor into an HPC package with the release of the p575. The p575 uses a 4.7 GHz version of the Power6, and caught a lot of attention in the general IT press for its reintroduction of an old idea: water cooling.

One p575 rack can hold 14 2U nodes, each with 32 4.7 GHz cores of Power6, and up to 256 GB of memory per node. That puts 448 cores in a rack; at 600 GLFOPS per node, this is 8.4 peak TFLOPS per rack. That’s a lot of compute power in a business where compute power equals heat, and the special cooling treatment supports the system’s big stats. (For more on the p575 architecture and engineering, check out our previous coverage.)

Schultz indicates that much of IBM’s growth in HPC is being driven by non-traditional users, especially at the low end and in the middle of the market. According to him, IBM is experiencing big uptake of HPC on the commercial side. Customers are coming from financial services with needs for financial instrument engineering, and from manufacturing where HPC is used for product and packaging designs for everything from washing machines to inhalers for asthma medication. Overall, Schultz estimates they are seeing double digit growth in the mid-market, “and not just the same people buying more equipment, but new entrants who have never been in HPC before.” In this market IBM knows that they have to deliver more than FLOPS, “SMB customers are looking for ease of ordering, installation, and maintenance,” says Schultz, in addition to a lower price point for the hardware itself.

IBM has announced interesting products and product evolutions for this market. iDataPlex, available this summer but announced in April, is IBM’s shot at server provisioning for the “web-scale” crowd with a nod to price-conscious HPC needs at the same time (more here). The hardware is commodity x86 fare, running the usual assortment of Linux. What makes iDataPlex interesting to the IT community is the engineering that has gone into getting a bunch of these things crammed in a rack economically.

An iDataPlex setup holds the nodes sideways from the usual orientation, and combines two racks worth of them in a package that’s wider than deep. Fans are in the back, but with a shorter distance to pull the cold air across, they use less power and keep components cooler. The iDataPlex is designed for a crowd that values price and quantity over reliability and other such fancy features. The system has fewer redundant components and a simpler design that favors a “pull and replace” approach to node failure over the traditional “predict and manage” approach.

IBM will offer up to 22 different chips and motherboard combinations for the nodes, allowing customers to tailor systems precisely for their needs. You want low power Xeons with slow memory? No problem. Or, for a no-frills-added HPC machine, you can max out on computing power and memory in a full rack of these, and keep it cool with optional water-cooled rear doors.

IBM is also still tuning its Cell B/E-based Q series of blades for HPC and, in particular, for business high performance computing. IBM’s new Q22, announced this month and shipping in June, sports a pair of modified Cell processors with native double precision computation (read more here). The system has also been modified to support up to 16 times the memory of the last Q blade, with up to 8 DIMM slots. These changes have resulted in a Cell blade that IBM feels is finally ready to move out of niche markets and into broader enterprise adoption supporting computationally-intensive tasks with large memory footprints. IBM is also using its RoadRunner experience to test out the Cell blades in hybrid systems with X86 nodes, in effect using the Cells as accelerators.

For Blue Gene, IBM has a big future in mind. You may have heard about Kittyhawk, a Watson Research Center project to “explore the construction and implications of a global-scale shared computer capable of hosting the entire Internet as an application.” IBM sees Blue Gene as having the potential to be the ultimate throughput machine: dense, power efficient, with thousands of processors at its disposal. Schultz talked briefly about modifications to the system software, to be announced in the near future, that will allow Blue Gene to operate in a high throughput mode. As he described them, the changes are primarily related to services offered by the OS, and to the scheduling of tasks on the machine that allow much finer-grained targeting of tasks to processors.

And IBM isn’t just focused on hardware and services when it comes to its plans for HPC. The company has decided, in a move that would seem to be a big departure from its usual way of looking at the world, that it should open source much of the software it has developed over the past 20 years for this market. IBM plans to open source applications like LoadLeveler and its high performance math libraries. Detailed plans are expected this summer, but Schultz did reveal that the software repository will be managed by the University of Illinois. Said Shultz: “IBM really feels there is a lot of value in this software, and opening up the IP will benefit the community much more than keeping it closed and putting out twice yearly releases.”