HPCwire: Cluster Lust

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

HPCwire >> Blogs

Blog: From the Editor

October 05, 2007

Cluster Lust

The combination of quad-core Opterons and DDR Infiniband is re-landscaping the HPC terrain and is propelling the largest clusters to the top of the high performance heap. A rash of recent announcements of big system purchases suggests good times ahead for HPC cluster vendors.

Or does it? Let's look at the good news first.

In the space of five days, over 500 teraflops of systems were announced -- all, coincidentally, for DOE labs. Last Friday, the Pacific Northwest National Laboratory (PNNL) revealed it had purchased a 163 teraflop HP Proliant server cluster. Then on Tuesday, Appro announced it had been awarded a contract to deliver 438 teraflops of computing hardware to three other DOE labs -- Lawrence Livermore, Los Alamos and Sandia. Add this to the 500 teraflop Ranger system currently being installed at the Texas Advanced Computing Center (TACC) and you have over a petaflop computing power -- all based on the new quad-core Opterons and DDR InfiniBand technology.

The Appro contract will result in eight new Linux clusters for the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, which supports the NNSA's mission to maintain the country's nuclear weapons. Three clusters are headed to Lawrence Livermore, two to Los Alamos and the remaining three to Sandia. Of these, the largest is a 162 teraflop machine for Livermore. The contract also contains an option for perhaps three additional clusters, representing 200 additional teraflops.

Prior to this, Appro had a presence at all ASC DOE labs, but nothing on this scale at Los Alamos and Sandia. The company's large three-cluster Peloton deployment at Livermore in 2006 was just a prelude to this week's ASC award. It's a huge win for Appro.

The contract also represents a new approach to purchasing computing resources by the ASC program, which chose to make a single large purchase for its capacity computing requirements for the next two years, instead of six smaller purchases over the same time period. Our feature article this week delves more deeply into the new clusters and the strategy behind the unified procurement for the three labs.

HP's 163 petaflop cluster, headed to PNNL in Richland, Washington, is just one of two 100-plus teraflop systems that the company will be delivering over the next year. The PNNL system will be used to drive the lab's molecular science research in applications such as aerosol formation, bioremediation, catalysis, climate change and hydrogen storage. The machine will be deployed in two phases and won't be fully operational until September 2008. The other system, a 182 teraflop blade-based cluster for an unnamed Swedish government agency, is expected to be fully operational as early as next month. That could give HP a more respectable presence in the top 10 of the Top500 list. The current top-ranked HP system is the 20.5 teraflop ASCI Q machine at number 62.

The dynamic duo of multicore x86 processors and InfiniBand should give clusters a real boost over the next few years. At least in raw performance, the top commodity-based clusters are now on par with the fastest high-end supercomputers from Cray and IBM. Until the petaflop-class XT and Blue Gene/P systems come online, the capability machines and the scaled-out clusters will vie for peak performance leadership. And while these clusters may not provide the same level of sustained application performance as their more expensive supercomputing brethren, their price-performance is about twice as good.

But the sub-100 teraflop cluster market is where the real market action is, and these systems are just getting less expensive and more popular every day. This makes them an increasingly attractive proposition for a wide range of users, which in turn makes them an increasingly attractive proposition for a wide range of vendors.

In a sense, that's the bad news. The promise of a fast growing market has attracted an abundance of companies. In general, the HPC cluster market is now over-served. There are a lot of cluster vendors out there without much fundamental product differentiation among them. Standard 64-bit x86 processors and InfiniBand interconnect technology have become the basis for hundreds of cluster computing offerings. These two elements -- compute and communications -- dominate how well a cluster performs.

Companies that have broken out of this mold, like SiCortex and (to a certain extent) Liquid Computing, have come up with truly innovative solutions that are unlike the traditional model. The challenge for them is gaining a foothold in a market that is risk-averse and extremely price sensitive. These companies have chosen a tough path, but at least they avoid direct competition with the 50 other system vendors selling what are essentially the same boxes.

To be fair, for the more mainstream cluster offerings, product differentiation does exist in technology, customization options, configuration, support, service, software bundling, storage offerings, etc., and these elements are all being applied for competitive advantage. But sometimes these advantages are only temporary. Rackable created a unique line of x86-based systems, with innovative power and cooling technology at competitive price points. Then everyone jumped onto the power/cooling bandwagon, dulling the benefit of Rackable's offerings and making them easy prey to aggressive pricing by Dell and HP. More recently, Rackable is fighting back with a mobile datacenter called Concentro, designed to go up against Sun Microsystems' new Blackbox offering. The lesson here is that vendors copy good ideas from each other to the extent possible, which, over time, marginalizes competitive advantages.

Since the HPC market -- and more specifically, the HPC cluster market -- is growing at double-digit rates, there has been little pressure for consolidation. Rapid growth is not conductive to the "normal" maturation process of eliminating weaker competitors and products. But I suspect the next sustained economic contraction will change that. With signs of inflation and rising interest rates starting to appear, an economic slowdown, if not full-blown recession, could happen next year.

Even in the absence of a deteriorating economy, there is constant pressure on buyers to look for purchasing efficiencies. The ASC program procurement strategy, mentioned above, is one such example. It could provide a model for how large organizations will buy high performance computing in the future. In the ASC case, the labs were specifically not looking for innovative computing. They already bought that in their high-end supercomputer systems, typified by Lawrence Livermore's Blue Gene/L. What they were looking for was a standard, replicable architecture for capacity computing that they could buy in bulk. But by replacing a number of smaller purchases with one large one, they explicitly limited the number of vendors in the ASC program.

Overall, the larger tier 1 system vendors seem to be least at risk, since they can offer more complete solutions, better support and a variety of services. They sell the same product lines into a broad customer base in both HPC and traditional enterprise computing. On the other hand, they are often less able to compete on price. The tier 3 shops fill a need for users looking for a one-night stand in high performance computing and can offer rock bottom prices. Tier 2 vendors provide an attractive middle ground, as was the case for the ASC procurement.

Whether market forces thin the herd quickly, through a recession, or more slowly, via a steady maturation of the ecosystem, depends upon future economic conditions. Over the long term, the larger vendors are going to be the most resilient. The smaller companies with narrower customer bases are already looking over their shoulder. Whatever the situation, if you're an x86 system vendor, hang on to your clusters. It's going to be a bumpy ride.

-----

As always, comments about HPCwire are welcomed and encouraged. Write to me, Michael Feldman, at editor@hpcwire.com.

Posted by Michael Feldman - October 5 @ 12:00AM

(Digg, Technorati, more)

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

Michael Feldman

Michael Feldman is the editor of HPCwire.

More Michael Feldman

Feature Articles

The Week in Review

C-DAC announces plans for a petaflop system; IBM researchers are working on vertical integration techniques to extend Moore's Law another 15 years. We recap those stories and more in our weekly wrapup.
Read More...

Moscow State University Supercomputer Has Petaflop Aspirations

The Moscow State University supercomputer, Lomonosov, has been selected for a high-performance makeover, with the goal of tripling its processing power to achieve petaflop-level performance in 2010. T-Platforms, who developed and manufactured the supercomputer, is the odds-on favorite to lead the project.
Read More...

Intel Ups Performance Ante with Westmere Server Chips

Right on schedule, Intel has launched its Xeon 5600 processors, codenamed "Westmere EP." The 5600 represents the 32nm sequel to the Xeon 5500 (Nehalem EP) for dual-socket servers. Intel is touting better performance and energy efficiency, along with new security features, as the big selling points of the new Xeons.
Read More...

Read more HPCwire features...

Top Headlines

Australia Commissions Cray Supercomputer

Mar 19 | OfficialWire | New super to support intelligence work Down Under. Read more...

Intel Partners See 'Easy' Upgrade Path With Xeon 5600 Chips

Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...

AMD: OEMs primed for Opteron 6100s

Mar 17 | The Register | But what about the tier ones? Read more...

Arrival of the Desktop Supercomputer

Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...

Scheduling HPC In The Cloud

Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...

Read more headlines...

Featured Whitepapers

Virtualization for Aggregation And The vSMP Architecture™

Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.

Copper Cable Technologies for High Performance Computing

Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.

View the White Paper Library

Multimedia

Webcast: Virtualized Data Center Roundtable

Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.

Webcast: Watch SC09 Birds of a Feather Video: Scalable Fault-Tolerant HPC Supercomputers

Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.

Webcast: High Performance Computing for a Smarter Planet

LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html