HPCwire: Cluster Lust

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

Leading HPC
Solution Providers

HPCwire >> Blogs

Blog: From the Editor

From the Editor | Main Blog Index

October 05, 2007

Cluster Lust

The combination of quad-core Opterons and DDR Infiniband is re-landscaping the HPC terrain and is propelling the largest clusters to the top of the high performance heap. A rash of recent announcements of big system purchases suggests good times ahead for HPC cluster vendors.

Or does it? Let's look at the good news first.

In the space of five days, over 500 teraflops of systems were announced -- all, coincidentally, for DOE labs. Last Friday, the Pacific Northwest National Laboratory (PNNL) revealed it had purchased a 163 teraflop HP Proliant server cluster. Then on Tuesday, Appro announced it had been awarded a contract to deliver 438 teraflops of computing hardware to three other DOE labs -- Lawrence Livermore, Los Alamos and Sandia. Add this to the 500 teraflop Ranger system currently being installed at the Texas Advanced Computing Center (TACC) and you have over a petaflop computing power -- all based on the new quad-core Opterons and DDR InfiniBand technology.

The Appro contract will result in eight new Linux clusters for the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, which supports the NNSA's mission to maintain the country's nuclear weapons. Three clusters are headed to Lawrence Livermore, two to Los Alamos and the remaining three to Sandia. Of these, the largest is a 162 teraflop machine for Livermore. The contract also contains an option for perhaps three additional clusters, representing 200 additional teraflops.

Prior to this, Appro had a presence at all ASC DOE labs, but nothing on this scale at Los Alamos and Sandia. The company's large three-cluster Peloton deployment at Livermore in 2006 was just a prelude to this week's ASC award. It's a huge win for Appro.

The contract also represents a new approach to purchasing computing resources by the ASC program, which chose to make a single large purchase for its capacity computing requirements for the next two years, instead of six smaller purchases over the same time period. Our feature article this week delves more deeply into the new clusters and the strategy behind the unified procurement for the three labs.

HP's 163 petaflop cluster, headed to PNNL in Richland, Washington, is just one of two 100-plus teraflop systems that the company will be delivering over the next year. The PNNL system will be used to drive the lab's molecular science research in applications such as aerosol formation, bioremediation, catalysis, climate change and hydrogen storage. The machine will be deployed in two phases and won't be fully operational until September 2008. The other system, a 182 teraflop blade-based cluster for an unnamed Swedish government agency, is expected to be fully operational as early as next month. That could give HP a more respectable presence in the top 10 of the Top500 list. The current top-ranked HP system is the 20.5 teraflop ASCI Q machine at number 62.

The dynamic duo of multicore x86 processors and InfiniBand should give clusters a real boost over the next few years. At least in raw performance, the top commodity-based clusters are now on par with the fastest high-end supercomputers from Cray and IBM. Until the petaflop-class XT and Blue Gene/P systems come online, the capability machines and the scaled-out clusters will vie for peak performance leadership. And while these clusters may not provide the same level of sustained application performance as their more expensive supercomputing brethren, their price-performance is about twice as good.

But the sub-100 teraflop cluster market is where the real market action is, and these systems are just getting less expensive and more popular every day. This makes them an increasingly attractive proposition for a wide range of users, which in turn makes them an increasingly attractive proposition for a wide range of vendors.

In a sense, that's the bad news. The promise of a fast growing market has attracted an abundance of companies. In general, the HPC cluster market is now over-served. There are a lot of cluster vendors out there without much fundamental product differentiation among them. Standard 64-bit x86 processors and InfiniBand interconnect technology have become the basis for hundreds of cluster computing offerings. These two elements -- compute and communications -- dominate how well a cluster performs.

Companies that have broken out of this mold, like SiCortex and (to a certain extent) Liquid Computing, have come up with truly innovative solutions that are unlike the traditional model. The challenge for them is gaining a foothold in a market that is risk-averse and extremely price sensitive. These companies have chosen a tough path, but at least they avoid direct competition with the 50 other system vendors selling what are essentially the same boxes.

To be fair, for the more mainstream cluster offerings, product differentiation does exist in technology, customization options, configuration, support, service, software bundling, storage offerings, etc., and these elements are all being applied for competitive advantage. But sometimes these advantages are only temporary. Rackable created a unique line of x86-based systems, with innovative power and cooling technology at competitive price points. Then everyone jumped onto the power/cooling bandwagon, dulling the benefit of Rackable's offerings and making them easy prey to aggressive pricing by Dell and HP. More recently, Rackable is fighting back with a mobile datacenter called Concentro, designed to go up against Sun Microsystems' new Blackbox offering. The lesson here is that vendors copy good ideas from each other to the extent possible, which, over time, marginalizes competitive advantages.

Since the HPC market -- and more specifically, the HPC cluster market -- is growing at double-digit rates, there has been little pressure for consolidation. Rapid growth is not conductive to the "normal" maturation process of eliminating weaker competitors and products. But I suspect the next sustained economic contraction will change that. With signs of inflation and rising interest rates starting to appear, an economic slowdown, if not full-blown recession, could happen next year.

Even in the absence of a deteriorating economy, there is constant pressure on buyers to look for purchasing efficiencies. The ASC program procurement strategy, mentioned above, is one such example. It could provide a model for how large organizations will buy high performance computing in the future. In the ASC case, the labs were specifically not looking for innovative computing. They already bought that in their high-end supercomputer systems, typified by Lawrence Livermore's Blue Gene/L. What they were looking for was a standard, replicable architecture for capacity computing that they could buy in bulk. But by replacing a number of smaller purchases with one large one, they explicitly limited the number of vendors in the ASC program.

Overall, the larger tier 1 system vendors seem to be least at risk, since they can offer more complete solutions, better support and a variety of services. They sell the same product lines into a broad customer base in both HPC and traditional enterprise computing. On the other hand, they are often less able to compete on price. The tier 3 shops fill a need for users looking for a one-night stand in high performance computing and can offer rock bottom prices. Tier 2 vendors provide an attractive middle ground, as was the case for the ASC procurement.

Whether market forces thin the herd quickly, through a recession, or more slowly, via a steady maturation of the ecosystem, depends upon future economic conditions. Over the long term, the larger vendors are going to be the most resilient. The smaller companies with narrower customer bases are already looking over their shoulder. Whatever the situation, if you're an x86 system vendor, hang on to your clusters. It's going to be a bumpy ride.

-----

As always, comments about HPCwire are welcomed and encouraged. Write to me, Michael Feldman, at editor@hpcwire.com.

Posted by Michael Feldman - October 5 @ 12:00AM

(Digg, Technorati, more)

Subscribe to HPCwire

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

Michael Feldman

Michael Feldman is the editor of HPCwire.

More Michael Feldman

Feature Articles

Spider Up and Spinning Connections to All Computing Platforms at ORNL

Spider, the world's biggest Lustre-based, centerwide file system, has been fully tested to support Oak Ridge National Laboratory's new petascale Cray XT4/XT5 Jaguar supercomputer and is now offering early access to scientists.
Read More...

Wolfram Alpha: A Web-Based Application That Embraced Supercomputers

Wolfram Alpha, the Web-based computational engine introduced in May, is not a traditional supercomputing application, but relies on supercomputers to satisfy its unique requirements.
Read More...

TeraGrid '09: Student Participation Soars

There was a new energy at this year's TeraGrid '09 conference thanks to an outstanding turnout for the student program. Thanks to support from the National Science Foundation, more than 100 high school, undergraduate and graduate students were able to participate in the conference.
Read More...

Read more HPCwire features...

Top Headlines

3D Seismic Data: Taking a Smarter Approach to Interpretation

Jul 09 | Engineer Live | The demand for computational tools to underpin the 3D seismic interpretation process has never been more apparent. Read more...

Engineering Unemployment Soared in 2Q to 8.6%

Jul 08 | EE Times | Unemployment for U.S. engineers has reached record levels, according to government figures. Read more...

Gartner Adjusts 2009 IT Spend Downward Again

Jul 08 | Network World | Global spending for 2009 projected to drop 6 percent, for a total of $3.2 trillion. Read more...

Concurrent and Parallel Are Not The Same

Jul 08 | Linux Magazine | Portability or efficiency? Neither is guaranteed when writing explicit parallel code. Read more...

800 TFLOP Real-Time Ray Tracing GPU Unveiled, Not for Gamers

Jul 07 | Ars Technica | Japanese company builds custom ASIC to accelerate real-time ray traced rendering for the auto industry. Read more...

Read more headlines...

Featured Whitepapers

Parallel Computing Without Parallel Programming

Jul 10 | | Engineers, scientists, and other domain experts depend on the productivity enabled by very high-level language (VHLL) tools like MATLAB® and Python. However, as datasets grow larger and programs get more sophisticated, ordinary desktop computers can no longer keep up. The paper explores how to run VHLL programs on high-performance platforms without low-level reprogramming. Work with large datasets and complex algorithms without sacrificing ease of use or reducing productivity.

Building High Performance Computing in a Green and Modular Solution Building Block

Apr 14 | | Many HPC IT departments are feeling the rising pressure to deliver more capacity computing and performance while trying to reduce the total cost of ownership. This white paper discusses how an environmentally-friendly and open-standards HPC building block based computing system using flexible interconnect options helps address capacity computing needs.

View the White Paper Library

Multimedia

Webcast: Dell Expands HPC Access and Adoption with Intel Cluster Ready Program

Source: Addison Snell, GM/VP, Tabor Research; sponsored by Dell

Many organizations that could benefit from the use of HPC clusters find that it is complicated to get the systems up and running because of limited IT resources or the complexities of the clusters themselves. Learn how the Intel Cluster Ready program, for which Dell was an original partner, seeks to address this challenge for entry level and mid-range HPC users.

Video White Paper: Architecting a Better Network Storage Solution

BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.

Webcast: HPC Development Solutions: Sun Studio & Sun HPC ClusterTools

Sun Studio Compilers and Tools and Sun HPC ClusterTools allow you to create high performance parallel applications for OpenSolaris, Solaris and Linux. Sun Studio Express 11/08 includes MPI performance analysis capabilities and full OpenMP 3.0 compiler support. Learn about all this and the latest in Sun HPC ClusterTools 8.1.

More Multimedia

Blogs by Topics

Blogs by Author

HPC Blogroll

Featured Events

July 13-16, 2009
WORLDCOMP'09
Las Vegas , NV
USA

August 24-28, 2009
Data Mining: Levels I, II, & III
Denver , CO
USA

HPCwire

Tabor Communications | Tabor Research | Tabor Publications & Events

Tabor Communications