Visit additional Tabor Communication Publications
October 05, 2007
The combination of quad-core Opterons and DDR Infiniband is re-landscaping the HPC terrain and is propelling the largest clusters to the top of the high performance heap. A rash of recent announcements of big system purchases suggests good times ahead for HPC cluster vendors.
Or does it? Let's look at the good news first.
In the space of five days, over 500 teraflops of systems were announced -- all, coincidentally, for DOE labs. Last Friday, the Pacific Northwest National Laboratory (PNNL) revealed it had purchased a 163 teraflop HP Proliant server cluster. Then on Tuesday, Appro announced it had been awarded a contract to deliver 438 teraflops of computing hardware to three other DOE labs -- Lawrence Livermore, Los Alamos and Sandia. Add this to the 500 teraflop Ranger system currently being installed at the Texas Advanced Computing Center (TACC) and you have over a petaflop computing power -- all based on the new quad-core Opterons and DDR InfiniBand technology.
The Appro contract will result in eight new Linux clusters for the National Nuclear Security Administration (NNSA) Advanced Simulation and Computing (ASC) program, which supports the NNSA's mission to maintain the country's nuclear weapons. Three clusters are headed to Lawrence Livermore, two to Los Alamos and the remaining three to Sandia. Of these, the largest is a 162 teraflop machine for Livermore. The contract also contains an option for perhaps three additional clusters, representing 200 additional teraflops.
Prior to this, Appro had a presence at all ASC DOE labs, but nothing on this scale at Los Alamos and Sandia. The company's large three-cluster Peloton deployment at Livermore in 2006 was just a prelude to this week's ASC award. It's a huge win for Appro.
The contract also represents a new approach to purchasing computing resources by the ASC program, which chose to make a single large purchase for its capacity computing requirements for the next two years, instead of six smaller purchases over the same time period. Our feature article this week delves more deeply into the new clusters and the strategy behind the unified procurement for the three labs.
HP's 163 petaflop cluster, headed to PNNL in Richland, Washington, is just one of two 100-plus teraflop systems that the company will be delivering over the next year. The PNNL system will be used to drive the lab's molecular science research in applications such as aerosol formation, bioremediation, catalysis, climate change and hydrogen storage. The machine will be deployed in two phases and won't be fully operational until September 2008. The other system, a 182 teraflop blade-based cluster for an unnamed Swedish government agency, is expected to be fully operational as early as next month. That could give HP a more respectable presence in the top 10 of the Top500 list. The current top-ranked HP system is the 20.5 teraflop ASCI Q machine at number 62.
The dynamic duo of multicore x86 processors and InfiniBand should give clusters a real boost over the next few years. At least in raw performance, the top commodity-based clusters are now on par with the fastest high-end supercomputers from Cray and IBM. Until the petaflop-class XT and Blue Gene/P systems come online, the capability machines and the scaled-out clusters will vie for peak performance leadership. And while these clusters may not provide the same level of sustained application performance as their more expensive supercomputing brethren, their price-performance is about twice as good.
But the sub-100 teraflop cluster market is where the real market action is, and these systems are just getting less expensive and more popular every day. This makes them an increasingly attractive proposition for a wide range of users, which in turn makes them an increasingly attractive proposition for a wide range of vendors.
In a sense, that's the bad news. The promise of a fast growing market has attracted an abundance of companies. In general, the HPC cluster market is now over-served. There are a lot of cluster vendors out there without much fundamental product differentiation among them. Standard 64-bit x86 processors and InfiniBand interconnect technology have become the basis for hundreds of cluster computing offerings. These two elements -- compute and communications -- dominate how well a cluster performs.
Companies that have broken out of this mold, like SiCortex and (to a certain extent) Liquid Computing, have come up with truly innovative solutions that are unlike the traditional model. The challenge for them is gaining a foothold in a market that is risk-averse and extremely price sensitive. These companies have chosen a tough path, but at least they avoid direct competition with the 50 other system vendors selling what are essentially the same boxes.
To be fair, for the more mainstream cluster offerings, product differentiation does exist in technology, customization options, configuration, support, service, software bundling, storage offerings, etc., and these elements are all being applied for competitive advantage. But sometimes these advantages are only temporary. Rackable created a unique line of x86-based systems, with innovative power and cooling technology at competitive price points. Then everyone jumped onto the power/cooling bandwagon, dulling the benefit of Rackable's offerings and making them easy prey to aggressive pricing by Dell and HP. More recently, Rackable is fighting back with a mobile datacenter called Concentro, designed to go up against Sun Microsystems' new Blackbox offering. The lesson here is that vendors copy good ideas from each other to the extent possible, which, over time, marginalizes competitive advantages.
Since the HPC market -- and more specifically, the HPC cluster market -- is growing at double-digit rates, there has been little pressure for consolidation. Rapid growth is not conductive to the "normal" maturation process of eliminating weaker competitors and products. But I suspect the next sustained economic contraction will change that. With signs of inflation and rising interest rates starting to appear, an economic slowdown, if not full-blown recession, could happen next year.
Even in the absence of a deteriorating economy, there is constant pressure on buyers to look for purchasing efficiencies. The ASC program procurement strategy, mentioned above, is one such example. It could provide a model for how large organizations will buy high performance computing in the future. In the ASC case, the labs were specifically not looking for innovative computing. They already bought that in their high-end supercomputer systems, typified by Lawrence Livermore's Blue Gene/L. What they were looking for was a standard, replicable architecture for capacity computing that they could buy in bulk. But by replacing a number of smaller purchases with one large one, they explicitly limited the number of vendors in the ASC program.
Overall, the larger tier 1 system vendors seem to be least at risk, since they can offer more complete solutions, better support and a variety of services. They sell the same product lines into a broad customer base in both HPC and traditional enterprise computing. On the other hand, they are often less able to compete on price. The tier 3 shops fill a need for users looking for a one-night stand in high performance computing and can offer rock bottom prices. Tier 2 vendors provide an attractive middle ground, as was the case for the ASC procurement.
Whether market forces thin the herd quickly, through a recession, or more slowly, via a steady maturation of the ecosystem, depends upon future economic conditions. Over the long term, the larger vendors are going to be the most resilient. The smaller companies with narrower customer bases are already looking over their shoulder. Whatever the situation, if you're an x86 system vendor, hang on to your clusters. It's going to be a bumpy ride.
As always, comments about HPCwire are welcomed and encouraged. Write to me, Michael Feldman, at firstname.lastname@example.org.
Posted by Michael Feldman - October 04, 2007 @ 9:00 PM, Pacific Daylight Time
Michael Feldman is the editor of HPCwire.
No Recent Blog Comments
Contributing commentator, Andrew Jones, offers a break in the news cycle with an assessment of what the national "size matters" contest means for the U.S. and other nations...
Today at the International Supercomputing Conference in Leipzing, Germany, Jack Dongarra presented on a proposed benchmark that could carry a bit more weight than its older Linpack companion. The high performance conjugate gradient (HPCG) concept takes into account new architectures for new applications, while shedding the floating point....
Not content to let the Tianhe-2 announcement ride alone, Intel rolled out a series of announcements around its Knights Corner and Xeon Phi products--all of which are aimed at adding some options and variety for a wider base of potential users across the HPC spectrum. Today at the International Supercomputing Conference, the company's Raj....
Jun 19, 2013 |
Supercomputer architectures have evolved considerably over the last 20 years, particularly in the number of processors that are linked together. One aspect of HPC architecture that hasn't changed is the MPI programming model.
Jun 18, 2013 |
The world's largest supercomputers, like Tianhe-2, are great at traditional, compute-intensive HPC workloads, such as simulating atomic decay or modeling tornados. But data-intensive applications--such as mining big data sets for connections--is a different sort of workload, and runs best on a different sort of computer.
Jun 18, 2013 |
Researchers are finding innovative uses for Gordon, the 285 teraflop supercomputer housed at the San Diego Supercomputer Center (SDSC) that has a unique Flash-based storage system. Since going online, researchers have put the incredibly fast I/O to use on a wide variety of workloads, ranging from chemistry to political science.
Jun 17, 2013 |
The advent of low-power mobile processors and cloud delivery models is changing the economics of computing. But just as an economy car is good at different things than a full size truck, an HPC workload still has certain computing demands that neither the fastest smartphone nor the most elastic cloud cluster can fulfill.
Jun 14, 2013 |
For all the progress we've made in IT over the last 50 years, there's one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?
Join our webinar to learn how IT managers can migrate to a more resilient, flexible and scalable solution that grows with the data center. Mellanox VMS is future-proof, efficient and brings significant CAPEX and OPEX savings. The VMS is available today.