Since 1986 - Covering the Fastest Computers in the World and the People Who Run Them

Language Flags
January 31, 2011

On Realistic Expectations for HPC Clouds

Nicole Hemsoth

Here is probably one of the simplest true or false games we could play—evaluate the following for its validity:

With HPC in the cloud, there are no problems, it is a silver bullet—Customers should expect that “Acquisition, installation, etc. are all handled. Their code will just run without issue. And it will be much faster than their internal systems—running the latest processors, the fastest Infiniband, the most memory…”

If you’ve ever managed an HPC job in the cloud, it would be interesting to hear how far into the paragraph you got before you rang the “false” buzzer…

Joe Landman of Scalable Informatics, a Michigan-based HPC resource provider, weighed in on his experiences with customers, noting that sometimes the expectations can outweigh what is possible. 

Landman took issue with the fact that HPC in the cloud is not always easy and that customer expectations in terms of hardware, ease of use, and fulfillment of true “on-demand” access cannot always be met.

Landman was surprisingly frank about customer dissatisfaction and the reasons behind the frustration, which is often caused by the fact that there is a mismatch between cheap and on-demand access and what could be possible on a physical, in-house cluster.

Having dealt with multiple customers, “HPC in the cloud isn’t nearly as easy as anyone would like.” Further complicating this sense that it should be a simple solution is the more general disconnect between the on-demand era that promises super-cheap resources at the click of a button—and the reality that service providers delivering these resources face.

Furthermore, expectations about the hardware environment tend to be unrealistic for some users. As “The day Intel released Nehalem X5690s, they will not magically appear in all/most/some HPC cloud infrastructure. We see HPC infrastructure with 2 and 3 year-old chips, RAM and Infiniband.”

Landman notes that when it comes to a lack of ease in using HPC clouds, “Code is implicated in some of this, as are workflow processes” but furthermore, that expectations about how HPC clouds should work are off center.

As Landman writes, “For the economics of HPC in the cloud to work, you need to be or below the price performance knee for hardware, where you can minimize the system cost while maximizing the cycles used. You may be able to charge more for faster cycles, but in general, most people don’t want to pay for much faster.”

We don’t usually identify a quote of the day here at HPC in the Cloud, but if we did, Landman’s statement below would certainly qualify:

“Cycles are cycles, some are more efficient and faster than others. There is a market for this, but customers are rarely willing to pay more for the faster cycles. This is fallout from Moore’s Law—they expect it to drop in price over time…Unfortunately, the people who implement these services need to amortize the costs across many runs on hardware that will age 3+ years before being replaced.”

Full story at Scalability.org

SC14 Virtual Booth Tours

AMD SC14 video AMD Virtual Booth Tour @ SC14
Click to Play Video
Cray SC14 video Cray Virtual Booth Tour @ SC14
Click to Play Video
Datasite SC14 video DataSite and RedLine @ SC14
Click to Play Video
HP SC14 video HP Virtual Booth Tour @ SC14
Click to Play Video
IBM DCS3860 and Elastic Storage @ SC14 video IBM DCS3860 and Elastic Storage @ SC14
Click to Play Video
IBM Flash Storage
@ SC14 video IBM Flash Storage @ SC14  
Click to Play Video
IBM Platform @ SC14 video IBM Platform @ SC14
Click to Play Video
IBM Power Big Data SC14 video IBM Power Big Data @ SC14
Click to Play Video
Intel SC14 video Intel Virtual Booth Tour @ SC14
Click to Play Video
Lenovo SC14 video Lenovo Virtual Booth Tour @ SC14
Click to Play Video
Mellanox SC14 video Mellanox Virtual Booth Tour @ SC14
Click to Play Video
Panasas SC14 video Panasas Virtual Booth Tour @ SC14
Click to Play Video
Quanta SC14 video Quanta Virtual Booth Tour @ SC14
Click to Play Video
Seagate SC14 video Seagate Virtual Booth Tour @ SC14
Click to Play Video
Supermicro SC14 video Supermicro Virtual Booth Tour @ SC14
Click to Play Video