Amazon Climbs Into the HPC Arena

By Michael Feldman

July 14, 2010

Amazon’s cloud platform got a high performance boost this week with the announcement of its Cluster Compute Instances (CCI). CCI specifically targets HPC workloads, incorporating high-end CPU horsepower and a low-latency interconnect fabric into the company’s popular EC2 computing on-demand offering. The new capability welcomes HPC into the most well-recognized public cloud in the world.

In a nutshell, the new offering is based on a new EC2 instance under the CCI category: the Cluster Compute Quadruple Extra Large Instance, which, for the sake of brevity, I’m going to refer to as the HPC instance. It is defined as of a dual-socket Intel Xeon X5570 (2.93 GHz, quad-core) server or virtual server with 23 GB of memory, and 1,690 GB of external storage. Servers are connected via a 10 Gigabit Ethernet network. The HPC instance is the ninth EC2 instance type offered by Amazon and the only one that actually spells out the specific CPU and I/O fabric being employed. For the other eight instances, you are provided a generic notion of capability based on a specified number of EC2 compute units and a general metric for network I/O performance (moderate or high).

For users of the HPC instance, the default cluster size (aka the instance limit) is eight servers, providing 64 cores. That’s probably the sweet spot for the type of customer Amazon is going after — presumably middle-range HPC users with moderately scalable applications. But, as in any computing on-demand offering worthy of that title, capacity can be extended dynamically.

“An instance limit is only an initial limit and can be easily removed by sending us an email, just like any other Amazon EC2 instance,” said Deepak Singh, business development manager for Amazon Web Services (AWS), in an email to HPCwire. “Customers can provision instances in minutes and shut them down and restart as they need in a truly scalable and elastic environment.” The exact extent of this elasticity is somewhat of a mystery though. And at this point, Amazon is not revealing how big a cluster can be devoted to a single customer.

It’s worth noting that Amazon has run Linpack on 880 of their HPC-style servers, reporting a performance result of 41.82 teraflops. That’s well into TOP500 territory (equivalent to the 146 slot on the June 2010 list). It’s also worth noting that, according to Intel, the peak performance on the Xeon X5570 CPU is 46.88 gigaflops, which means the Linpack efficiency for the EC2 cluster is just a shade over 50 percent. That’s pretty much on par with vanilla GigE clusters, although the best 10 GbE cluster can hit 84 percent Linpack efficiency and most InfiniBand-based systems will be in the 70 to 92 percent range.

Customers won’t care about unimpressive Linpack yields, but it may remind potential users that even the new HPC instance may behave less like a supercomputer than they might be expecting. Amazon has provided few details about the 10 GbE setup or how the Hardware Virtual Machine (HVM) virtualization scheme being employed might impact performance. And since there are no performance metrics publicly available for real applications, it’s too early to tell how traditional MPI codes will fare. To its credit, Amazon is being careful not to make claims it can’t demonstrate.

“During our private beta period, customers ran a variety of MPI codes, including MATLAB, in-house computational fluid dynamics software for aircraft and automobile design, and molecular dynamics codes for protein simulation like NAMD,” said Singh. “Our partners and AWS used standard benchmark packages like HPCC and IMB. Now that the service is available to the broad public, we expect an increased variety in the types of applications our customers will be running.”

The Magellan Cloud research team at the National Energy Research Scientific Computing Center (NERSC) was one of those beta customers and got a chance to test drive the new EC2 offering prior to this week’s official launch. They reported that a series of HPC application benchmarks “ran 8.5 times faster on Cluster Compute Instances for Amazon EC2 than the previous EC2 instance types.” But considering the lesser CPUs and GigE configurations on the non-HPC instances, that may end up being faint praise.

EC2 has surely left some room at the high end for more performant on-demand platforms and for customers that require a greater level of HPC expertise than Amazon can muster. Experienced HPC vendors like IBM, SGI, Penguin Computing, and others are already staking out this territory. While those vendors may be gratified that a company like Amazon thinks the HPC on-demand model is ready for prime time, those same companies will now have to prove their offerings are better than Amazon’s.

Penguin Computing seems more than willing to make that case. From CEO Charles Wuischpard’s point of view, his company’s one-year old Penguin On-Demand (POD) HPC rental service has some clear differentiation with Amazon’s new HPC offering. At the hardware level, POD offers more memory per core than EC2, InfiniBand connectivity, a GPU acceleration option, and Panasas-based parallel file storage.

But the big differentiator, according to Wuischpard, is the level of engineering support they’re able to provide. Every POD deal comes with its own HPC engineer, who makes sure the whole software stack — cluster management, network drivers, compilers, and so on — is configured correctly for the end-user applications. “The customers we have today are truly not computer scientists and we help them through the whole process,” said Wuischpard.

Unit pricing is somewhat comparable. POD charges $0.25 per core hour for compute time, while Amazon offers one HPC instance (two quad-core CPUs) for 1.60 per hour. Both provide cost incentives for longer time commitments. But overall, Wuischpard thinks POD will offer better value than Amazon. It should be remembered that wall clock time is the key metric here. If an on-demand platform can run a given application twice as fast as their competitor, they’ve effectively cut their per unit cost in half. “As long as we’re less expensive overall, I’m pretty comfortable with where we are,” said Wuischpard.

For a broader perspective of Amazon’s HPC launch, see Amazon Adds HPC Capability to EC2 and related coverage at HPC in the Cloud.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from the nanoscale to the astronomic, from calculating quantum effe Read more…

By Ken Strandberg

What Will IBM’s AI Debater Learn from Its Loss?

February 14, 2019

The utility of IBM’s latest man-versus-machine gambit is debatable. At the very least its Project Debater got us thinking about the potential uses of artificial intelligence as a way of helping humans sift through al Read more…

By George Leopold

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst of bankruptcy proceedings. According to Dutch news site Drimb Read more…

By Tiffany Trader

HPE Extreme Performance Solutions

HPE Systems With Intel Omni-Path: Architected for Value and Accessible High-Performance Computing

Today’s high-performance computing (HPC) and artificial intelligence (AI) users value high performing clusters. And the higher the performance that their system can deliver, the better. Read more…

IBM Accelerated Insights

Medical Research Powered by Data

“We’re all the same, but we’re unique as well. In that uniqueness lies all of the answers….”

  • Mark Tykocinski, MD, Provost, Executive Vice President for Academic Affairs, Thomas Jefferson University

Getting the answers to what causes some people to develop diseases and not others is driving the groundbreaking medical research being conducted by the Computational Medicine Center at Thomas Jefferson University in Philadelphia. Read more…

South African Weather Service Doubles Compute and Triples Storage Capacity of Cray System

February 13, 2019

South Africa has made headlines in recent years for its commitment to HPC leadership in Africa – and now, Cray has announced another major South African HPC expansion. Cray has been awarded contracts with Eclipse Holdings Ltd. to upgrade the supercomputing system operated by the South African Weather Service (SAWS). Read more…

By Oliver Peckham

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from th Read more…

By Ken Strandberg

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

UC Berkeley Paper Heralds Rise of Serverless Computing in the Cloud – Do You Agree?

February 13, 2019

Almost exactly ten years to the day from publishing of their widely-read, seminal paper on cloud computing, UC Berkeley researchers have issued another ambitious examination of cloud computing - Cloud Programming Simplified: A Berkeley View on Serverless Computing. The new work heralds the rise of ‘serverless computing’ as the next dominant phase of cloud computing. Read more…

By John Russell

Iowa ‘Grows Its Own’ to Fill the HPC Workforce Pipeline

February 13, 2019

The global workforce that supports advanced computing, scientific software and high-speed research networks is relatively small when you stop to consider the magnitude of the transformative discoveries it empowers. Technical conferences provide a forum where specialists convene to learn about the latest innovations and schedule face-time with colleagues from other institutions. Read more…

By Elizabeth Leake, STEM-Trek

Trump Signs Executive Order Launching U.S. AI Initiative

February 11, 2019

U.S. President Donald Trump issued an Executive Order (EO) today launching a U.S Artificial Intelligence Initiative. The new initiative - Maintaining American L Read more…

By John Russell

Celebrating Women in Science: Meet Four Women Leading the Way in HPC

February 11, 2019

One only needs to look around at virtually any CS/tech conference to realize that women are underrepresented, and that holds true of HPC. SC hosts over 13,000 H Read more…

By AJ Lauer

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Assessing Government Shutdown’s Impact on HPC

February 6, 2019

After a 35-day federal government shutdown, the longest in U.S. history, government agencies are taking stock of the damage -- and girding for a potential secon Read more…

By Tiffany Trader

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This