Amazon EC2 Versus the Competition

By Tiffany Trader

November 6, 2012

Independent cloud computing analyst Cloud Spectator released a report today comparing Amazon EC2 cloud servers to that of nine other IaaS providers with regard to performance and stability.

Cloud Spectator’s CloudSpecs Performance System uses industry-standard open-source benchmarks to measure virtual server performance. The Cloud Performance and Stability Report examined four aspects of the EC2 cloud: CPU, RAM, disk, and internal network (VPN connections).

Over a consecutive 30-day period, the CloudSpecs System performed a series of 19 tests on servers located within the Amazon EC2 East (1a) datacenter: 3 server system tests, 5 CPU tests, 2 disk tests, 2 RAM tests, and 7 VPN network tests.

Test Results (compared with industry average)

 Metric Performance Stability
 CPU  2% above average  15% more stable than average
 RAM  36% above average 10% more stable than average 
 Disk  68% below average  91% more stable than average
VPN Connection  49% below average   5% less stable than average

Details from Cloud Spectator report

The performance of Amazon’s virtual CPUs was on par with the IaaS industry standard, while RAM performance came in above average. VPN and disk performance, on the other hand, were below average.

As for stability, disk performance did very well, with virtual CPU and RAM coming in slightly above average. VPN was again lacking.

Overall, CPU and RAM were winners with above average performance and decent stability. On the flip side is Amazon’s VPN; performance and stability were below average. The report authors say that this could help “explain performance issues with more complex, multi-server applications.”

“There has been a lot of talk among the industry about Amazon EC2 and its stable performance,” noted Kenny Li, founder and CEO at Cloud Spectator. “Our analysis validates that indeed there are inconsistencies in service performance. But more than that, we provide detailed data that shows the highs and lows of the service. This level of research will help not only Amazon, but companies who are evaluating Amazon. Inconsistency doesn’t have to be bad, it depends on what you are using the service to accomplish.”

Although it did so anonymously, Cloud Spectator signed up for an Amazon account using the same process as a typical AWS EC2 client. The CloudSpecs system automatically runs tests 4 times a day, 365 days a year, and this particular report was based on a 30-day consecutive period period from August 18, 2012, to September 16, 2012.

All tests were run on Amazon’s EC2 Extra Large server, with the VPN connection test also using the EC2 Medium instance as a dummy server. Server benchmarks include the byte-Unixbench (http://code.google.com/p/byte-unixbench/) and Phoronix Test Suite (http://www.phoronix-test-suite.com/). The complete list of benchmarks and the significance of each is detailed in the full report.

To compile the industry averages, Cloud Spectator measured the performance of nine cloud IaaS providers over 30 days, using the same benchmark tests. Stability metrics were obtained by “calculating the standard deviation of the performance results of the CloudSpecs tests over a period of 30 days, also compared to an industry average.”

In a separate report, also released today, Cloud Spectator compares the pricing structure of various IaaS offerings. The analysis is based on data from 20 cloud providers, including Amazon, Microsoft Azure, SoftLayer, PEER1 Zunicore, Rackspace, CloudSigma, GoGrid and others.

These reports come on the heels of another AWS outage. Although Amazon held up pretty well against Super Storm/Hurricane Sandy, the service suffered some downtime the week before. Github, Pinterest, Reddit and Netflix were among the high-profile companies that were affected.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Nvidia Debuts Turing Architecture, Focusing on Real-Time Ray Tracing

August 16, 2018

From the SIGGRAPH professional graphics conference in Vancouver this week, Nvidia CEO Jensen Huang unveiled Turing, the company's next-gen GPU platform that introduces new RT Cores to accelerate ray tracing and new Tenso Read more…

By Tiffany Trader

HPC Coding: The Power of L(o)osing Control

August 16, 2018

Exascale roadmaps, exascale projects and exascale lobbyists ask, on-again-off-again, for a fundamental rewrite of major code building blocks. Otherwise, so they claim, codes will not scale up. Naturally, some exascale pr Read more…

By Tobias Weinzierl

STAQ(ing) the Quantum Computing Deck

August 16, 2018

Quantum computers – at least for now – remain noisy. That’s another way of saying unreliable and in diverse ways that often depend on the specific quantum technology used. One idea is to mitigate noisiness and perh Read more…

By John Russell

HPE Extreme Performance Solutions

Introducing the First Integrated System Management Software for HPC Clusters from HPE

How do you manage your complex, growing cluster environments? Answer that big challenge with the new HPC cluster management solution: HPE Performance Cluster Manager. Read more…

IBM Accelerated Insights

Super Problem Solving

You might think that tackling the world’s toughest problems is a job only for superheroes, but at special places such as the Oak Ridge National Laboratory, supercomputers are the real heroes. Read more…

NREL ‘Eagle’ Supercomputer to Advance Energy Tech R&D

August 14, 2018

The U.S. Department of Energy (DOE) National Renewable Energy Laboratory (NREL) has contracted with Hewlett Packard Enterprise (HPE) for a new 8-petaflops (peak) supercomputer that will be used to advance early-stage R&a Read more…

By Tiffany Trader

STAQ(ing) the Quantum Computing Deck

August 16, 2018

Quantum computers – at least for now – remain noisy. That’s another way of saying unreliable and in diverse ways that often depend on the specific quantum Read more…

By John Russell

NREL ‘Eagle’ Supercomputer to Advance Energy Tech R&D

August 14, 2018

The U.S. Department of Energy (DOE) National Renewable Energy Laboratory (NREL) has contracted with Hewlett Packard Enterprise (HPE) for a new 8-petaflops (peak Read more…

By Tiffany Trader

CERN Project Sees Orders-of-Magnitude Speedup with AI Approach

August 14, 2018

An award-winning effort at CERN has demonstrated potential to significantly change how the physics based modeling and simulation communities view machine learni Read more…

By Rob Farber

Intel Announces Cooper Lake, Advances AI Strategy

August 9, 2018

Intel's chief datacenter exec Navin Shenoy kicked off the company's Data-Centric Innovation Summit Wednesday, the day-long program devoted to Intel's datacenter Read more…

By Tiffany Trader

SLATE Update: Making Math Libraries Exascale-ready

August 9, 2018

Practically-speaking, achieving exascale computing requires enabling HPC software to effectively use accelerators – mostly GPUs at present – and that remain Read more…

By John Russell

Summertime in Washington: Some Unexpected Advanced Computing News

August 8, 2018

Summertime in Washington DC is known for its heat and humidity. That is why most people get away to either the mountains or the seashore and things slow down. H Read more…

By Alex R. Larzelere

NSF Invests $15 Million in Quantum STAQ

August 7, 2018

Quantum computing development is in full ascent as global backers aim to transcend the limitations of classical computing by leveraging the magical-seeming prop Read more…

By Tiffany Trader

By the Numbers: Cray Would Like Exascale to Be the Icing on the Cake

August 1, 2018

On its earnings call held for investors yesterday, Cray gave an accounting for its latest quarterly financials, offered future guidance and provided an update o Read more…

By Tiffany Trader

Leading Solution Providers

SC17 Booth Video Tours Playlist

Altair @ SC17

Altair

AMD @ SC17

AMD

ASRock Rack @ SC17

ASRock Rack

CEJN @ SC17

CEJN

DDN Storage @ SC17

DDN Storage

Huawei @ SC17

Huawei

IBM @ SC17

IBM

IBM Power Systems @ SC17

IBM Power Systems

Intel @ SC17

Intel

Lenovo @ SC17

Lenovo

Mellanox Technologies @ SC17

Mellanox Technologies

Microsoft @ SC17

Microsoft

Penguin Computing @ SC17

Penguin Computing

Pure Storage @ SC17

Pure Storage

Supericro @ SC17

Supericro

Tyan @ SC17

Tyan

Univa @ SC17

Univa

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This