Bad Moon Rises Over Cloud Perceptions

By Nicole Hemsoth

April 28, 2011

Let’s pretend for a moment that you are the owner or technical lead on a web application that recently captured the public’s attention and swelled in popularity—to the point that serving those visitors or customers without building a new data center would be impossible.

Instead of resorting to that up-front investment, what you need, of course, is infrastructure that will scale with your traffic or transactions—a resource that will allow you to avoid the cost and expertise required to maintain a system robust enough to handle the flood. What you need, in other words, is a cloud-based infrastructure provider—preferably one that offers an attractive guarantee of uptime and continual reassurance that no matter what, that data is backed up and replicated to death in the event of disaster.

So, there you have it; you’re all set to serve users and wash your hands of the whole infrastructure problem. In one relatively swift move you’ve shed the need for expensive, cumbersome hardware and can roll ahead with your business.

What could be more convenient? The resources simply scale along with demand, you pay for that demand as it happens, and outside of your maintenance of the core business applications, you can sit back and relax.

Right?

According to CTO and founder of the cloud management firm Rightscale, Thorsten von Eicken, a former colleague of AWS chief Werner Vogels,  this assumption is part of what sparked some of the trouble for those who had their business lifeblood in the cloud. Those guarantees and assurances backing any cloud computing outlet were as good as gold, weren’t they?

Following the initial Amazon Web Services outage, von Eicken wrote that although many customers were able to resort to a solid “Plan B” in the case of an outage, some were not adequately prepared for such an event. He claims that “because of Amazon’s reliability has been incredible, many users were not well-prepared leading to widespread outages. Additionally, some users got caught by unforeseen failure modes rendering their failure plans ineffective.”

So is this to say that if you experienced extensive, damaging outages and data loss the burden ultimately falls on you due to lack of disaster recovery plans? Not necessarily. However, what Von Eicken and other notable experts on the cloud movement suggest is that there might have been some over-confidence in Amazon’s ability to take care of everything. That aside, one has to wonder if even AWS as a whole had been a little too confident.

Now let’s go back to the scenario with you at helm from before. You’ve prepared yourself in any way you thought was necessary or appropriate given Amazon’s very solid track record of performance and uptime, the extensive service level agreements (SLAs) and their several success stories of mission-critical cloud operations.

Nonetheless, you woke up this morning (if you were lucky enough to sleep following the outage) to the following message that Amazon sent out to some of its customers today:

Hello,
 
A few days ago we sent you an email letting you know that we were working on recovering an inconsistent data snapshot of one or more of your Amazon EBS volumes.  We are very sorry, but ultimately our efforts to manually recover your volume were unsuccessful.  The hardware failed in such a way that we could not forensically restore the data.
 
What we were able to recover has been made available via a snapshot, although the data is in such a state that it may have little to no utility…
 
If you have no need for this snapshot, please delete it to avoid incurring storage charges.
 
We apologize for this volume loss and any impact to your business.

Sincerely,

Amazon Web Services, EBS Support

This message was produced and distributed by Amazon Web Services LLC, 410 Terry Avenue North, Seattle, Washington 98109-5210

You read this a few times. It doesn’t really sink in at first since after all, wasn’t everything protected under, like 50 layers of different protection and duplication efforts on Amazon’s side?

You see key phrases and nothing else… “failed in such a way that we could not forensically restore the data…”

“..data is in such a state that it may have little to no utlilty…”

And your favorite line of all after you’ve had a few moments to really think about it:

 “If you have no need for this snapshot, please delete it to avoid incurring storage charges…”

At these lines, you cock a brow over your left eye, which started twitching occasionally a few days ago in the wake of the outage and now appears to be possessed.

Seriously?… This apologetic letter about your complete and total loss of my data and you’re warning me that I am going to be incurring storage charges?

(insert select profanities here)

While certainly Amazon could have massaged this message, lending a spoonful of sugar for such medicine, this is not the only way that communication has played a significant role in the increasingly bad press the normally stable infrastructure provider is receiving today.

As von Eicken noted of the initial outage, Amazon receives an “F” for its ability to effectively communicate with users throughout the first signs of trouble. Many are now claiming that failure extends to the data loss matter at hand now. At a time like this, however, good communication is needed more than ever—it seems that either they have no idea of the extent and cause of the loss or they are afraid to let people know how bad it is and how far it extends. Either way, this does not bode well for the company as it has opened the door to a bumrush of negative speculation.

The problem is, in Amazon’s defense, this is the first major problem with multiple, compounding failures that it has ever experienced. There have been latencies and delays in select zones in the past but nothing on this level—not even close to it.

In addition to opening the door for widespread criticism and speculation, it has also allowed competitors (not to mention the six or more backup/recovery companies that are rapid-firing press, barely able to contain their excitement over this outage and loss) to claim dominance—to state that they are immune from such disasters.

But we all know, no one is immune. If Amazon suffers this type of problem, Rackspace could suffer the same issue. If Rackspace trips momentarily, so could Microsoft’s services.

And the point that no one brings up here is that this same problem—and worse—could happen in your very own data center if you chose not to hop on the cloud bandwagon. And it could be far more destructive and expensive.

We’ll spare Amazon for a moment and say that this was written in haste. After all, they have been raked over the coals since news of sporadic data destruction broke…and broke in a very public way.

Just as cloud computing hit the mainstream media outlets in a big way over the past year, so too did news of the problems that could arise when you push your core business into the ether.

The announcement today that data was not only lost or temporarily unavailable—that instead it was actually destroyed—certainly doesn’t bode well for the future of mission critical applications being exclusively hosted on cloud computing infrastructure. It is unfortunate for IaaS providers that this should happen right at that much-anticipated golden moment of growing comfort with cloud computing, but perhaps we should consider this event in light of a few points that major media outlets aren’t talking about.

By the way, the media that I’ve encountered this morning paints the picture of what happened with the Amazon cloud in some rather black and white terms. Mainstream outlets necessarily be condemned for this however—after all, it’s not easy to come up with live news that is approachable for the folks that just a few months ago learned that clouds were more than just the puff upstairs while still painting a picture of the outage that is technically dense and thus in better context.

With that said, the media really doesn’t have a leg to stand (nor does anyone else at this point) when Amazon has been (notoriously) uncommunicative about what actually happened. Aside from sporadic updates following the initial outage and some updates that were dense but not necessarily revealing, the public, not to mention the users whose data may have been chewed up, are in the dark.

If an infrastructure provider of any size has a problem like this, the first item on the agenda should be communication. This not only protects them from wild speculation across media outlets, but it also protects the very notion of the cloud as a reasonable solution for everyone—no matter who they choose to rent hardware from.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Hedge Funds (with Supercomputing help) Rank First Among Investors

May 22, 2017

In case you didn’t know, The Quants Run Wall Street Now, or so says a headline in today’s Wall Street Journal. Quant-run hedge funds now control the largest Read more…

By John Russell

IBM, D-Wave Report Quantum Computing Advances

May 18, 2017

IBM said this week it has built and tested a pair of quantum computing processors, including a prototype of a commercial version. That progress follows an an Read more…

By George Leopold

PRACEdays 2017 Wraps Up in Barcelona

May 18, 2017

Barcelona has been absolutely lovely; the weather, the food, the people. I am, sadly, finishing my last day at PRACEdays 2017 with two sessions: an in-depth loo Read more…

By Kim McMahon

HPE Extreme Performance Solutions

Exploring the Three Models of Remote Visualization

The explosion of data and advancement of digital technologies are dramatically changing the way many companies do business. With the help of high performance computing (HPC) solutions and data analytics platforms, manufacturers are developing products faster, healthcare providers are improving patient care, and energy companies are improving planning, exploration, and production. Read more…

US, Europe, Japan Deepen Research Computing Partnership

May 18, 2017

On May 17, 2017, a ceremony was held during the PRACEdays 2017 conference in Barcelona to announce the memorandum of understanding (MOU) between PRACE in Europe Read more…

By Tiffany Trader

NSF, IARPA, and SRC Push into “Semiconductor Synthetic Biology” Computing

May 18, 2017

Research into how biological systems might be fashioned into computational technology has a long history with various DNA-based computing approaches explored. N Read more…

By John Russell

DOE’s HPC4Mfg Leads to Paper Manufacturing Improvement

May 17, 2017

Papermaking ranks third behind only petroleum refining and chemical production in terms of energy consumption. Recently, simulations made possible by the U.S. D Read more…

By John Russell

PRACEdays 2017: The start of a beautiful week in Barcelona

May 17, 2017

Touching down in Barcelona on Saturday afternoon, it was warm, sunny, and oh so Spanish. I was greeted at my hotel with a glass of Cava to sip and treated to a Read more…

By Kim McMahon

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Cray Offers Supercomputing as a Service, Targets Biotechs First

May 16, 2017

Leading supercomputer vendor Cray and datacenter/cloud provider the Markley Group today announced plans to jointly deliver supercomputing as a service. The init Read more…

By John Russell

HPE’s Memory-centric The Machine Coming into View, Opens ARMs to 3rd-party Developers

May 16, 2017

Announced three years ago, HPE’s The Machine is said to be the largest R&D program in the venerable company’s history, one that could be progressing tow Read more…

By Doug Black

What’s Up with Hyperion as It Transitions From IDC?

May 15, 2017

If you’re wondering what’s happening with Hyperion Research – formerly the IDC HPC group – apparently you are not alone, says Steve Conway, now senior V Read more…

By John Russell

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

HPE Launches Servers, Services, and Collaboration at GTC

May 10, 2017

Hewlett Packard Enterprise (HPE) today launched a new liquid cooled GPU-driven Apollo platform based on SGI ICE architecture, a new collaboration with NVIDIA, a Read more…

By John Russell

IBM PowerAI Tools Aim to Ease Deep Learning Data Prep, Shorten Training 

May 10, 2017

A new set of GPU-powered AI software announced by IBM today brings automation to many of the tedious, time consuming and complex aspects of AI project on-rampin Read more…

By Doug Black

Bright Computing 8.0 Adds Azure, Expands Machine Learning Support

May 9, 2017

Bright Computing, long a prominent provider of cluster management tools for HPC, today released version 8.0 of Bright Cluster Manager and Bright OpenStack. The Read more…

By John Russell

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Google Pulls Back the Covers on Its First Machine Learning Chip

April 6, 2017

This week Google released a report detailing the design and performance characteristics of the Tensor Processing Unit (TPU), its custom ASIC for the inference Read more…

By Tiffany Trader

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Last week, Google reported that its custom ASIC Tensor Processing Unit (TPU) was 15-30x faster for inferencing workloads than Nvidia's K80 GPU (see our coverage Read more…

By Tiffany Trader

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

Since our first formal product releases of OSPRay and OpenSWR libraries in 2016, CPU-based Software Defined Visualization (SDVis) has achieved wide-spread adopt Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a ne Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Leading Solution Providers

Facebook Open Sources Caffe2; Nvidia, Intel Rush to Optimize

April 18, 2017

From its F8 developer conference in San Jose, Calif., today, Facebook announced Caffe2, a new open-source, cross-platform framework for deep learning. Caffe2 is Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which w Read more…

By Tiffany Trader

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling Read more…

By Steve Campbell

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Eng Read more…

By Tiffany Trader

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular Read more…

By John Russell

US Supercomputing Leaders Tackle the China Question

March 15, 2017

As China continues to prove its supercomputing mettle via the Top500 list and the forward march of its ambitious plans to stand up an exascale machine by 2020, Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu's Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural networ Read more…

By Tiffany Trader

DOE Supercomputer Achieves Record 45-Qubit Quantum Simulation

April 13, 2017

In order to simulate larger and larger quantum systems and usher in an age of "quantum supremacy," researchers are stretching the limits of today's most advance Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This