Bad Moon Rises Over Cloud Perceptions

By Nicole Hemsoth

April 28, 2011

Let’s pretend for a moment that you are the owner or technical lead on a web application that recently captured the public’s attention and swelled in popularity—to the point that serving those visitors or customers without building a new data center would be impossible.

Instead of resorting to that up-front investment, what you need, of course, is infrastructure that will scale with your traffic or transactions—a resource that will allow you to avoid the cost and expertise required to maintain a system robust enough to handle the flood. What you need, in other words, is a cloud-based infrastructure provider—preferably one that offers an attractive guarantee of uptime and continual reassurance that no matter what, that data is backed up and replicated to death in the event of disaster.

So, there you have it; you’re all set to serve users and wash your hands of the whole infrastructure problem. In one relatively swift move you’ve shed the need for expensive, cumbersome hardware and can roll ahead with your business.

What could be more convenient? The resources simply scale along with demand, you pay for that demand as it happens, and outside of your maintenance of the core business applications, you can sit back and relax.

Right?

According to CTO and founder of the cloud management firm Rightscale, Thorsten von Eicken, a former colleague of AWS chief Werner Vogels,  this assumption is part of what sparked some of the trouble for those who had their business lifeblood in the cloud. Those guarantees and assurances backing any cloud computing outlet were as good as gold, weren’t they?

Following the initial Amazon Web Services outage, von Eicken wrote that although many customers were able to resort to a solid “Plan B” in the case of an outage, some were not adequately prepared for such an event. He claims that “because of Amazon’s reliability has been incredible, many users were not well-prepared leading to widespread outages. Additionally, some users got caught by unforeseen failure modes rendering their failure plans ineffective.”

So is this to say that if you experienced extensive, damaging outages and data loss the burden ultimately falls on you due to lack of disaster recovery plans? Not necessarily. However, what Von Eicken and other notable experts on the cloud movement suggest is that there might have been some over-confidence in Amazon’s ability to take care of everything. That aside, one has to wonder if even AWS as a whole had been a little too confident.

Now let’s go back to the scenario with you at helm from before. You’ve prepared yourself in any way you thought was necessary or appropriate given Amazon’s very solid track record of performance and uptime, the extensive service level agreements (SLAs) and their several success stories of mission-critical cloud operations.

Nonetheless, you woke up this morning (if you were lucky enough to sleep following the outage) to the following message that Amazon sent out to some of its customers today:

Hello,
 
A few days ago we sent you an email letting you know that we were working on recovering an inconsistent data snapshot of one or more of your Amazon EBS volumes.  We are very sorry, but ultimately our efforts to manually recover your volume were unsuccessful.  The hardware failed in such a way that we could not forensically restore the data.
 
What we were able to recover has been made available via a snapshot, although the data is in such a state that it may have little to no utility…
 
If you have no need for this snapshot, please delete it to avoid incurring storage charges.
 
We apologize for this volume loss and any impact to your business.

Sincerely,

Amazon Web Services, EBS Support

This message was produced and distributed by Amazon Web Services LLC, 410 Terry Avenue North, Seattle, Washington 98109-5210

You read this a few times. It doesn’t really sink in at first since after all, wasn’t everything protected under, like 50 layers of different protection and duplication efforts on Amazon’s side?

You see key phrases and nothing else… “failed in such a way that we could not forensically restore the data…”

“..data is in such a state that it may have little to no utlilty…”

And your favorite line of all after you’ve had a few moments to really think about it:

 “If you have no need for this snapshot, please delete it to avoid incurring storage charges…”

At these lines, you cock a brow over your left eye, which started twitching occasionally a few days ago in the wake of the outage and now appears to be possessed.

Seriously?… This apologetic letter about your complete and total loss of my data and you’re warning me that I am going to be incurring storage charges?

(insert select profanities here)

While certainly Amazon could have massaged this message, lending a spoonful of sugar for such medicine, this is not the only way that communication has played a significant role in the increasingly bad press the normally stable infrastructure provider is receiving today.

As von Eicken noted of the initial outage, Amazon receives an “F” for its ability to effectively communicate with users throughout the first signs of trouble. Many are now claiming that failure extends to the data loss matter at hand now. At a time like this, however, good communication is needed more than ever—it seems that either they have no idea of the extent and cause of the loss or they are afraid to let people know how bad it is and how far it extends. Either way, this does not bode well for the company as it has opened the door to a bumrush of negative speculation.

The problem is, in Amazon’s defense, this is the first major problem with multiple, compounding failures that it has ever experienced. There have been latencies and delays in select zones in the past but nothing on this level—not even close to it.

In addition to opening the door for widespread criticism and speculation, it has also allowed competitors (not to mention the six or more backup/recovery companies that are rapid-firing press, barely able to contain their excitement over this outage and loss) to claim dominance—to state that they are immune from such disasters.

But we all know, no one is immune. If Amazon suffers this type of problem, Rackspace could suffer the same issue. If Rackspace trips momentarily, so could Microsoft’s services.

And the point that no one brings up here is that this same problem—and worse—could happen in your very own data center if you chose not to hop on the cloud bandwagon. And it could be far more destructive and expensive.

We’ll spare Amazon for a moment and say that this was written in haste. After all, they have been raked over the coals since news of sporadic data destruction broke…and broke in a very public way.

Just as cloud computing hit the mainstream media outlets in a big way over the past year, so too did news of the problems that could arise when you push your core business into the ether.

The announcement today that data was not only lost or temporarily unavailable—that instead it was actually destroyed—certainly doesn’t bode well for the future of mission critical applications being exclusively hosted on cloud computing infrastructure. It is unfortunate for IaaS providers that this should happen right at that much-anticipated golden moment of growing comfort with cloud computing, but perhaps we should consider this event in light of a few points that major media outlets aren’t talking about.

By the way, the media that I’ve encountered this morning paints the picture of what happened with the Amazon cloud in some rather black and white terms. Mainstream outlets necessarily be condemned for this however—after all, it’s not easy to come up with live news that is approachable for the folks that just a few months ago learned that clouds were more than just the puff upstairs while still painting a picture of the outage that is technically dense and thus in better context.

With that said, the media really doesn’t have a leg to stand (nor does anyone else at this point) when Amazon has been (notoriously) uncommunicative about what actually happened. Aside from sporadic updates following the initial outage and some updates that were dense but not necessarily revealing, the public, not to mention the users whose data may have been chewed up, are in the dark.

If an infrastructure provider of any size has a problem like this, the first item on the agenda should be communication. This not only protects them from wild speculation across media outlets, but it also protects the very notion of the cloud as a reasonable solution for everyone—no matter who they choose to rent hardware from.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Microsoft Wants to Speed Quantum Development

December 12, 2017

Quantum computing continues to make headlines in what remains of 2017 as tech giants jockey to establish a pole position in the race toward commercialization of quantum. This week, Microsoft took the next step in advanci Read more…

By Tiffany Trader

ESnet Now Moving More Than 1 Petabyte/wk

December 12, 2017

Optimizing ESnet (Energy Sciences Network), the world's fastest network for science, is an ongoing process. Recently a two-year collaboration by ESnet users – the Petascale DTN Project – achieved its ambitious goal t Read more…

HPC-as-a-Service Finds Toehold in Iceland

December 11, 2017

While high-demand workloads (e.g., bitcoin mining) can overheat data center cooling capabilities, at least one data center infrastructure provider has announced an HPC-as-a-service offering that features 100 percent fre Read more…

By Doug Black

HPE Extreme Performance Solutions

Explore the Origins of Space with COSMOS and Memory-Driven Computing

From the formation of black holes to the origins of space, data is the key to unlocking the secrets of the early universe. Read more…

HPC Iron, Soft, Data, People – It Takes an Ecosystem!

December 11, 2017

Cutting edge advanced computing hardware (aka big iron) does not stand by itself. These computers are the pinnacle of a myriad of technologies that must be carefully woven together by people to create the computational c Read more…

By Alex R. Larzelere

Microsoft Wants to Speed Quantum Development

December 12, 2017

Quantum computing continues to make headlines in what remains of 2017 as tech giants jockey to establish a pole position in the race toward commercialization of Read more…

By Tiffany Trader

HPC Iron, Soft, Data, People – It Takes an Ecosystem!

December 11, 2017

Cutting edge advanced computing hardware (aka big iron) does not stand by itself. These computers are the pinnacle of a myriad of technologies that must be care Read more…

By Alex R. Larzelere

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Microsoft Spins Cycle Computing into Core Azure Product

December 5, 2017

Last August, cloud giant Microsoft acquired HPC cloud orchestration pioneer Cycle Computing. Since then the focus has been on integrating Cycle’s organization Read more…

By John Russell

GlobalFoundries, Ayar Labs Team Up to Commercialize Optical I/O

December 4, 2017

GlobalFoundries (GF) and Ayar Labs, a startup focused on using light, instead of electricity, to transfer data between chips, today announced they've entered in Read more…

By Tiffany Trader

HPE In-Memory Platform Comes to COSMOS

November 30, 2017

Hewlett Packard Enterprise is on a mission to accelerate space research. In August, it sent the first commercial-off-the-shelf HPC system into space for testing Read more…

By Tiffany Trader

SC17 Cluster Competition: Who Won and Why? Results Analyzed and Over-Analyzed

November 28, 2017

Everyone by now knows that Nanyang Technological University of Singapore (NTU) took home the highest LINPACK Award and the Overall Championship from the recently concluded SC17 Student Cluster Competition. We also already know how the teams did in the Highest LINPACK and Highest HPCG competitions, with Nanyang grabbing bragging rights for both benchmarks. Read more…

By Dan Olds

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Japan Unveils Quantum Neural Network

November 22, 2017

The U.S. and China are leading the race toward productive quantum computing, but it's early enough that ultimate leadership is still something of an open questi Read more…

By Tiffany Trader

GlobalFoundries Puts Wind in AMD’s Sails with 12nm FinFET

September 24, 2017

From its annual tech conference last week (Sept. 20), where GlobalFoundries welcomed more than 600 semiconductor professionals (reaching the Santa Clara venue Read more…

By Tiffany Trader

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

Leading Solution Providers

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Tensors Come of Age: Why the AI Revolution Will Help HPC

November 13, 2017

Thirty years ago, parallel computing was coming of age. A bitter battle began between stalwart vector computing supporters and advocates of various approaches to parallel computing. IBM skeptic Alan Karp, reacting to announcements of nCUBE’s 1024-microprocessor system and Thinking Machines’ 65,536-element array, made a public $100 wager that no one could get a parallel speedup of over 200 on real HPC workloads. Read more…

By John Gustafson & Lenore Mullin

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Intel Launches Software Tools to Ease FPGA Programming

September 5, 2017

Field Programmable Gate Arrays (FPGAs) have a reputation for being difficult to program, requiring expertise in specialty languages, like Verilog or VHDL. Easin Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This