7:33 AM, August 31, 2019, a power failure hit Amazon Web Services (AWS) US-East-1 datacenter in North Virginia. As expected, the datacenter’s backup generators kicked in.
Then at about 9 AM the generators started failing.
The result was 7.5% of Amazon’s data storage units were unavailable for several hours. Some of the affected websites and services included Reddit, Styleseat, Fortnite, Sendhub … and many smaller sites and services. Power was restored, and around noon most of the drives were functioning. But the outage rendered some data unrecoverable. For those that had not backed up their data, the power failure was a disaster. 1
Just a few months before this incident, AWS upgraded its guaranteed uptime to 99.99%. Stated that way it sounds really good, but that’s the same as saying you can expect 9.6 hours’ outage per year. And, if AWS misses the mark and your business is without services for a longer period, the penalties to them — which show up as credits on your invoice — are trivial. In this vein, the AWS License Agreement protects Amazon from real indemnity: “[Amazon has] no liability whatsoever for any damages, liabilities, losses … or any other consequences.”
5000 US governmental agencies use AWS. 2 There is no business with better resources, support or redundancy than Amazon. I bring up this outage, not as an indictment. Rather this is a reminder about the difference between marketing promises and the risky reality of storing data in the cloud.
As programmer Andy Hunt tweeted, “The cloud is just a computer in Reston with a bad power supply.” 3
Bryley’s recommended best practices help ensure that data is not lost and downtime is minimized. Follow the principles of the 3-2-1 rule: Three copies of important data. Two different media types. One copy offsite. For cloud-hosted services, though not inexpensive, it’s not unreasonable to take a multi-cloud approach, in which one copy of your data is kept on your premises, one copy is at a remote server (such as AWS) and a third copy is at a different remote server that can be spun up in an emergency (such as Bryley’s Managed Cloud Services). Contact Bryley to help you find a solution that will keep your data secure and accessible, 978-562-6077 or ITExperts@Bryley.com
2 https://aws.amazon.com/government-education/government/
3 ttps://www.bleepingcomputer.com/news/technology/amazon-aws-outage-shows-data-in-the-cloud-is-not-always-safe/
