[0:45 am] The latest news from Amazon: Lightning hit a power plant supplying power to the Amazon infrastructure. They’re working to restore power.
[8:40 am] Most of our services were back online at 4:00 am this morning. Still, indexing of our layers for the Nearby catalog page is not working and analytics on layer usage might not be running correctly. All in all, not a very good night to start the week.
[5.45 pm] We are fully operational again. Amazon has handled this the best they could actually and were very clear today in their communication to us about how to get back all our RDS instances.
At the moment of writing this post, our services are still down. This is our longest ever major service disruption since we moved our services to Amazon’s Web Services infrastructure 18 months ago. Four hours ago, at 8 p.m. Amsterdam time, I was warned by Jurg, from 9Apps.net, that there was major connectivity loss in Amazon’s network. That was just 5 minutes after the problems started.
Since then I’ve been continuously following #AWS on Twitter and looking at our AWS dashboards to see the progress. It’s frustrating, there’s nothing you can do except wait. You read of other web services in Europe that are down. You know Amazon engineers are probably working their best to get things back to normal. And you hope that when they do, you don’t have to restore from backups.
Right now, the instance serving this blog is working. So we’ve redirected our domain to point to the blog for now. Best thing you can do if you need Layar, is wait till Monday. I’m sure we’ll be back online by then.