Code Spaces and Cloud Diversity


This afternoon when I saw @scott_lowe‘s tweet about the attack on Code Spaces, I was shocked. The injustice, violation, and unrecoverable impact of the event hit hard.  That might seem odd, since I frankly didn’t know Code Spaces existed a moment earlier. But like anyone who doesn’t want to see work lost, especially on such a vast scale, I sympathized deeply with the customers who lost code and the Code Space operators who lost their livelihood. Someone burned their digital house to the ground.

After reading the account on, I began processing what it means to the cloud community. What should we learn about our dependence on Software-as-a-Service (SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service (IaaS)? Code Spaces thought they were safe with highly available cloud services, snapshots, and off-site backups…


The weak link in it all was their portal access. I hadn’t really thought about that part of Amazon EC2, Microsoft Azure, etc. Like my VMware vCenter Server access at the office, an attacker doesn’t need to get inside the virtual machines to wreck shop. They can simply delete them. And with backups controlled through the same Amazon interface, nothing else was sacred either.

codespaces_logoThat brings me to cloud diversity. The only thing that might have saved Code Spaces would have been a backup (or redundant) platform in another cloud (public or private) with a separate login. To be safe, if password reset is available via email, a different email account should probably be used as well, but first things first. Then, when EC2 was wiped, they could have pulled from their truly off-site, off-portal backups.

I’m truly sorry for the Code Spaces admins and owners. It’s a dark day, but I second their hope to one day return and reinstate their services.


One Comment

  1. John said:

    Personal rule of thumb: If a backup can be deleted with latency less than a week, it is not a backup.

    A second rapidly-deletable copy of the data would not really solve that problem because it is not uncommon that malicious attacks like this are correlated. Multiple clouds might be vulnerable to the same exploit, for example (even if they are based on different platforms). There may be two exploits available to an attacker, one for each cloud. Computer security history should tell us that 2 unpatched exploits existing at the same time is hardly unthinkable!

    A tape backup, for example, would solve the problem because it is not easy to delete rapidly over the internet. While every backup is in some sense “rapidly deletable” (say a fire would do that for a tape backup), the failure modes of tape and cloud backups are quite different, and offline storage is far less likely to be affected by the same thing that wiped all the other copies of your data.

    I had the same reaction as you. It’s possible there are people out there who were badly affected and feel suicidal about this: if so, please talk to somebody, because you will feel differently later.

    June 22, 2014

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.