Backing up woes
You’ve got your website up and chugging along very well (shared hosting or dedicated server or a vps) and life is good – the traffic is increasing, you are hiring and one fine day the site crashes. You panic, check with your hosting provider and you are told that it was a server crash. The hard drives didn’t survive and your account will be restored from a backup.
The backup could be a day old, a week old or a month old. What also is of note is whether the backup was stored on the server itself, on a different server or on an offnetwork backup setup – or a combination of any of the above. You may wish to check with your provider about the backup setup because if your data really matters you, then you should be careful and proactive enough to find the best solutions that work for you.
Why o why?
Some of the above mentioned backup systems may not be helpful or be enough to save any of your data in case the server was compromised. If you backup all your stuff back onto the same server and a hacker finds a way into your machine – there is a chance he will destroy both your primary data and backups. You’re left with nothing after this, except if you had the foresight to do other forms of backups as well.
Lets say you backed up your stuff within another server in the data center, you are again not scot free. Say the DC goes down, or gets raided by the FBI (check HA post) or just a simple case wherein the server which was compromised is connected to the backup server using SSH keyless login (kinda required for easier setup of regular rsync backups) – here again you could have the possibility of losing all your data. Kinda scary isn’t it?
All this makes a case for Off network backups i.e DC down, FBI raid, earthquake, flash flood, or err.. an errant truck crashing into a pole and taking the DC’s electricity offline.
But wait, if you still allow your primary machine to login keylessly to your external backup system, a hacker can take out your data and backups as well. If you are doing an external backup (within DC) or outside the DC – then work with solutions where you can’t login to the backup systems without knowing the login/pass and the login/pass should never be stored on the primary machine. Take a look at solutions like Evaut or R1soft (we use this) to do backups of all your servers/accounts to an external provider.
We use R1soft because of a couple of features/advantages it allow us – first it does sector level incremental backups and therefore it doesn’t use too much outbound bandwidth as it transfers only the changed files (well, just like rsync), secondly it provides a control panel which lets us restores individual files, directories from any of our backups – we tend to maintain 30 snapshots of our servers at all times and in some cases over 240 snapshots. Finally the killer feature is bare-metal restore – say your box crashed – all you need to do is get a new box up, and specify the R1soft setup to restore stuff. It will replicate everything as per the last snapshot, including the OS. Kinda life saving if you ever need it. If you folks use any other backup setup, I’d love to hear 
While you might get all things right – I’ve seen cases where these backups were not verified and all the hardwork has gone down the drain because the integrity of the backups were not verified on a regular basis. Also you may want to try and restore your backups on a spare server sometime to ensure you have gotten it right. There are various backup options available today, opensource and commercial – but the above are some of the problems we take seriously with our data and work accordingly. It is never possible to have a 100% secure setup (someone just needs to find one loophole or exploit, while you have to continuously patch 100s of them) – but do spend the time and take the effort to build a backup system that appropriately reflects your value for the data. You can’t always spend a bomb to create a backup system, when you may be fine with losing 1 day worth of data
As you think critically of HA, your backup solutions too needs to be thought about in a critical manner that reflects the importance you accord it.
Comments
One Response to “Backing up woes”
Leave a Reply
Hacker Destroys Avsim.com, Along With Its Backups. Check out the link for more.
http://it.slashdot.org/article.pl?sid=09/05/15/0138204&from=rss