Clean Backups Using Windows Server Backup and EBS Snapshots

One of the powerful features of AWS is the ability to snapshot any EBS volume you have operating. The challenge when utilising Windows 2008 R2 is that NTFS doesn’t support point-in-time snapshotting which can result in an inconsistent EBS snapshot. You can still create a snapshot but it may be that one or more file that was mid-change may not be accurate (or usable) in the resulting snapshot.

How to solve

The answer, it turns out, is fairly easy (if a little convoluted) – Windows Server Backup PLUS EBS snapshots.

In order for this to work you will need:

1. Windows Server Backup installed on your Instance (this does not require a reboot on install).

2. An appropriately sized EBS volume to support your intended backup (see the Technet arcticle on how to work out your allowance – remember you can always create a new volume later if you get the size wrong!)

3. Windows Task Scheduler (who needs Quartz or cron ;)).

4. Some Powershell Foo.

Volume Shadow Copy using the new EBS Volume

Once you have created your new EBS volume and attached it to the Instance you want to backup, open up Windows Server Backup and select what you want to backup (full, partial, custom – to be honest you shouldn’t need to do a “bare metal” backup….)  Note that Windows Server Backup will delete *everything* on the target EBS volume and it will no longer show up in Windows Explorer – it will still be attached and show up in Disk Manager, but you can’t copy files to or from it.

The benefit of Windows Server Backup is that it will utilise Windows Volume Shadow Copy Service (VSS)  resulting in a clean copy of files – even those in-flight at the time of backup.  This gets around the inability to “freeze” NTFS at runtime.

Set your schedule and save the backup.

Snapshot the backup volume

Now you have a nice clean static copy of the filesystem you can go ahead and create an EBS snapshot.  The best way I found to schedule this was to use some Powershell magic written by the guys over at messors.com.  I slightly modified the Daily Snapshot Powershell to snapshot a pre-defined Instance and Volume by calling the CreateSnapshotForInstance function that takes an Instance and Volume identifier.  The great thing with this script setup is it auto-manages the expiry of old snapshots so my S3 usage doesn’t just keep on growing.

I invoke this Powershell on the success of the initial Windows Server Backup scheduled task by using the Task Scheduling – see this post at Binary Royale on how to do this (they send an email – I use the same approach to do a snapshot instead).

The great thing about the Powershell is that it will send you an email once completed – I have also setup mail notifications at completion of the Windows Server Backup (success, failure, disk full) so I can know when / if the backup fails at an early stage.  Note that in Windows Server 2008 R2 you can send an email right in Windows Task Scheduler – on AWS you’ll need to have an IIS smart host up and running to support this though – see my earlier post on how to set up a IIS SES relay host.

Summary

The benefit of this approach is a consistent state for your backup contents as well as an easy way to do a controlled restore at a later date (“create volume from snapshot” in the AWS management console and then attach to an Instance).  Additionally you achieve a reasonably reliable backup for not much more than the cost of EBS I/O and S3 I/O and storage costs.

Hope you find this useful.

Tagged , , , ,

14 thoughts on “Clean Backups Using Windows Server Backup and EBS Snapshots

  1. Carlton says:

    Great script. After looking at the messors script it looks like they are powering off the instance during before the snapshot. Do you still need to use VSS backup for consistent backup?

    • Simon says:

      Carlton – in my scenario I’m snapshotting a drive that will have little or no activity and the files I’m interested in should definitely not be in the process of being updated – there may be other files being processed but I’m happy if they aren’t in a conistent state. As my scenario’s so simple I don’t need to restart / stop to take the snapshot or utilise VSS. If I were to try and snapshot a system drive of an active VM then I would need to consider VSS and / or an instance stop or restart.

  2. Carlton says:

    Simon,
    On a few occasions windows update has broken the boot process on my VM. In addition to your Windows backup are you taking a snapshot of the system volume before installing updates?

    • Simon says:

      No, I don’t automate snapshots and updates. Your best bet would be to use load balanced machines and some scripting to do snapshots after removing the machine from the load balancer and then check the system health before adding back to the load balancer.

      If you have a highly customised VM you need to make it easily replacable as it might even be that the underlyinflg hardware or hypervisor dies and you need to rebuild.

  3. Carlton says:

    I work for a non-profit so it’s not critical the instances run 24/7so I’ll probably automate snapshots for system and data volumes while they are powered off. (and save money on monthly instance charges) I can then copy those snapshots to another region for disaster recovery I the event a region goes down.
    Thanks again for your blog and advice!

  4. Carlton says:

    Simon,
    I’ve setup the scripts and am running the Daily script now. You mentioned you modified the Daily script. I’d like to also change it so it only snaps specific volumes for each instance. Mind showing your code change?

    • Simon says:

      I will dig out the change and post the detail here.

    • Simon says:

      Carlton, you need to modify DailySnapshots.ps1. Firstly you need to identify the volume name of the disk(s) you wish to snapshot. Once you have that information you can update DailySnapshot.ps1 so that instead of calling CreateSnapshotsForInstances (plural) you can call CreateSnapshotForInstance (singular) for each volume to backup. I modified this function in AWSUtilities.ps1 so that it would accept an array of volumes to backup as well as the hostname (which is only used for naming purposes). I don’t have access to the source anymore – if you get stuck come back to me.

      • Carlton says:

        Simon,
        It’s been way too long since I’ve worked with arrays. I can see the CreateSnapshotForInstances function and understand replacing it with CreateSnapShotForInstance, but how can you pull certain volumes with it?
        For example, if my instance name was “i-56789023” and I had three volumes, but only wanted to snap two volumes called “vol-Cdrive” and “vol-Edrive” how would I add that to the scripts?

  5. Ian says:

    Hi Simon,
    So glad I found your article but I’m struggling to get to a working solution. What options are you using in Windows Server Backup? All I seem to get is a VHD image which is not much use for creating a EBS volume from. It tried the Full and custom options but it didn’t delete anything on the volume and the volume still shows up in Windows Explorer. I must be doing something different…
    Thanks

    • Simon says:

      Ian – it sounds like you are backing up the entire volume (this is why you would have a VHD file). I think you’re missing a step or two though – the drive on which the VHD is stored is the drive which you must snapshot the EBS volume of. The VHD is simply a file contained inside that snapshot. When you create a copy of the EBS snapshot that copy will contain the VHD which you can then restore. I suggest a re-read and concentrate on the Snapshot the backup volume section. Any questions let me know. Simon.

  6. Ian says:

    Hi Simon, the main issue is actually that I’m trying to get a system backup and not just data volume, I should have explained this too. My db and transaction logs are backed up to S3 already so they’re OK just trying to get a consistent system backup. This doesn’t appear to be possible in AWS using the method above as there is no way to run the system recovery. I was hoping for a mirror copy of the volumes that I could snapshot and then restore through AWS. Thanks for your reply.

    • Simon says:

      Ian,

      Your best bet is probably to try and schedule a regular window where you can shutdown the EC2 instance and then simply snapshot the EBS volumes hosting the filesystem for it. This way you would have an entire state snapshot that would be restorable in the event that the instance becomes corrupted. I typically don’t bother taking a system snapshot as I try to make the instances as simple as possible to make rebuild / re-provision as easy as I can.

      HTH, Simon.

      • Ian says:

        Yep, that’s pretty much what we’re doing currently but was trying to find something without taking systems offline. Long term we’re changing the architecture to be more highly available. Tanks again.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: