Tiny Deathstars of Foulness

I was sitting at the JAMF National User Conference today and I couldn’t help but notice that CrashPlan is a sponsor. It got me to thinking about something someone said last week at the MacTech conference about the fact that CrashPlan was great for clients but that it would need lots of agents for backing up Servers. So at 318, we’ve developed a tool for doing a lot of the service configuration backup in a Mac OS X Server environment, in the form of sabackup and our Open Directory Archiver package. This tends to get most everything we need to backup for the Apple platform. But what about Windows Servers… What is System State? System State Backups are a critical aspect of backing up Windows Servers. The System State is any data that would be required to restore Active Directory and Windows Server to working order. The new CrashPlan Pro is turning out to be just a monster of a tool. Most data in System State is duplicative and CrashPlan has great technology for deduplicating data to multiple sets, with some potentially onsite and others offsite. So this seemed like a match made in heaven when I first approached it. So let’s look at doing so. This starts out with what exactly is System State. It usually includes the following:
  • Active Directory Domain Services data
  • The Certificate Services database
  • Boot files, including system files
  • Registry hives
  • COM registration database
  • sysvol
  • Internet Information Services (IIS) meta-directory
  • Cluster Service information (if applicable)
A System State backup then takes this data and places it into a container, such as an uncompressed .bkf file or the more modern .vhd with some metadata in .xml files. When you back the System State up from any modern backup tool, it’s leveraging technology provided to the vendor via an API. This API is also exposed by Microsoft in the form of the wbadmin tool from PowerShell.
Note: You will need PowerShell installed for this.
Backup wbadmin is a tool included with Windows Server 2008 used to backup the server. The wbadmin tool is pretty simple to use, simply run it along with a start verb, indicating a systemstatebackup and a target location in the form of a -backuptarget operator. The target location is indicated by drive letter, as the System State Backup uses mapped drives rather than directory or URI-based target locations. In this case, we’ll use the C: drive. Finally, we’ll throw the -quiet option in there as the systemstatebackup subcommand actually prompts for a yes or no when running, so quiet automatically gives it a yes. wbadmin start systemstatebackup -quiet -backuptarget:c: If you run this command as listed above it will fail. Then why did I show you how to do it that way? To make a point. You cannot backup to the source. Well, that and I’m kinda’ mean. But while I don’t see a valid reason to do this, you can by adding the AllowSSBToAnyVolume registry value at the HKLMSYSTEMCurrentControlSetServiceswbengineSystemStateBackup. This would be a DWORD with a payload of 1. But it really isn’t a great idea unless you have absolutely nowhere else to back up to… And in this case, we’ll be looking at backing up the data stored here to the cloud. CrashPlan’s cloud as an example, but the same concepts can be used with any cloud service. But the way I am doing this is to go to a separate volume, such as F: and then to backup that volume with CrashPlan, thus not accidentally maybe filling up my boot volume and putting myself into a situation where I actually need to use a system state restore… Once files are backed up through Windows, they can then be backed up through CrashPlan. The backup files are no longer in the bkf format, long used by ntbackup. My preference again is to have a dedicated drive, but if you do not have one then you can filter files in the WindowsImageBackup directory on the target volume from within CrashPlan. These are .vhd files but I really wouldn’t filter by file type. Restoration Now to restore. Once you restore data from CrashPlan, you’ll need to restore the data into the System State as well (keep in mind that CrashPlan is not a “bare metal backup” solution, so you’ll need a functional 2008 Server to perform this). Or more common, you’ll use a restore to correct corruption issues. Given that the systemstatebackup option was used to backup, you probably guessed by now that the systemstaterecovery option is used to restore. Because you can have multiple versions of the backup (each time the backup is run you will create a new version), use the get verb followed by the versions option to see a list of versions: wbadmin get versions The version identifiers are date and time stamped. You then use these after the same command as above to kick off an actual restore: wbadmin start systemstaterecovery -version: 11/08/2011-22:17 To then monitor the status of the restore, use the get subcommand again, but this time followed with status: wbadmin get status Logic Because the systemstatebackup subcommand is going to create a good bit of data on systems not used to having that much data, it’s worth throwing a little logic into our mix. Let’s say that we’re backing up our System State on a daily basis. We can rotate those every week using the delete systemstatebackup subcommand along with an option to -keepVersions followed by a number, which decides how often that the backups would be rotated: wbadmin delete systemstatebackup -keepVersions:7 We don’t need to keep the older versions as presumably provided CrashPlan is setup properly they can be kept there. Instead of keeping 7, we could just delete the oldest: wbadmin delete systemstatebackup –backuptarget:e: -deleteOldest –quiet But we can monitor for backups running long using the get status subcommand previously mentioned. If we trap for any jobs that go over the time limit, we can then stop job, again using quiet so as not to prompt: wbadmin stop job -quiet Note: These backups usually take upwards of an hour for decent sized environments. The last thing to do is schedule all this, done using the Windows Scheduler MMC. Task Scheduler is located in either Control Panel -> System or in Control Panel -> Administrative Tools -> Task Scheduler. Make sure to “run where user is logged in or not” and “run with the highest privileges.” You can also trigger the backup to run hourly. Because of the deduplication tech built into CrashPlan this shouldn’t cause a boat load of new data to suddenly arrive in your backups. And with the fact that you’re most commonly going to be restoring these to circumvent corruption issues it is worth mentioning that backing up the data via CrashPlan then provides more historical flexibility that most people currently have with other tools. Conclusion Traditionally, we like to think that we need an agent for a software package like Backup Exec, CommVault or something of the sort that has an option for backing up the Windows Server System State. This goes way back to the days when Windows Servers were young. And it never hurts to have all your eggs in one basket. Or does it? There is no reason not to use a mature backup tool for such tasks if you already have it licensed. CrashPlan is obviously interested in various aspects of backing up servers. If you look at this article, they’ve written up techniques on working with VSS data already, potentially huge for some environments as means of open file backup (not a cheap module from some vendors). For older versions of Windows Server (e.g. NT through 2003) you can use ntbackup to run a system state backup. The syntax would be a little bit different, but would look something like this: ntbackup.exe backup systemstate /v:yes /hc:off /m normal /j “fsmo1 system state backup to K” /l:f /f “K:fsmo1fsmo1-system-state.bkf” You can script a backup of a Windows Server, then back that up with CrashPlan, or some other tool. The flexibility behind doing so is that with the deduplication done on the data allows for a lot more backups to be stored than with traditional solutions. The only downside here is in the number of moving parts. If the process is more likely to break down because it has moving parts then that might offset the long-term flexibility to having Active Directory data and other information from the system state saved. However, you can layer other solutions and use this as one tool in the tool belt, or even just build more logic behind reporting on errors into your scripts. Seems like a wash with the pros and cons and it comes at a much cheaper point of entry. Finally, if you aren’t doing anything now then it absolutely cannot hurt to jump on something like this to get that data backed up!

November 8th, 2011

Posted In: Active Directory, Windows Server

Tags: , , , , , , ,