Mass Deploying Time Machine in Mac OS X Lion

A lot of environments want to use Time Machine at scale. But prior to Lion there hasn’t been a simple way to do so. Apple has introduced a new weapon in the war to backup client computers in the new command tmutil that was introduced in OS X Lion. The tmutil command allows administrators to enable Time Machine, make snapshots, kick off backups, delete snapshots, perform restores, configure options within Time Machine and, with a little scripting, build a centralized dashboard, pulling in Time Machine statistics from clients.

Enabling Time Machine

The first thing to know is that pretty much everything you do in Time Machine is going to require elevated privileges. So if you are writing a script, it should run as such, or if you’re running each command independently you will likely need to prefix them with sudo. Let’s start with a computer that doesn’t have Time Machine enabled. To enable it, use tmutil along with the enable verb:

tmutil enable

To disable Time Machine, use the disable verb:

tmutil disable

This is the equivalent of sliding the Time Machine slider between the ON and OFF positions.

We’ll also enable local backups, turning on snapshots:

tmutil enablelocal

But these don’t yet associate Time Machine with any disks or configure any of the settings. One of the first things people usually do when they enable Time Machine is to configure a destination volume for backups as you cannot backup if you don’t have a place to backup to. This is done using the setdestination verb. The destination can be a local file system or a network mounted share.  To set a destination as a local volume, simply follow the setdestination verb with an argument that indicates the path to use. For example, if you are pointing backups to a volume called remade:

tmutil setdestination /Volumes/reamde

Setting a destination will either write data into a DestinationVolumeUUIDs key in /Library/Preferences/com.apple.TimeMachine.plist. The contents of the key match the Volume UUID output of diskutil info. For example:

diskutil info disk1s2 | grep Volume UUID

Therefore, it is possible to swap UUIDs using a script on a biweekly or weekly basis or using tmutil along with the volume name, to match an offsite rotation rather than changing the volume in the System Preference pane.

Dealing with Network Mounts

In the case of a network mounted share, you would still use the setdestination verb, but define that the target location is a network mount by embedding a URL into the command rather than a file system path. The traditional URL will consist of protocol followed by :// followed by the hostname/sharename. We can go an extra step and also embed the username and password delimited by a colon and prefixing the hostname, using an @ to separate the credentials and the hostname. For example, if we wanted to define a hostname of tm.krypted.com with a share of snowcrash and a username of neal with a password of theU to access that share we would use the following:

tmutil setdestination afp://neal:theU@tm.krypted.com/snowcrash

Given that you might not want the password embedded into the command, you can use -p to enter a password manually (the password will not be displayed in the terminal screen). In this case, leave the username embedded into the path as follows:

tmutil setdestination -p afp://neal@tm.krypted.com/snowcrash

While the inclusion of a computer name in the path of actual Time Machine backups seems to indicate that it is OK to allow multiple computers to use it, doing so seems discouraged in Apple’s Time Machine documentation. Therefore, sticking with one computer per share will likely be the most secure and least corruptible means of backup. While creating a bunch of shares for backups might seem daunting at first, it’s worth mention that you can script share creation, per client computer in OS X Server using the sharing command. For example, to create a share for a computer named neal in /Shared with AFP only and no guest access:

sharing -a /Shared/neal -s 100 -g 000

To list computers in Open Directory:

dscl /LDAPv3/127.0.0.1 -list Computers

Variabalizing the dscl output into an array and creating machine-specific shares would then net a share per computer (assuming all computers have corresponding records in the directory service). Likewise, shares can be built using a DeployStudio, Absolute Manage or Casper machine export as well.

Configuring the Backup Source

In Time Machine, all data is backed up by default. Therefore, rather than define what the source is, you define what the source is not. Once a target location has been defined, the next thing many Time Machine users do is define any data that is not to be kept in the Time Machine backups. This is done with the addexclusion verb. These exclusions are defined using the Options button of the Time Machine System Preference pane as well.

To use the addexclusion verb, simply define a list of items that are not to be backed up as arguments separated by spaces. The tmutil command will then use those items as an array. If you have one item to exclude, simply list the path. For example, to exclude the OS X Developer Tools:

tmutil addexclusion /Developer

Or to disable a number of items (below we are only backing up /Users):

tmutil addexclusion /System /Library /Applications /var /etc /Developer /Groups /Incompatible Software /Volumes /bin /cores /usr /tmp /temp /opt /net /home /Shared Items /Network /Groups

Provided no errors occur the command should have run properly. The isexcluded verb then allows you to see which source locations are being excluded. Use the verb similarly to addexclusion:

tmutil isexcluded /Developer

A minus sign means it’s being excluded and a plus sign means it’s being backed up. You could also just grab the first position of the output:

tmutil isexcluded /Developer | cut -c 1

You can also use this as a sanity check prior to performing restores at a lower depth. For example, there is no reason to try to recover a file called /Users/cedge/Desktop/systemoftheworld.pdf if it hasn’t been backed up:

tmutil isexcluded /Users/cedge/Desktop/systemoftheworld.pdf | cut -c 1

The arguments for addexclusion are not all of the items being excluded. Instead, you are adding items, but others may already be present. Also, you can define the same exclusion multiple times without adding each item to the list of excluded items. To remove an item, use the removeexclusion verb (you can separate these with spaces as well):

tmutil removeexclusion /Volumes

Finally, addexclusion and removeexclusion have a -p option. By default, if you move an item that has been defined as an exclusion, the exclusion will move with the item. You can specify a -p option to set the path for the exclusion as static:

tmutil addexclusion -p /etc

There are also a number of exclusions that are included by default. These are defined in the .exclusions.plist. The non-default exclusions are stored in the ExcludeByPath array in /Library/Preferences/com.apple.TimeMachine.plist. These are not shown to an end user in the Time Machine System Preference pane though. Those paths can be found in the SkipPaths array within the same file.

By default, the backup source needs to be connected to power. This setting corresponds to the Back up while on battery power checkbox in the Time Machine System Preference pane’s Option overlay. That setting can be disabled using defaults to write a 1 into the RequiresACPower key:

defaults write /Library/Preferences/com.apple.TimeMachine RequiresACPower 0

Manually Running Backups

Once you have defined your source and target, it’s time to test a backup. The tmutil command allows you to kick off a backup immediately run tmutil with the startbackup verb.

tmutil startbackup

Either the backup will work or the Finder will display an error that the backup could not complete. If the system performance is poor during backups or you need to stop one for another reason: use the stopbackup verb:

tmutil stopbackup

In Time Machine a snapshot is an incremental or a fill (aka initial) backup. These are stored on a target volume, or backup disk. For example, the previously used snowcrash volume will contain a snapshot:

Each machine will have its own entry, meaning you can move Time Machine volumes between hosts or use a single network mount to allow backups for multiple clients (although there are some interesting security implications behind doing so). To create a new local snapshot (seen above in the path), use the snapshot verb:

tmutil snapshot

Managing Backups

The Time Machine System Preference pane doesn’t give you a lot of features for managing retention and recycling media. But with OS X Lion, you can now manage and delete given snapshots of data, thus allowing for manually cleaning out your backups (or doing so with a script that has a lot more logic than the default settings).

To see a list of snapshots, use the listbackups verb:

tmutil listbackups

You will see output of each snapshot on the computer:

/Volumes/EncryptedTMBackup/Backups.backupdb/stephenson.krypted.com/2011-08-09-181840
/Volumes/EncryptedTMBackup/Backups.backupdb/stephenson.krypted.com/2011-08-09-195105

To just see that last snapshot:

tmutil latestbackup

The fitting delete verb is used for such a task and is able to take the argument of an old snapshot (or an array of such) to delete. For example, to delete snapshot 2011-08-09 -181840 for computer stephenson.krypted.com on /Volumes/EncryptedTMBackup

tmutil delete /Volumes/EncryptedTMBackup/Backups.backupdb/stephenson.krypted.com/2011-08-09-181840

You can also calculate how much drift has occurred between snapshots:

tmutil calculatedrift /Volumes/EncryptedTMBackup/Backups.backupdb/stephenson.krypted.com/2011-08-09-195105

The calculatedrift verb will show the amount of data added, removed and changed between each backup as well as output the averages of drift between backups (helpful in capacity planning and reporting).

To compare a snapshot to a file system path (or paths), use the compare verb. This is handy for figuring out which snapshots you might be able to nuke if you’re scripting a delete process. The compare verb is one of the more complicated as there are a number of options for how to compare data.

tmutil compare /Volumes/EncryptedTMBackup/Backups.backupdb/stephenson.krypted.com/2011-08-09-195105

The output can be a bit verbose as it looks at each directory. You can limit the depth using a -D option. You can also specify a number of different options to specify what differences to look for when performing lookups. The output of compare is helpful if you preflight your backups with a sanity check to verify there is enough room (otherwise the user might get a Time Machine error dialog).

Other Options

The tmutil command also has options for troubleshooting, moving disks and restores. The restore verb can be handy as you can send restore scripts to clients over ARD or even build a self-restore portal with more options than can be found in the TIme Machine restore screen (I’d recommend using the -v option with restores, btw). The inheritbackup verb can be used to take ownership of a machine directory, useful when moving disks or shares between clients. The associatedisk verb can be used to attach a disk to a backup, thus allowing you to skip beginning backups all over again if the UUID of a disk changes.

Also, the options in 10.6 are still applicable. To suppress the dialog to make all new disks a TimeMachine volume:

defaults write com.apple.TimeMachine DoNotOfferNewDisksForBackup -bool YES

Backups are also still kicked off by com.apple.backupd-auto.plist, stored in /System/Library/LaunchDaemons and the interval between backups can be changed using the StartInterval key here. For example, if you set it to 360 then backups will occur every 6 minutes instead of 60, or more likely, if you set the integer to 14400 then your backups will occur every 4 hours instead of every hour.

Puzzle Pieces

To take a few pieces from this article and combine them. Setting up a basic backup (provided that you have a known volume name per client) is as easy as the following basic, quick and dirty shell script:

/usr/bin/tmutil enable
/usr/bin/tmutil enablelocal
/usr/bin/tmutil setdestination /Volumes/BACKUP
/usr/bin/tmutil addexclusion /System /Library /Applications /var /etc /Developer /Groups /Incompatible Software /Volumes /bin /cores /usr /tmp /temp /opt /net /home /Shared Items /Network /Groups
/usr/bin/defaults write /Library/Preferences/com.apple.TimeMachine RequiresACPower 0
defaults write com.apple.TimeMachine DoNotOfferNewDisksForBackup -bool YES

  • TeckniX

    Thanks for the good explanation of the command lines available.
    Just a quick typo issue:
    diskutil latestbackup
    should be:
    tmutil latestbackup

    :)

    • http://www.krypted.com Charles

      Yikes. Corrected!

  • Pingback: Charles theu | Susankienitz()

  • Krister Laag

    You can use [tmutil isexcluded *] to view all inclusions and exclusions.

  • Krister Laag

    The information listed by [tmutil isexcluded] is not on par with what System Preferences:Time Machine: Alternatives show.
    If you exclude a Volume in the GUI, it will not show up with [tmutil isexcluded]

  • Sunil Punjabi

    Great article! One question.
    Lion has the a new feature that allows laptops to make local snapshots to the boot disk, when a time machine disk is not connected. Later once the dive is connected the backups are moved tot he TM drive.

    My MBP has two internal drives SSD (boot) and HDD (data) is there a way to force the mobile snapshots to use the HDD instead of the SSD?

    • http://www.krypted.com Charles

      I don’t know of a way to configure that. Not sure it’s exposed to the command line…

  • not getting paid enough for this

    It would be nice if you could figure out how [tmutil associatedisk is supposed to work or if it doesn’t work with Leopard volumes. Trying to pull from backups to migrate to Lion from Leopard and it’s pretty joyless. Keeps telling me he chosen volume is not a snapshot volume but by all I can discover, it is.

    • http://www.krypted.com Charles

      Ick. Haven’t looked at backwards compatibility but my guess is no dice…

  • Nathan

    How do I get around “setdestination requires root privileges?”

    Thanks

    • http://www.krypted.com Charles

      Put sudo in front of the command.

  • Olivier Roux

    With a mix of your post and this one http://www.garth.org/archives/2010,07,16,124,fixing-time-machine-sparsebundle-network-backup-errors.html/comment-page-1#comment-56685, I managed to get my Time Capsule backup back on tracks.
    Thanks a lot and thank the web!

  • Kurt

    Are you aware of a way to delete one file from TM backups? We have identified a security issue and need to find a delete a single file that may live in users backups.

    • http://www.krypted.com Charles

      Now that is a good question. I’ll look into it and write something up if I find something.

  • dAn

    Will this work using tmutil to backup a Lion client to a Lion Server?

    • http://www.krypted.com Charles

      Absolutely

  • Pingback: Using Time Machine Server in Mountain Lion Server | Krypted.com()

  • Pingback: Time Machine Travels to Mac Land - Compute Cycle()