Category Archives: Time Machine

cloud FileMaker Mac OS X Mac OS X Server Mac Security Mass Deployment Network Infrastructure Time Machine Xsan

Obtain Information From Watchman Monitoring Using a Script

Watchman Monitoring is a tool used to monitor computers. I’ve noticed recently that there’s a lot of traffic on the Watchman Monitoring email list that shows people want a great little (and by little I mean inexpensive from a compute time standpoint) monitoring tool to become a RMM (Remote Management and Monitoring) tool. The difference here is in “Management.” Many of us actually don’t want a monitoring tool to become a management tool unless we are very deliberate about what we do with it. For example, that script that takes a machine name of ‘rm -Rf /’ that some ironic hipster of a user decided to name their hard drive because, well, they can – well that script that was just supposed to run a fix permissions because that ironic jackass of a user in his v-neck with his funny hat and unkempt beard just accidentally cross-site script attacked himself and he’s now crying out of his otherwise brusque no-lense having glasses and you’re now liable for his data loss because you didn’t sanitize that computer name variable before you sent it to some script.

Since we don’t want the scurrilous attention of hipsters everywhere throwing caustic gazes at us, we’ll all continue using a standard patch management system like Casper, Absolute, Munki, FileWave, etc. Many organizations can still take value out of using Watchman Monitoring (and tools like Watchman) to trigger scripted events in their environment.

Now, before I do this I want to make something clear. I’m just showing a very basic thing here. I am assuming that people would build some middleware around something a little more complicated than curl, but given that this is a quick and dirty article, curl’s all I’m using for examples. I’m also not giving up my API key as that would be silly. Therefore, if I were using a script, I’d have two variables in here. The first would be $MACHINEID, the client/computer ID you would see in Watchman. This would be what you see in red here, when looking at an actual computer.

Screen Shot 2013-07-03 at 9.35.54 AM

The second variable is my API token. This is a special ID that you are provided from our friends at Watchman. Unless you’re very serious about building some scripts or middleware like right now, rather than bug them for it, give it a little while and it will be available in your portal. I’ve given the token $APITOKEN as my variable there.

The API, like many these days is json. This doesn’t send entire databases or even queries, but instead an expression of each variable. So, to see all of the available variables for our machine ID, we’re going to use curl (I like to add -i to see my headers) and do the following lookup:

curl -i https://318.monitoringclient.com/clients/$MACHINEID.json?auth_token=$APITOKEN

This is going to spit out a bunch of information, parsed with a comma, whereas each variable and then the contents of that variable are stored in quoted text. To delimit my results, I’m simply going to awk for a given position (using comma as my delimiter instead of the default space). In this case, machine name is what I’m after:

curl -i https://318.monitoringclient.com/clients/$MACHINEID.json?auth_token=$APITOKEN | awk -F"," '{ print $4}'

And there you go. It’s that easy. Great work by the Watchman team in making such an easy to use and standards compliant API. Because of how common json is I think integrating a number of other tools with this (kinda’ like the opposite of the Bomgar implementation they already have) is very straight forward and should allow for serious automation for those out there that are asking for it. For example, it would be very easy to say take this output and weaponize it to clear caches before bugging you:

“plugin_id”:1237,”plugin_name”:”Check Root Capacity”,”service_exit_details”:”[2013-07-01] WARNING:  92% (276GB of 297GB) exceeds the 90% usage threshold set on the root volume by about 8 GB.”

Overall, I love it when I have one more toy to play with. You can automatically inject information into asset management systems, trigger events in other systems and if need be, allow the disillusioned youth the ability to erase their own hard drives!

Football Mac OS X Mac OS X Server Mac Security Mass Deployment Time Machine

2012 Penn State MacAdmins Conference

Don’t let the theft of the Paternoville sign fool ya’, State College is as safe as ever. That is, until a bunch of Mac guys descend on the Nittany Lion Shrine. Yes, it’s that time of the year again when Mac guys from around the world (and yes, all of the speakers are male) descend upon Pennsylvania State University from throughout the Big 10 and beyond to discuss the Penn State mascot, the Nittany Lion. Actually, it’s a mountain lion, so we can’t discuss it quite yet at that point, but we can talk about a slightly bigger cat: Lion.

Lion deployment, scripted tools, Munki, InstaDMG, Puppet, migrations, “postPC,” PSU Blast, Dual Boot, NetBoot, reboot (just threw that in there because it sounded like it fit, but I’m sure much rebooting will be done anyway) and even iOS. Oh, and don’t forget lecture capture, launchd, monitoring, scripting, Boot Camp via BitTorrent (wait, what?), Damn Logs, Subversion (long live git), IPv6 (long live IPv4), DeployStudio (long live the French), Reposado (long live the mouse), Luggage, Casper (long live Minnesota!), ARD (long live the friggin’ App Store), troubleshooting, FileVault (long live Howard Hughes’ legacy), Tivoli (long live that 1984 video), Munki (crap, I already said that) and even iPad (which runs iOS I think).

Overall, the lineup is superb and looking at it, I am honored to be giving a session on Lion Server amidst all the cool stuff going on around me. I’m very impressed with the number and level of speakers and very excited to be a part of it. I’m also excited to be participating with Allister Banks, a cohort from 318, who will be giving talks on InstaDMG and Munki. Overall, it is sure to be a great conference and I look forward to hopefully seeing you all there if I don’t get arrested at the airport for wearing University of Minnesota socks.

Speaking of the Big 10. Did you know there are 12 teams in the Big 10? Did you know the Big East now has teams in Idaho and California? Did you know that the Big 12 has 10 teams? Did you know that the Pac 12 has 4 teams in 3 states that don’t touch the Pacific ocean? What does all this mean? No, it does not mean that we will discuss basic arithmetic and geography at the conference; however, we might show off some apps that can help the math professors at the member institutions of these higher education conferences teach these basic subjects a bit better. Disclaimer: I went to the University of Georgia and am required by having done so to poke fun at other conferences whenever it is possible. Having said that: how many Georgia programmers does it take to change a light bulb?


They can’t, it’s a hardware problem! OK, terrible joke. So here’s a picture of the Georgia mascot chomping down on an opposing (Auburn) player.

Seems like I’m going through football season withdrawals all of a sudden… Point of all this, go to the conference. It’s sure to be a hoot, and I’m sure there will be plenty of talk about football, er, I mean Mountain Lions, er, wait, I mean Mac OS X and iOS!

Mac OS X Server Mac Security Time Machine

Using ServerBackup to Backup Lion Servers

ServerBackup is a new command included in Lion Server, located in the /usr/sbin/ServerBackup directory. The ServerBackup command is used to backup the server settings for services running on a Lion Server. The command is pretty easy and straight forward to use, but does require you to be using Time Machine in order to actually run.

In the most basic form, ServerBackup is invoked to run a backup using the backup command. Commands are prefixed with a -cmd followed by the actual command. As you might be able to guess, the commandlet to fire off a backup is backup. The backup command requires a -source option which will almost always be the root of the boot volume (/):

/usr/sbin/ServerBackup -cmd backup -source /

The data backed up begins in a .ServerBackups directory on the root of the host running Time Machine. Once the backup is complete the data is moved over to the actual Time Machine volume, using a path of:

/Volumes/<TimeMachine_volume_name>/Backups.backupd/<hostname>/<date>/<GUID>/<Source_Volume_Name>/.ServerBackups

The output of a backup should look similar to the following:

2012-02-01 10:05:17.888 ServerBackup[15716:107] Error encountered creating ServerMetaDataBackupFolder at path := /.ServerBackups!
*** nextPath := 40-openDirectory.plist
*** nextPath := 45-serverSettings.plist
*** nextPath := 46-postgresql.plist
*** nextPath := 55-sharePoints.plist
*** nextPath := 65-mailServer.plist
*** nextPath := 70-webServer.plist
2012-02-01 10:05:18.480 ServerBackup[15716:107] SRC := /etc/apache2/
DST := /.ServerBackups/webServer
Failed to copy /etc/apache2/ to /.ServerBackups/webServer/etc/apache2; ret -> 0
2012-02-01 10:05:18.483 ServerBackup[15716:107] SRC := /etc/certificates/
DST := /.ServerBackups/webServer
Failed to copy /etc/certificates/ to /.ServerBackups/webServer/etc/certificates; ret -> 0
*** nextPath := 75-iChatServer.plist
*** nextPath := com.apple.ServerBackup.plist
curServicePath := /.ServerBackups/openDirectory/openDirectory.browse.plist
WARNING: Service openDirectory folder does not exist for browsing.
curServicePath := /.ServerBackups/serverSettings/serverSettings.browse.plist
WARNING: Service serverSettings folder does not exist for browsing.
curServicePath := /.ServerBackups/postgresql/postgresql.browse.plist
WARNING: Service postgresql folder does not exist for browsing.
curServicePath := /.ServerBackups/sharePoints/sharePoints.browse.plist
WARNING: Service sharePoints folder does not exist for browsing.
curServicePath := /.ServerBackups/mailServer/mailServer.browse.plist
WARNING: Service mailServer folder does not exist for browsing.
curServicePath := /.ServerBackups/webServer/webServer.browse.plist
WARNING: Service webServer folder does not exist for browsing.
curServicePath := /.ServerBackups/iChatServer/iChatServer.browse.plist
WARNING: Service iChatServer folder does not exist for browsing.

There are usually a lot of warnings, as any given server might not be in use on the server. There is a postBackupComplete commandlet that is supposed to remove the .ServerBackups directory following the backups; however, the default behavior seems to be to remove the directory without requiring that option.

You can then view the backup snapshots by path (they can also be viewed by cd’ing straight into them):

/usr/sbin/ServerBackup -cmd list

To delete a snapshot from the list shown (where <PATH> is a path from the output of list):

/usr/sbin/ServerBackup -cmd purgeSnapShot -path <PATH>

The backup files themselves are actually the service name followed by a .conf extension; however, the data in the configuration files are just the output of a serveradmin settings of the service, such as what you would get from the following:

serveradmin settings afp > afp.conf

For running services, there’s also a .status file (personally, I’d prefer a .fullstatus file instead if I had my druthers). While all services are exported, and can be manually restored by flipping that > from the above command to a <, some services can also be restored using the services commandlet. To see a list of services that are backed up specifically and can be granularly installed as an option:

/usr/sbin/ServerBackup -cmd services

To restore:

/usr/sbin/ServerBackup -cmd restore -path /Volumes/VOLUMENAME/Backups.backupdb/HOSTNAME/SNAPSHOT -target /

To restore a specific service (for example, the iCal Server):

/usr/sbin/ServerBackup -cmd restoreService -path /Volumes/VOLUMENAME/Backups.backupdb/HOSTNAME/SNAPSHOT -target / -service

Currently, ServerBackup is not included in the daily, nightly or monthly periodic scripts and it does not back up actual data, just settings, so if you’re going to rely on it, you might need to automate server settings backups as needed. The ServerBackup command does a few pretty cool things. However, there is a lot more work needed to get it to be holistic. We’ve been working on scripts for similar tasks for a long time. For more information on that see sabackup.sourceforge.net (although we’re likely to relocate it to github soon). For more information on ServerBackup itself, see the help page (no man page as of yet):

/usr/sbin/serverbackup -help

To see what version that ServerBackup is using (not actually very helpful but can be used to programatically verify ServerBackup is using the latest version):

/usr/sbin/ServerBackup -cmd version

Supposedly there is a prefs command, but I have yet to actually get it to do anything:

/usr/sbin/ServerBackup -cmd prefs

Finally, if you are scripting this stuff, don’t forget quotes (as you might have a space in the hostname). Also, a quick sanity check to determine size and make sure there’s available capacity using the size command let, which only outputs the required space for a ServerBackup backup:

/usr/sbin/ServerBackup -cmd size

Business Mac OS X Mac OS X Server Mac Security Mass Deployment public speaking Time Machine

My New Book on Time Machine Now Available

I have published a new book on Time Machine (Time Capsule, deployment/Managed Prefs and Time Machine Server as well). I wrote it months and months ago and it finally ended up getting posted (publishing is a weird world like that sometimes). It is available for Kindle (Amazon) for now and should be up on the iBooks store as soon as the good people from iTunes Connect get back from their holiday break. To quote the Amazon excerpt:

Time Machine is Apple’s built-in backup solution that comes bundled with Mac OS X. In this book, we will explore Time Machine, looking at how to enable Time Machine, configure what to back up and where to back up to.

Much of Time Machine has to do with the network environment that a computer is in, or the ecosystem. In this book, we look at using Apple AirPort and Time Capsule in such an ecosystem. We also look at using network attached storage and other 3rd party solutions, as most environments are heterogenous.

This book is written from the ground up for Lion. As such, tools like FileVault 2 are covered. We also look at getting more granularity in your backup configuration, as well as third party tools used to backup Lion computers. And of course, no book about Time Machine in Lion would be complete without taking a look at Time Machine Server, a way to centralize backups in an environment around the Time Machine solution.

Finally, Time Machine is more scalable than ever in Lion; however, mass integration may require centralized management (such as Managed Preferences) or scripting automations to configure backups. In this book, we will look at typical deployment scenarios and what else needs to go into moving Time Machine from a basic backup tool to a much more comprehensive backup solution.

This is my first foray into the eBook publishing thing, so if you see anything off, that I missed, etc please let me know. The book is available here or using the link below:

Mac OS X Mac OS X Server MobileMe Time Machine

Mac in the Cloud

A few days ago I noticed a post in Tim O’Reilly’s twitter feed asking whether or not it would matter whether people ran a Mac or a PC once everyone had migrated to a cloud.  Well, there are a few things about Mac OS X that make it fairly difficult to run in a cloud environment:

  • EFI – Mac OS X doesn’t use a BIOS like most Operating Systems.  This makes the bootup process fairly difficult in a distributed computing environment where the Guest OS would be OS X and the Host OS would be something else.
  • Lack of Firepower – I love the Xserve.  I always have.  They’re some of the most beautiful rack mount servers you can get.  But even an Octacore is gonna’ choke if you throw too many VMs on it.  If I were architecting a large, distributed computing environment I would want some blades, an IBM Shark, etc.  Having said this, Xgrid could pose an interesting option if VMware or Parallels were to allow distributed processing through it.
  • Licensing – The Mac OS X Server software is the only software licensed for a cloud type of environment, if you read your EULA.  This has only recently been introduced and has left Mac OS X without Xen or other open source alternatives in the virtualization space.
Having said all of this, Mac OS X is a wonderful system.  There is a lot it has to offer and I, as much as anyone would like to see it capable of utilizing services like Amazon S3, but I would be on the lookout for some other strategic moves rather than a full-blown Mac OS X capable of running independently in a cloud environment.  For example:
  • Mac Backups to the Cloud – Time Machine, Bakbone, Atempo, Retrospect, etc.  I cannot imagine that one of them will not be able to back up to Google or Amazon S3 at some point in the near future.  GUI level support needs to be there for it to gain wide-scale adoption with the Mac user base (like using Backup.app to backup to MobileMe but with enough capacity to back up an Xsan and enough bandwidth to do full backups in less than 72 hours).
  • Xgrid – There needs to be some kind of port of Xgrid to Amazon EC2 or support from render farm companies for EC2 or some other cloud/grid computing platform.  
  • Apple – The Pro Apps will need to support SaaS, Software + Services, etc.  Many Apple users are leveraging Google Apps, but once it comes from Apple it will be legitimate.
So look for it.  You’ll notice the companies that are really leveraging trends in IT as they come to market with products that allow the Mac to leverage the cloud.  If Apple makes a push towards this then you’ll see more wide-scale adoption, but don’t expect much and you won’t risk getting too let down. 
Mac OS X Mac OS X Server Time Machine

Mac OS X 10.5: Time Machine Config

You can customize what Time Machine does not back up by using the following plist:

/System/Library/CoreServices/backupd.bundle/Contents/Resources/StdExclusions.plist

Simply add the strings that you don’t want to back up and it will no longer back up those locations.  Remove the strings to re-add them at a later date.
In the UserPathsExcluded key, you can exclude paths that in relation to users home directories.
Mac OS X Mac OS X Server Time Machine

Mac OS X 10.5: Using Unsupported Disks with Time Machine

I originally posted this at http://www.318.com/TechJournal

If you want to use an unsupported disk type for your Time Machine archives, running the following command on workstations will allow you to do so:
defaults write com.apple.systempreferences TMShowUnsupportedNetworkVolumes 1