Sync'ing iTunes Libraries

I recently spent a few days trimming down the amount of space consumed by my home folder. In so doing I discovered a number of things I could be doing better with regards to utilization of my drive space. So I decided to offload most of my media (photos, movies, etc) off my laptop and onto my Mac Mini server. I also decided that one thing I’d like to live on both is iTunes. Note: Before you do anything in this article you should verify you have a good back up. Also, both machines will end up needing to be Authorized for your iTunes account. There are a lot of ways to keep two iTunes libraries in sync. There are also a number of 3rd party tools that can help you do so. I tested all the tools I could find and decided I’d rather just script it myself. Scripting a synchronization operation in Mac and Linux always seems to come down to a little rsync action. Given that rsync is a little old in Mac OS X, I started out by updating rsync to the latest (3.0.7) using the steps provided on bombich.com (I added using /tmp):
mkdir /tmp/rsyncupdate cd /tmp/rsyncupdate curl -O http://rsync.samba.org/ftp/rsync/src/rsync-3.0.7.tar.gz tar -xzvf rsync-3.0.7.tar.gz curl -O http://rsync.samba.org/ftp/rsync/src/rsync-patches-3.0.7.tar.gz tar -xzvf rsync-patches-3.0.7.tar.gz cd rsync-3.0.7 curl -o patches/hfs_compression.diff http://www.bombich.com/software/opensource/rsync_3.0.7-hfs_compression_20100701.diff curl -o patches/crtimes-64bit.diff https://bugzilla.samba.org/attachment.cgi?id=5288 curl -o patches/crtimes-hfs+.diff https://bugzilla.samba.org/attachment.cgi?id=5966 patch -p1 <patches/fileflags.diff patch -p1 <patches/crtimes.diff patch -p1 <patches/crtimes-64bit.diff patch -p1 <patches/crtimes-hfs+.diff patch -p1 <patches/hfs_compression.diff ./prepare-source ./configure make sudo make install sudo rm -Rf /tmp/rsyncupdate /usr/local/bin/rsync –version
Provided the version listed is 3.0.7 then we have a good build of rsync and can move on with our next step, getting a target volume mounted. In this case, I have a volume shared out called simply Drobo (I wonder what kind of RAID that is?!?!). Sharing was done from System Preferences -> Sharing -> File Sharing -> click + -> Choose Drobo and then assign permissions. The AFP server is on an IP address of 192.168.210.10. For the purposes of this example, the username is admin and the password is mypassword. So we’ll do a mkdir in /Volumes for Drobo:
mkdir /Volumes/Drobo
Then we’ll mount it using the mount_afp command along with a -i option:
mount_afp “afp://admin:mypassword@192.168.210.10/Drobo” /Volumes/Drobo
Now that we have a mount we’ll need to sync the library up. In this case, the Music directory on the Drobo has a symlink from ~/Music. This was created by copying my Music folder to the drobo and then rm’ing it (fails when trying from Finder):
rm -Rf ~/Music
Then using ln to generate the symlink:
ln -s ~/Music /Volumes/Drobo/Music
Now sync the files. I’m not going to go into all of the options and what they do, but make sure you have permissions to both the source and the target (using the username and password from the user whose data your changing helps):
/usr/local/bin/rsync -aAkHhxv –fileflags –force –force-change –hfs-compression –delete –size-only ~/Music/iTunes /Volumes/Drobo/Music
Note: If you get a bunch of errors about operations failing then consider disabling the Ignore ownership on this volume setting for any external media you may be using. Now fire up iTunes on the target machine and make sure it works. At this point, I could also share out the Music folder from my laptop and sync back as well, which would effectively allow me to make changes on both machines. However, for now, I only want to make changes on the laptop and not the desktop so there’s no need for a bidirectional sync. Once the sync is complete, we can tear down our afp mount:
diskutil unmount /Volumes/Drobo
Now that we can sync data, we still need to automate the process as I’m not going to want to type all this every time I run it. First up, I’m going to create a .sh file (let’s just say /scripts/synciTunes.sh):
touch /scripts/synciTunes.sh
Then I’m going to take the commands to mount the drive, sync the data and then unmount the drive and put them in order into the script:
/bin/mkdir /Volumes/Drobo mount_afp “afp://admin:mypassword@192.168.210.10/Drobo” /Volumes/Drobo /usr/local/bin/rsync -aAkHhxv –fileflags –force –force-change –hfs-compression –delete –size-only ~/Music/iTunes /Volumes/Drobo/Music /usr/sbin/diskutil unmount /Volumes/Drobo
Once created, the script should be run manually and provided it succeeds then it can be automated (ie – creating a LaunchDaemon). If it works after a little while, then you can consider synchronizing your iPhoto and anything else if you so choose. Also, I ended up actually using ssh pre-shared key authentication and doing rsync over ssh. That allows you not to put the password for a host on your network into an unencrypted form in a script. You could do some trickeration with the password, but you might as well look into pre-shared keys if you’re going to automate this type of thing to run routinely. Finally, I also later ended up removing the iTunes Genius files as I started to realize they were causing unneeded data to sync and they would just rebuild on the other end anyway. Hope this helps anyone else looking to build an iLife server of their own!

Missing Sync for Android, iPhone, Pre Mobile Phones

“Trumpy, you can do stupid things!” So an Android, an iPhone, a Blackberry and a Palm Pre walk into a bar in sync. Get it?!?! Not gonna’ happen. Or is it? If you’re a Mac or a Windows user and you don’t mind the ole’ cradled sync, and you’re an absolute gadget-head, then the Missing Sync is for you. We’ve used it for years with Blackberry and back in the days before Microsoft purchased and summarily hastened the already upcoming doom of the Sidekick platform. But now, it supports Android! So if you’re like some of us and you need to always be accessible and therefore travel with 2 mobiles, then check out the Missing Sync – it might just bring a little sanity to an otherwise insane task (watch out for duplicates). Having said all that, if you’re just after Contacts + Calendars then pretty much every device on the planet now supports Active Sync. This goes further than Active Sync, including a whole host of other options to sync in addition to the traditional PIM type date.

Flow: Amazon S3, iDisk

Flow is a nice little FTP client. But it also supports WebDAV and SFTP as well as Amazon’s S3 and mounting an iDisk from a Mobile Me account. Unlike JungleDisk it doesn’t seem to mount S3 as an actual disk in Mac OS X, but it can be used to take files from iDisk to S3, which is fairly interesting. Flow also supports discovering all of the local services over Bonjour, which can be pretty helpful. Overall, it’s a nice little application that’s pretty sleek and I look forward to seeing where they go with it.

Sync Bookmarks Between IE, Firefox & Safari

If you are a Safari user and you have a MobileMe account then you can already synchronize Safari bookmarks between multiple Macintosh computers.  But what if you want to synchronize to that corporate sanctioned Windows XP machine in the office that runs only Internet Explorer?  What if you also want to synchronize to Firefox, running on another machine?  All three can synchronize together in one harmonious bookmarking ménage à trois.  How is all of this made possible?Xmarks. Formerly Foxmarks, Xmarks now supports more browsers and runs on more platforms. You store a backup of your bookmarks in the Xmarks cloud (which, yes, means that you’ll need to create an account) and then you sync it down to whichever platform you wish. A great solution for anyone who has to jump between browsers on a routine basis, or anyone who doesn’t have a MobileMe account and wants to keep their bookmarks backed up! Also great if you’re having trouble migrating bookmarks between platforms for OS switchers.

Mac in the Cloud

A few days ago I noticed a post in Tim O’Reilly’s twitter feed asking whether or not it would matter whether people ran a Mac or a PC once everyone had migrated to a cloud.  Well, there are a few things about Mac OS X that make it fairly difficult to run in a cloud environment:
  • EFI – Mac OS X doesn’t use a BIOS like most Operating Systems.  This makes the bootup process fairly difficult in a distributed computing environment where the Guest OS would be OS X and the Host OS would be something else.
  • Lack of Firepower – I love the Xserve.  I always have.  They’re some of the most beautiful rack mount servers you can get.  But even an Octacore is gonna’ choke if you throw too many VMs on it.  If I were architecting a large, distributed computing environment I would want some blades, an IBM Shark, etc.  Having said this, Xgrid could pose an interesting option if VMware or Parallels were to allow distributed processing through it.
  • Licensing – The Mac OS X Server software is the only software licensed for a cloud type of environment, if you read your EULA.  This has only recently been introduced and has left Mac OS X without Xen or other open source alternatives in the virtualization space.
Having said all of this, Mac OS X is a wonderful system.  There is a lot it has to offer and I, as much as anyone would like to see it capable of utilizing services like Amazon S3, but I would be on the lookout for some other strategic moves rather than a full-blown Mac OS X capable of running independently in a cloud environment.  For example:
  • Mac Backups to the Cloud – Time Machine, Bakbone, Atempo, Retrospect, etc.  I cannot imagine that one of them will not be able to back up to Google or Amazon S3 at some point in the near future.  GUI level support needs to be there for it to gain wide-scale adoption with the Mac user base (like using Backup.app to backup to MobileMe but with enough capacity to back up an Xsan and enough bandwidth to do full backups in less than 72 hours).
  • Xgrid – There needs to be some kind of port of Xgrid to Amazon EC2 or support from render farm companies for EC2 or some other cloud/grid computing platform.  
  • Apple – The Pro Apps will need to support SaaS, Software + Services, etc.  Many Apple users are leveraging Google Apps, but once it comes from Apple it will be legitimate.
So look for it.  You’ll notice the companies that are really leveraging trends in IT as they come to market with products that allow the Mac to leverage the cloud.  If Apple makes a push towards this then you’ll see more wide-scale adoption, but don’t expect much and you won’t risk getting too let down. 

MobileMe and Downtime

Today I got the following email:
We have already made many improvements to MobileMe, but we still have many more to make. To recognize our users’ patience, we are giving every MobileMe subscriber as of today a free 60 day extension. This is in addition to the one month extension most subscribers have already received. We are working very hard to make MobileMe a great service we can all be proud of. We know that MobileMe’s launch has not been our finest hour, and we truly appreciate your patience as we turn this around. Read this article for more details. The MobileMe Team
Now, I realize that the transition to MobileMe from .Mac hasn’t been a seamless one.  But honestly, I’ve barely noticed any blips in service.  Sure, there were a couple of days there where I needed to use webmail instead of Mail.app or Entourage.  But overall, the transition has been performed with minimal downtime considering the quantity of accounts and systems in play.  So now, given that their poor engineers are likely sweating bullets and wondering who gets canned the next time there’s downtime, MobileMe goes ahead and gives away 90 days of service for free.  It’s nice that Apple has deep pockets to do so, but I really, really hope that it doesn’t come with deprecated service given that they just arguably lost 25% of their revenue for a fiscal year… So, if you’re thinking this transition is a fiasco please, think about the amount of downtime over the course of the last few years that .Mac has encountered.  Consider the fact that for the same price you’re going to have tons of new features and consider the fact that there’s basically always downtime in an upgrade.  I mean, if you were replacing all of the RAIDs in an Xsan wouldn’t you expect there to be a little downtime while the data is copying from your old volume to the new?