Recently I’ve been noticing a trend where organizations with Xsan (and sometimes StorNext) are replacing older metadata LUNs with newer faster LUNs. This often involves replacing an Xserve RAID that sometimes has tens of thousands of hours of spin time on them with a Promise E-class or an ActiveRAID. The trend isn’t just with people I interact with though, as Duncan McCracken mentioned this at MacSysAdmin 2010 and Kuppusamy Ravindran (aka ravi) mentioned it back in 2008 in a post at Xsanity (he actually went way further and looked at actually splitting Metadata and Journaling, a post that is definitely worth a read).
But as the pace seems to quicken to retire those old LUNs with tens of thousands of hours of spin time, I thought I’d just do a quick post on how to migrate those metadata LUNs, which can save tons of time in backing up and restoring your Xsan volumes before you do a LUN swap. So, to get started read the man page for a tool called sndiskmove that is installed with Xsan. The sndiskmove command is going to copy the contents of a LUN (most commonly metadata as the article’s title would suggest) to a new LUN and then rename the old LUN with a .old at the end (configurable with sndiskmove operators) of the old LUN and rename the new LUN with the name the old LUN had.
Before you get started, first do a cvgather and copy the contents of /Library/FileSystems/Xsan/config into a tar somewhere. Also make sure you have a good backup (while this hasn’t failed me yet, it could). Next, label the new LUN. I normally go with something temporary that I can find easily (and type easily in the command). For the purpose of this example, let’s call the new LUN “TARGETLUN”. This can be done with Xsan Admin or with cvlabel. Once the LUN has been labeled then find the name for the metadata LUN of your old metadata LUN, which we’ll call “SOURCELUN” for the purposes of this example. Then unmount all of the clients from the volume and stop it. Finally, check the other metadata controllers and verify there are no fsm processes and run a cvfsck to verify there is no corruption.
Now you’re ready to do the move. Run the cvlabel command and make sure you see both listed in the output, also making sure the capacity for TARGETLUN is greater than that of SOURCELUN:
cvlabel -l
Now run sndiskmove. I usually like to write the output into a temporary text file, which is done using the -S operator. For the purpose of this example, we’ll call that /tmp/sndiskmoveoutput.txt. I like my output verbose, so I use the -v operator for practically every command that will let me. After the operators, use the source volume name then the target name, making the command look as follows:
/Library/Filesystems/Xsan/bin/sndiskmove -v -S /tmp/sndiskmoveoutput.txt SOURCELUN TARGETLUN
Then prepare to wait. Usually it’s about an hour per 250 GB going from Xserve RAID. Once the command is complete you should either reboot the metadata controllers or run cvadmin with the refresh disks option if they report they can’t see your new LUN (happens):
cvadmin ‘refresh disks’
You should see in the output that you now have a SOURCELUN and a SOURCELUN.old. You should then be able to safely remove SOURCELUN.old from the fabric and have that port free for something else, like more data storage or clients…