StorNext and Xsan go pretty well together. I wrote up an article going on two years ago for Xsanity on setting up RedHat clients for Xsan environments at http://www.xsanity.com/article.php?story=2009011213072797&query=stornext. But I didn’t go into much detail on troubleshooting. There isn’t a ton, beyond the traditional steps you take in Mac OS X, when troubleshooting Xsan clients as there isn’t a lot that can go wrong. But, let’s look at how I normally proceed when I only have one volume that will not mount.
The first step is to stop and then start up cvfs. To stop cvfs, run the following command:
/etc/init.d/cvfs stop
To then start it back up:
/etc/init.d/cvfs start
At this point, a file will be written to with some typically detailed notes on why a volume didn’t mount at /usr/cvfs/debug/mount.VOLUME.out (where VOLUME is replaced by your volume name). If the file is empty then the volume didn’t even attempt to mount. Assuming that we are only looking at a mount (meaning cvadmin will show you the Xsan or StorNext volumes) then it could also mean that you can’t stat the volume. If the error specifically indicates that it cannot stat the volume then it is either that there is not a folder with the same name as the Xsan in /mnt (or wherever you are attempting to stat the volume to) or that the permissions on the folder on the local file system are bad. Changing these to 777 temporarily will likely resolve if it is permissions.
Next, check cvadmin and verify that the volume is being hosted by a metadata controller that is accessible by the Redhat client. The server that is actually hosting the metadata for a given volume will have an * by the name of that server. Also verify that in /usr/cvfs/config/fsnameservers you see that server. Keep in mind that when you add and remove metadata controllers in Xsan that the fsnameservers file does not synchronize to non-Apple products (ie – StorNext) and that you will need to hand roll these changes. Also, keep in mind that per StorNext, the order of the objects within this file needs to be consistent across all clients no matter the platform.
Next: consider licensing. Make sure that licensing files for each metadata controller are available to the clients. If a single license file is out-dated then even if you can get a volume to mount, failover will not be possible.
Another possible (and likely on new volumes) candidate for why a given volume will not mount is that there are inaccessible LUNs. If this is the case then you will see errors that indicate that there is a “stripe group down”, as with Xsan. To isolate these, I usually compare the results of a cvlabel -l command with what I see in Xsan Admin for a given LUN. A little grep will make this process go by very quickly. If you have moved a LUN then I’ve had to do a full physical reboot to get the cvlabel cache to actually update following the move. Also, along this same line of troubleshooting, if you are using a version of StorNext that is a bit older, then you will want to check and verify that it supports LUNs greater than 2TB. This is an older issue, but some still run old software…
That’s about all I have time for, but it’s very specific towards troubleshooting single volume mount issues in StorNext environments. While this was geared towards Apple Xsan environments with StorNext clients, all of the information is also pertinent to StorNext metadata controlling environments as well.
