In Xsan 2, MultiSAN was introduced. MultiSAN allows you to assign different sets of primary, secondary and tertiary metadata controllers to volumes. This provides a performance benefit for some environments that have saturated resources on a given metadata controller. However, MultiSAN does not allow you to build separate SANs. All of the volumes are still members of that single “Xsan”, meaning that volumes can be mounted and/or controlled for any of the clients. You can have 2 volumes, each with a dedicated metadata controller, but both sharing a single backup metadata controller. You can also have 2 volumes, each with a dedicated metadata controller that fails over to the other metadata controller.
But if you do have 2 SANs, completely separate from one another, you cannot then have a client on both. Therefore, having multiple SANs isn’t exactly MultiSAN. MultiSAN means going from having a single metadata controller that controls your entire environment to having a slightly more object-oriented approach to metadata controllers. Which brings us to the question “do I need MultiSAN?” My answer is an if/then statement. If your metadata controller’s fsm or fsmpm processes are spiraling out of control then yes, you may (or you may have corruption in metadata somewhere. For coming up with the answer to my question I posted an Xsan monitor app awhile back on my apps page. But if your fsm/fsmpm processes barely tip above 1% and you’re still having bandwidth problems then look at stripe breadth/block size, fabric and defragmentation before considering buying a bunch of iron to move metadata controlling off onto dedicated hardware.
Now, if you do decide to integrate MultiSAN then to do so is a very straight forward process. First add the all of the metadata controllers as clients to the SAN. Then, create the volume. When creating a volume you will eventually come to a screen to define Volume Failover Priority. Here, you will see each of the clients that you have installed (in the beginning this might only be the metadata controllers). Check the box for each of the clients that you would like to be a metadata controller (I recommend no less than two but in most installations no more than 3 metadata controllers per volume). In this screen you can then also set priority by dragging each controller higher or lower in the list of controllers. If you only have two metadata controllers then the primary would be at the top of the list followed by the backup metadata controller. When you are satisfied with your configuration click on the Continue button and complete the volume configuration as you normally would. You can also invoke the Volume Priority screen from within Xsan Admin.
Once you’ve set the metadata controllers, you’ll notice that you can use cvadmin to see all of the metadata controllers for your entire SAN. When running a select statement from cvadmin you’ll see an entry for each volume and each metadata controller that manages each volume. From here you can use fail, followed by the numeric entry for each backup metadata in order to fail over to the backup metadata controller for each volume.
While cvadmin shows you live data, you can also see each of the metadata controllers available in the /Library/Filesystems/Xsan/config/fsnameservers list; however, that will not differentiate between those used for specified volumes and so doesn’t impact MultiSAN. When you augment the volume failover information or remove metadata controllers you will then update the /Library/Filesystems/Xsan/config directory for each client of the SAN, so you can track when changes are pushed out based on the date and time stamps there. If any clients don’t receive updates you can then nuke and pave that directory using the steps outlined here.
Overall, MultiSAN is a great feature of Xsan and easy to configure. Occasionally you will have issues with configurations not getting pushed out or clients not showing up properly. But, those are fairly straight forward to resolve as shown and are typically not problems with Xsan but with DNS or infrastructure. If you have a high transaction environments (which by the way is about iops and not space), where you see your Xsan processes spiking MultiSAN can be a way to get around some of the resource limitations.
