The first thing you should do when installing Xsan in any OS, much less Mavericks, is make sure your hostnames are awesome. Forward, reverse, views if you use them, etc. You have to have a dedicated metadata Ethernet network, so you should have a zone entry for both your primary and metadata network interfaces.
You should also have fibre channel and storage configured and ready to use. Given that a lot of storage arrays take a long time to configure these days, I like to actually start that and then do my server setups while my LUNs/arrays are baking.
Setting Up Xsan
Once you have DNS entries, storage and fibre channel connectivity, then you can go ahead and configure Xsan. To do so, first open the Server app and click on Show for Advanced. The last item will be Xsan, so click on the Xsan entry.
You are then prompted for the Xsan Computer Type. Given that this is the beginning of the article, we’re going to assume that we’re configuring the initial Metadata Controller on a new SAN. Therefore, use the Configure as Primary Xsan Metadata Controller entry and then click on Next.
At the Configure your server’s network, make sure that you have two network interfaces. The one at the top of the list should be your primary network, that you use to access the Internet and other items.
The second item in the list should be your metadata network. It’s common to only provide an IP address and Subnet mask for this interface.
At the Xsan Directory Settings screen, you have the option to let Xsan Admin control users and groups, attach to an existing Open Directory server or basically have Xsan just run and BYODS (Bring Your Own Directory Service). I would basically always use the final option. You should stand up Open Directory independently, or Active Directory and use one of those and not tie it into Xsan, as that’s going to provide the most predictable of outcomes.
Select the option (we’re going to select to configure a directory server later) and click Next.
When prompted that Xsan is ready, click Finish.
Configuring a SAN
At the Xsan pane of the Server app, click Launch Xsan Admin.
Provided this is the first time the app has been launched, click Continue at the Introduction screen.
Provided you don’t have an existing SAN, click Configure new SAN (if you have an existing SAN and you’re setting up a new system you would click Manage existing SAN and then provide the hostname of a metadata controller along with the appropriate local administrative account information to access the system). Click Continue.
At the SAN Settings screen, provide a name for the SAN (this is different than the name of volumes provided by the SAN) along with the name and email address of an administrator (optional) and click Continue.
At the Add Computers screen, add all of the computers that will be members of the SAN. If you haven’t yet enabled Xsan on client computers using the Xsan System Preference pane check the box for the server; otherwise select the computers that you’ll be adding and click Continue.
At the Authenticate SAN Computers screen, provide a username and password that is on all the systems (or use Authenticate to SAN computers one by one to enter the information manually for each) and click Continue.
At the Private Metadata Network screen, choose “Yes, manage private Ethernet network settings” and then click Continue.
At the Summary screen, review the displayed information, paying particular attention to the Ethernet Metadata Network to make sure it’s your metadata network and not your production network. Provided everything is correct, click Continue.
Volume Creation
You can create your first volume at this time, or later if the storage hasn’t yet been configured.
For the purposes of this example, we’ll go ahead and create our volume by clicking Continue.
At the Volume Name and Type screen, provide a name for the volume you’re creating. Also, select the type of data that the volume will contain.
There are a lot of factors that go into planning how the volume will perform, such as stripe breadth, block size, iNode expansion, etc. For most environments one of the canned items will work. However, for some environments you might want to get more specific by clicking on the Advanced Settings button.
At the Advanced overlay, choose the options appropriate to your environment. Some to be made aware of:
- Block Allocation Size: The size of each block being written.
- Allocation Strategy: Round Robin is the fastest but evenly distributes data across potentially uneven pools of storage. Balance automatically chooses the allocation to even pool utilization. Fill fills each pool before moving to the next pool. 99% of the SANs I’ve configured have been Round Robin, unless I’m trying to reallocate data, which I typically do with the snfsdefrag command, so most will select the default setting here of Round Robin.
- Spotlight: If you see a lot of mdutil errors in the logs or find certain apps weird, you can disable this but as of Mountain Lion, the service works much better than it did in previous versions, so I usually leave it enabled now.
- Native Extended Attributes: I usually disable this option as we’ve had lots of problems with Extended Attributes on Xsan-based volumes.
- Access Control Lists: Allow permissions via ACLs.
- Cases Insensitivity: Disable this feature to switch to a case-sensitive file system.
- Cache Settings: Can be helpful for database and 4k video workflow environments.
Once you’ve entered the appropriate settings, click OK (and keep in mind, you can always destroy this volume later and recreate with different block sizes to try and optimize performance).
At the Configure Volume Affinities screen, click on each LUN and then provide a label for them. Click LUN Details to see detailed information about the LUN (including a device name such as /dev/rdisk2 and the WWN of the LUN). Click OK to get rid of the screen.
Then delete all the affinity tags in the right hand screen, except the first two (unless you’re building a larger volume and know what you’re doing). Because we only have one LUN (or array) we’re going to delete all but one here, but for your purposes you should probably have a metadata LUN that is pretty small and then a bunch of larger LUNs. The smaller LUN goes to the MetadataAndJournal pool and the larger ones get placed into the remaining affinity tag (e.g. Video) so they can be split into multiples of four and configured in pools with tags.
Click Settings.
At the Settings screen, choose a Stripe Breadth (for many an environment you can divine the block size by 1024 to get what the stripe breadth should be. Because we’ve only got one LUN we’re going to select that we can put Any Data in it; however, the typical SAN will have a metadata pool with only Journaling and meatadata and then data pools with Any Data. Each can have a unique Stripe Breadth. Click OK once you’ve configured as desired and then Continue to go to the next step of the volume creation wizard.
If you are doing what I’m doing (which is testing btw, so not going into production) and use the same LUN for data and metadata then you would get a warning. Click Continue Anyway (or hopefully if you see this screen and it’s a production bound volume, click Make Changes to go back a screen.
If you don’t have a multiple of 4 (or 2 according to the volume type you created) you will also get a sanity check that you’re not really making a standard volume. Hopefully you won’t get this screen, but if you do and the volume is as you’d like it, click Continue Anyway.
At the Volume Failover Priority screen, check the box for each Metadata Controller you’d like to manage the volume and then click on OK. You can always add and remove Metadata Controllers in the future; however, all systems will need to be online to do so and the more computers there are, the longer the process requires.
At the SAN Setup screen, click Continue and Xsan Admin will open.
Use Xsan Admin
Xsan Admin is the tool you will use to manage most aspects of the SAN. It can be accessed from within the Server app any time, and pinned to the dock if you won’t really be using the Server app for other tasks (common on Xsan Metadata Controllers).
When the SAN screen opens you’ll see a number of sections. These correspond to different abstraction layers of data to volumes or of computers to volumes or even job tasks such as managing permissions and quotas. The sections are:
- Overview: Shows volumes, capacity of each and some basis statistics.
- Volumes: Click here to see more detailed information about volumes, including pools and LUNs that comprise those volumes.
- LUNs: Click here to see a list of the arrays, or LUNs that are available and labeled. One of the columns also displays the volume that each LUN is a part of (LUNs can only be part of one pool or volume in order to keep data from overwriting segments of the drives).
- Computers: A list of computers, where you can authenticate to systems, add computers to the SAN (provided Xsan is enabled on the client and you have authentication credentials) and see errors (e.g. no LUNs visible might mean that the fibre channel is disconnected).
- Mounts: See a list of volumes and which clients have which volumes mounted.
- File Management: Configure affinities (which map storage pools to folders) and augment permissions on objects within volumes.
- Quotas: Configure maximums for placing data on volumes.
Click on Volumes from Xsan Admin. Here, you’ll see a list of volumes.
You’ll also see which metadata controller is currently hosting the volume, the size and available capacity of the volume.
Use the Inspector (click on Inspector with a volume highlighted) to look at details of the volume
If you click on LUNs, you can label new LUNs when preparing to add storage to Xsan or if a LUN has a yellow warning light on it, you can identify storage that the SAN can’t see.
If you click on Computers, you can use the Inspector to see more detailed information about each.
Click on Mounts and then you can select each volume and control who has it mounted and whether it’s mounted as Read Only or Read & Write.
Click on File Management. Here, you can browse through different folders on the volumes (provided you have them mounted). Click on the Cog Wheel icon.
At this menu you can setup an affinity (which maps a storage pool to a directory), configure permissions (POSIX and ACLs), propagate permissions (applying the permissions configured to child objects of the highlighted directory) and create a directory (which is a useful button to have here so you can build affinities and permissions using this single pane of glass).
Click on Quotas.
Here, you can configure quotas (as the name implies). I don’t use these much (I’ve had problems with them but YMMV). If you click the plus sign, you’ll see a list of groups. Click each to assign a quota for that user or group.
Click on OK to see the objects you just applied quotas to displayed.
Conclusion
Those are the basics of an Xsan. There are a lot of other tasks, like expanding volumes, deleting volumes, deleting clients and re-adding them, etc that you can do as well; however, we’ve configured a basic Xsan and can get into some of those more finely grained tasks as time goes on. Overall, Xsan in Mavericks is much more stable than ever before and provides as many GUI level options as it ever has. Later we’ll also take a look at command line options for Xsan, but until then, good luck!
