Mac OS X Server,  Network Infrastructure,  Xsan

Fibre Channel Switches vs. Hubs

In the FC-AL (Fibre Channel Arbitrated Loop) protocol, when devices enter loops, they send out a Loop Initialization Primitive (LIP) request for an address. All other activity on the loop then comes to a halt as each node reestablishes its connection. Since a hub-based fabric is one large arbitrated loop (which can in many cases similarly be built without the actual hub but with less aggregated speed), it must be entirely rebuilt every time any device is rebooted, added to the loop or removed from it. This causes the potential for processing errors, iNode issues/volume errors and even physical disk failures.

A “LIP storm” can also cause multiple devices to send repetitive streams of initialization commands until the problematic target or HBA is located and the loop can be re-estabilized.

Overall the use of Fibre Channel hubs, like that of ethernet hubs has become an edge case, although we do still see them out there. Primarily they are used for backup environments (non-SAN) and other scenarios where high availability of the fabric is not absolutely critical. You should absolutely never ever use a Fibre Channel hub with Xsan.