sFlow is an industry standard that allows network equipment with the appropriate agents to send data to sFlow collectors, which then analyze network traffic. You can install sFlow on routers, switches, and even put agents on servers to monitor traffic. Brocade (along with most other switch manufacturers) supports sFlow.
Before you do anything log into the switch and check the current flow configuration:
To configure, log into the switch and use the the int command to access an interface. From within the interface, use the following command:
Then exit the interface using the very difficult to remember exit command:
Repeat the enablement of forwarding for any other necessary interfaces. Next, we’ll configure a few globals that would be true across all interfaces. The first is the destination address, done using the destination verb followed by the IP and then the port (I’m using the default 6343 port for sFlow):
sflow destination 192.168.210.87 6343
Set the sample
sflow sample 512
Set the polling interval:
sflow polling-interval 30
Finally, enable sFlow:
Yosemite brings Xsan 4, which brings a new way to add clients to an Xsan. Xsan Admin is gone. From now on, instead of scanning the network using Xsan Admin. we’ll be adding clients using a Configuration Profile. This is actually a much more similar process to adding Xsan clients to a StorNext environment than it is to adding clients to Metadata Controllers running Xsan 3 and below. But instead of making a fsnameservers file, we’re plugging that information into a profile, which will do that work on the client on our behalf. To make the Xsan configuration profile, we’re going to use Profile Manager.
To get started, open the Profile Manager web interface and click on a device or device group (note, these are scoped to systems so cannot be used with users and user groups). Then click on the Settings tab for the object you’re configuring Xsan for.
Click Edit for the profile listed (Settings for <objectname>) and scroll down until you see the entry for Xsan.
From the Xsan screen, click Configure.
This next screen should look a little similar, in terms of the information you’ve plugged into the Xsan 4 setup screen. Simply enter the name of the Xsan in the Xsan Name field, the IP address or host names of your metadata controllers in the File System Name Servers field and the Authentication Secret from the Xsan screen in the Server app into the Authentication Secret field. Click OK to close the dialog.
Click Save to save your changes. Then you’ll see the Download button become clickable.
The profile will download to your ~/Downloads directory as Settings_for_<OBJECTNAME>.mobileconfig. So this was called test and will result in a name of Settings_for_test.mobileconfig. That profile will automatically attempt to install. If this is an MDC where you’re just using Profile Manager to bake a quick profile, or if you don’t actually want to install the profile yet, click Cancel.
If you haven’t worked with profiles that much, note that when you click Show Profile, it will show you what is in the profile and what the profile can do.
Simply open this file on each client (once you test it of course) and once installed, they’ll automatically configure to join your Xsan. If you don’t have a Profile Manager server, you can customize this file for your environment (YMMV): Settings_for_test.mobileconfig
The iSCSI Initiator that we use for connecting Windows to iSCSI targets has a friend. It’s called Microsoft Windows Storage Server, which you can use to turn a DAS RAID in a Windows box into a LUN for iSCSI. Good stuff. Check out the data sheet here:
Now that’s not to say they’re the only game in town. iSCSI Target is also a feature of OpenSolaris:
And there’s a nifty little Open Source Project called iSCSI Enterprise Target:
I originally posted this at http://www.318.com/TechJournal
The amount of data used by Small Businesses is on target to rise 30% to 35% in 2006. Sarbanes-Oxley, HIPPA and SEC Rule 17a-4 have introduced new regulations on the length of time data must be kept and in what format. Not only must data be kept, it must be backed up and secured. These factors have the cost of data storage for the Small Business increasing exponentially.
Corporations valued at more than 75 million dollars are generating 1.6 billion gigabytes of data per year. Small and medium sized companies can reap the benefits of developments being made with larger corporations. Different methods and classifications for data are one of these.
Information Lifecycle Management (ILM) is a process for maximizing information availability and data protection while minimizing cost. It is a strategy for aligning your IT infrastructure with the needs of your business based on the value of data. Administrators must analyze the trade-offs between cost and availability of data in tiers by differentiating production or transactional data from reference or fixed content data.
ILM includes the policies, practices, services and tools used to align business practices with the most appropriate and cost-effective data structures. Once data has been classified into tiers then storage methods can be chosen that are in line with the business needs of each organization. The policies to govern these practices need to be clearly documented in order to keep everyone working towards the same goals.
Online storage is highly available with fast and redundant drives. The XRAID and XSAN are considered online storage, which is best used for production data as it is dynamic in nature. This can include current projects and financial data. This data must be backed up often and be rapidly restored in the event of a loss. It is not uncommon to use an XRAID to backup another XRAID for immediate restoration of files and a Tape Library to maintain offsite backups of the XRAID.
Offline storage is used for data retained for long periods of time and rarely accessed. Data often found on offline media includes old projects and archived email. Media used for offline storage is often the same as media used for backup such as tape drives and Optical media. When referring to offline storage we refer to archives, not backups. Archives are typically static whereas backups are typically dynamically changed with each backup. Offline storage still needs to be redundant or backed up, but the schedules for backup are often more lax than with that of other classifications of storage. In a Small or Medium Sized company offline media is often backed up, or duplicated, to the same type of media that it is housed on. There may be two copies of a tape (one onsite and one offsite) or two copies of DVDâ€™s that the data has been burned onto, with each copy stored in a different physical location.
Near-line storage bridges the gap between online and offline storage by providing faster data access than archival storage at a lower cost than primary storage. Firewire Drives are often considered near-line storage because they are slower and usually not redundant. Near-line can refer to recent projects, old financial data, office forms that are updated rarely and backups of online storage to be made readily available for rapid recovery. Backup of Near-line storage will probably be to tape.
Mission Critical data is typically stored in online storage. This data is the day-to-day production data that drives information-based businesses. This includes the jobs being worked on by designers, the video being edited for commercials and movies, accounting data, legal data (for law firms) and current items within an organizations groupware system.
For the small business, Vital and Sensitive data are often one and the same. Vital data is data that is used in normal business practices but can be down for minutes or longer. Sensitive data is often accounting data that a company can live without for a short period of time, but will need to be restored in the event of a loss in a short amount of time. Small business will typically keep Vital and Sensitive data on the same type of media but may have different backup policies for it. For example, a company may choose to encrypt sensitive data and not vital data.
Non-Critical data includes items such as digital records and personal data files of network users. Non-Critical data could also include a duplicate of Mission Critical data from online storage. Non-Critical data often resides on near-line or off-line media (as is the case with Email archives). Non-critical data primarily refers to data kept as part of a companies risk management strategy or for regulatory compliance. This includes old emails and financial records and others.
The chronological method for classifying data is often one of the easiest and most logical. For example, a design firm may keep their mission critical current jobs on an Xraid, vital jobs less than three months old on a Firewire drive attached to a server and non-critical jobs older than three months on backup tapes or offline Firewire drives. It would not be possible to implement this classification without having the data organized into jobs first. Another way to look at this method is that data over 180 days old automatically gets archived.
This characteristic method of data organization means that data with certain characteristics can be archived. This can applied to accounting and legal firms. Whether a client is active or not simply represents a characteristic. If a type of clothing is in style or not represents another possible characteristic. Provided that data is arranged or labeled by characteristic, it is possible to archive using a certain characteristic as a variable or metadata. Many small and medium sized companies are not using metadata for files yet, so a good substitution can be using a file name to denote attributes of the files data.
The hierarchical method of data organization means that files or folders within certain areas of the file system can be archived. For example, if a company decides to close down their Music Supervision department then the data stored in the Music Supervision share point on the server could be archived.
Service Level Agreements
The final piece of the ILM puzzle is building a Service Level Agreement for data management within a company. This is where the people that use each type of data within an organization sit down with IT and define how readily available that data needs to be and how often that data needs to be backed up.
In a Small Business it is often the owners of companies that make this decision. In many ways, this makes coming to terms with a Service Level Agreement easier than in a larger organization. The owner of a small business is more likely to have a picture of what the data can cost the company. When given the cost difference between online and near-line storage, small business owners are more likely to make concessions easier than managers of larger organizations who do not have as much of an ownership mentality towards a company.
Building a good Service Level Agreement means answering questions about the data, asked per classification. Some of the most important questions are:
How much data is there?How readily available does the data need to be?How much does this cost the company, including backups? Given the type of storage used to house this data, how much is it costing the company? If nearly half the data can be moved to near-line storage what will the savings be to the company? In the event of a loss, how far back in time is the company willing to go for retrieval? Is the data required it to be in an inalterable format for regulatory purposes? How fast must data be restored in the event of a loss? How fast must data be restored in the event of a catastrophe? Will client systems be backed up? If so, what on each client system will be backed up?
Information Lifecycle Management
Most companies will use a combination of methods to determine their data classification. Each classification should be mapped to a type of storage by building a SLA. Once this is done software programs such as BRU or Retrospect can be configured for automated archival and backups. The backup/archival software chosen will be the component that implements the SLA, so should fill the requirement of the ILM policies put into place.
The schedules for archival and backups should be set in accordance with the businesses needs. Some companies may choose to keep the same data in online storage for longer than other companies in the same business because they have invested more in online storage or because they reference the data often for other projects. The business logic of the organization will drive the schedule using the SLA as a roadmap.
Setting schedules means having documentation for what lives where and for how long. Information Lifecycle Management means bringing the actual data locations inline with where the data needs to be. Once this has been done, the cost to house and back up data becomes more quantifiable and cost efficient. The SLA is meant to be a guideline and should be revisited at roadblocks and intervals along the way. Checks and balances should be put into place to ensure that the actual data management situation accurately reflects the SLA.
ILM and regulatory compliance are more about people and business process than about required technology changes. The lifecycle of data is important to understand. As storage requirements spiral out of control, administrators of small and medium sized organizations can look to the methods of Enterprise networking for handling storage requirements with scalability and flexibility.