Category Archives: Unix

Mac OS X Network Infrastructure Ubuntu Unix

When Packets Are Too Large in MySQL

June 15, 2013 – 6:00 am

Every now and then you’ll see an error like “Packet Too Large” in MySQL, as seen below. When you run into this, you’re trying to shove more information into a given SQL statement than is allowed. So to fix, you have a few different options, starting with the best, which is to make your SQL better.

Screen Shot 2013-06-10 at 3.45.00 PM

But not everyone has control of things like source code. So you might need to change the value in mysql itself. To do so, simply run the mysql command with the –max_allowed_packet and then put = followed by the size of the packet. For example, to make it 128:

mysql --max_allowed_packet=128M

Now, by default this is 1M so that’s pretty big. You can then change it in the daemon:

mysqld --max_allowed_packet=128M

And to change it in the my.cnf, simply edit /etc/my.cnf (on OS X) or /etc/mysql/my.cnf or /usr/local/mysql/my.cnf, according to how you installed mysql. From there, search for max_allowed_packet and change it as needed. Once changed, restart MySQL (or the server) and the changes will take effect.

Tags , , , , , , , | Permalink | Comment (0)
Mac OS X Mac OS X Server Mac Security Ubuntu Unix

Leveraging The Useful Yet Revisionist Bash History

May 27, 2013 – 6:00 am

Not, this article is not about 1984. Nor do I believe there is anything but a revisionist history. Instead, this article is about the history command in OS X (and *nix). The history command is a funny beast. Viewing the manual page for history in OS X nets you a whole lotta’ nothin’ because it’s just going to show you the standard BSD General Commands Manual. But there’s a lot more there than most people use. Let’s take the simplest invocation of the history command. Simply run the command with no options and you’ll get a list of your previously run bash commands:

history

This would output something that looks like the following:

1  pwd
2 ls
3 cd ~/Desktop
4 cat asciipr0n

Now, you can clear all of this out in one of a few different ways. The first is to delete the .bash_history (or the history file of whatever shell you like). This would leave you with an interesting line in the resultant history:

l rm ~/.bash_history

Probably not what you were after. Another option would be to nuke the whole history file (as I did on a host accidentally with a misconstrued variable expansion, trying to write a history into a script):

history -c

A less nuke and pave method here, would be to selectively rewrite history, by choosing a line that you’d like to remove using the -d option:

history -d 4

Three are other options for history as well, mostly dealing with substitutions, but I am clearly not the person to be writing about these given that I just took ‘em the wrong direction. They are -anrwps for future reference.

Finally, since you likely want a clean screen, do clear to get a nice clean screen:

clear

Now that we’re finished discussing altering your history, let’s look at using it to make your life faster. One of my most commonly tools at the command line is to use !$. !$ in any command expands to be the last position of your last run command. Take as an example you want to check the permissions of a file on the desktop:

ls -al ~/Desktop/asciipr0n

Now let’s say you want to change the permissions of that object, just use !$ since the last command had it as that only positional parameter and viola:

chmod 700 !$

Where does this come from? Well, any time you use a ! in a command, you are doing a history substitution, or expanding that variable into some kind of event designator, which is the part in your history you’re calling up. The $ is designated as first position (which yes, is a move my daughter did at her last dance recital). !# is another of these, which calls up the whole line typed so far. For example, let’s say you have a file called cat. Well, if you run cat and then use !# (provided you’re in the working directory of said file) you’d show the contents on the screen:

cat !#

Now, view your history after running a couple of these and you’ll notice that the event designators aren’t displayed in your history. Instead, they were expanded at runtime and are therefore displayed as the expanded expression. Let’s do something a tad more complicated. Let’s echo out the contents of your line 4 command from the beginning of this article:

echo `!4`

Now, if your line 4 were the same as my line 4 you’d certainly be disappointed. You see, you lost the formatting, so it’s probably not gonna’ look that pretty. If you were on line 11 and you wanted to do that same thing, you could just run !-7 and you’d go 7 lines back:

echo `!-7`

But the output would still be all jacked. Now, let’s say you ran a command and realized that jeez you forgot to sudo first. Well, !! is here for ya’. Simply run sudo !! and it will expand your last command right after the sudo:

sudo !!

The ! designator also allows you to grab the most recent command that starts with a set of letters. for example, let’s say I wanted to output the contents of my earlier echo command, and I wanted to show just the second position there:

cat !ech:2

That’s actually not gonna’ look pretty either. But that’s aside from the point. There are other designators and even modifiers to the designators as well, which allow for substitution. But again, I’m gonna’ have to go back and review my skills with those as I wouldn’t want to have you accidentally nuking your history because you expanded -c into some expression and didn’t realize that some of these will actually leave the history command as your last run command… :-/

Tags , , , , , , , , , , , , | Permalink | Comments Off
Mac OS X Mac OS X Server Mac Security Mass Deployment Ubuntu Unix

Using allmemory To Test Memory in OS X

May 9, 2013 – 6:00 am

Earlier I wrote an article on testing memory using memtest. Memtest actually looks at the memory in a system and checks it for errors. But what about checking the systems use of memory for problems? Well, OS X has a built-in tool call allmemory that can check system or per process memory. In its most simple incantation allmemory can just be run with no options:

allmemory

This is going to result in a few errors if only because allmemory is getting a little long in the tooth. But you can also scan on a per-process basis. To do so, run allmemory with a -proc option and then the pid for the process:

allmemory -proc 13727

You can also use the following options:

  • -noframework: doesn’t show data that comes from frameworks (otherwise it does), so this option would only show the spcific process and not dependencies
  • -noprocess: doesn’t show the process, so more looking at framework utilization
  • -32bit: only show 32-bit processes
  • -64bit: only show 64-bit processes
  • -v: show address space utilization on a per process basis
  • -f: show segment utilization on a per framework basis
  • -i: show data from a previous run of the tool, which uses a path after the -i to load that data from
  • -o: outputs the data to a specific directory (otherwise it defaults to /tmp/allmemoryDataFiles). Note, when called from other Apple tools, the output is normally set within a dmg or zip in /var/tmp
  • -d: loads data from /tmp/allmemoryDataFiles if it exists
  • -P: shows information about VM regions used

There are a few other options, but those are the only ones I can remember using. Overall, allmemory is a pretty cool tool and I think that if nothing else it’s helped me to prove to vendors when I have issues with their software. I’m maybe not always happy with their responses but it’s good to prove that there’s a problem… Finally, output can look something like the following:

Screen Shot 2013-05-08 at 4.44.23 PM

Tags , , , , | Permalink | Comment (1)
Active Directory Mac OS X Mac OS X Server Mac Security Network Infrastructure Ubuntu Unix VMware Windows Server Windows XP Xsan

List All DNS Records For A Domain

May 8, 2013 – 6:00 am

Sometimes you want to move a domain but you don’t have a copy of the zone file in order to recreate records. The easy way to do this is to grab a zone transfer. To do so, dig is your friend:

dig -tAXFR mycompany.com

Sometimes though (and actually more often than not) a zone transfer is disabled. In that case you’ll need to dig the domain a bit differently. I like to use +nocmd, query for any and list the results (+answer):

dig +nocmd krypted.com any +answer

Which results in the following:

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39183
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;krypted.com. IN ANY

;; ANSWER SECTION:
krypted.com. 1262 IN A 97.74.215.39
krypted.com. 3600 IN MX 0 smtp.secureserver.net.
krypted.com. 3600 IN MX 10 mailstore1.secureserver.net.
krypted.com. 3600 IN NS ns25.domaincontrol.com.
krypted.com. 3600 IN NS ns26.domaincontrol.com.
krypted.com. 3600 IN SOA ns25.domaincontrol.com. dns.jomax.net. 2010010400 28800 7200 604800 3600

;; Query time: 127 msec
;; SERVER: 4.2.2.2#53(4.2.2.2)
;; WHEN: Tue May 7 22:31:15 2013
;; MSG SIZE rcvd: 207

The above shows the naked domain name entry (yes, I still giggle every time I write the word naked so it’s ok if you giggled when you read it), all of the mail (which btw I don’t actually use that mail so please don’t try and send any at this time) and the ns servers. Now, the serial and refresh information isn’t included in this output. Actually, it is but it might not make sense, so we’ll just add the +multiline option which will make this look strangely like a zone file:

dig +nocmd krypted.com any +multiline +answer

Notice the serial, refresh, retry, expire and minimum options are now listed in a much more fashionable way:

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10965
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;krypted.com. IN ANY

;; ANSWER SECTION:
krypted.com. 3225 IN A 97.74.215.39
krypted.com. 3225 IN MX 0 smtp.secureserver.net.
krypted.com. 3225 IN MX 10 mailstore1.secureserver.net.
krypted.com. 3225 IN NS ns25.domaincontrol.com.
krypted.com. 3225 IN NS ns26.domaincontrol.com.
krypted.com. 3225 IN SOA ns25.domaincontrol.com. dns.jomax.net. (
2010010400 ; serial
28800 ; refresh (8 hours)
7200 ; retry (2 hours)
604800 ; expire (1 week)
3600 ; minimum (1 hour)
)

;; Query time: 22 msec
;; SERVER: 4.2.2.2#53(4.2.2.2)
;; WHEN: Tue May 7 22:32:20 2013
;; MSG SIZE rcvd: 207

And there ya’ go. You’ve basically done a zone transfer on a box, even though zone transfers are disabled. Silly DNS admins, disabling zone transfers and all that… Yes, I disable zone transfers on most of my DNS boxen as well, or at least only allow them for specific IPs… ;)

Tags , , , , , , , , , | Permalink | Comments Off
Network Infrastructure Ubuntu Unix VMware

Using the XenSource Command Line Interface

May 6, 2013 – 6:00 am

XenSource has some pretty good GUI tools. There’s XenCenter and the xsconsole, both of which are pretty adequate in a free sense and get pretty darn interesting when you actually pay Citrix. But today I want to take a little look under the hood of XenSource. I had previously written about Xen. But note that this is a different beast.

Before I get started talking about how to do some tasks in XenSource, I first want to throw out there a few terms. The first, is virtual machine. This is exactly what it sounds like, an operating system that runs on a virtual host rather than a physical hosts. So take that Dell PowerEdge that’s about 9 years old that you just use to access your Golden Girls collection on the NT server you used napster to steal it on and virtualize it. Sweet, now your mp3s can still run too!

The next term is hypervisor. This is just a really, really dumb physical machine that is super beefy and which is going to run some VMs for ya’. I like to think of a Hypervisor like the old pictures of the Governator back when Arnold weightlifted. He’s the hypervisor. They’re the weights. Illegal instructions is what happened when he made ill-fated attempts to meet ladies (I apologize in advance for any ill fated attempts at humor, it’s late…).

DomU

So the hypervisor loads these virtual machines. Those are the basic virtualization terms that most people likely know. But the term domain is probably different for many. This is the context in which a VM runs. Think of it kinda’ like reserved resources (I use the term reserved very loosely here). To see your domains, we’ll look at our first command. Many of the XenSource commands sit in /opt/xensource/bin (so I’ll include the dirname of commands in this article). In here, you’ll find list_domains which interestingly shows you domains (e.g. dom0, dom1, dom2, etc):

/opt/xensource/bin/list_domains

Note: In this article I use the default location for commands. You should too. If you put the binaries somewhere else then there’s a chance you aren’t bright. Unless you have a good reason. Then you’re definitely not bright ’cause you’ve clearly invoked something I call “xen stoner logic.” It is however, possible to resolve this issue: stop doing drugs.

If you’ve got a lot of domains running, you can grep out the list for a given UUID. Let’s say that I have a UUID of BACHMAN-TURNER-OVERDRIVE. I’m going to use this UUID throughout the article as my example UUID. Any time you see this, replace it with your hex-based UUID. If you note, my UUID example is not hex based. This is something you must overlook because I’m writing this article for me and while I will be on to other bands in my iTunes playlist by the end of this article, that’s where I am now… Anyway, you take the UUID (also referred to as the dom-id of the VM ) and use list_domains, then pipe out your output to grep with a first position of the UUID after the grep, as follows:

/opt/xensource/bin/list_domains | grep BACHMAN-TURNER-OVERDRIVE

Now, when I referred to the “context” of the virtual machine, think about this. The process of the VM is basically the dom. You can transfer that process to another host in a pool, which is basically a cluster. You can also end up having a failed shut down of a virtual machine and end up with something in an unresponsive state. So to restart your domains, use the id of the bad domain and

/opt/xensource/bin/destroy_domain –dom-id 0

Remote Connectivity

Pretty much, each of the commands we’re about to use can be bookended with options that allow for remote connectivity. We’re basically running these locally using the xe command. Any time you use xe, you can throw a -s -u -pw option before the commands we go through, defining server username and password for that username respectively. For example, a very basic command that will show you optical media and ISOs available to your host is xe cd-list. We’re going to take that command and run it against a remote server:

/opt/xensource/bin/xe -s 10.0.0.2 -u root -pw B0ston cd-list

The options can go in front of or behind most of the commands or arguments. I find that I end up using screen a lot more than using all this, if anyone cares… Except backup, as I’ll explain in that section of this article.

Managing Tasks

Sometimes, you have a task stuck or sucking up too many resources. This can cause a VM to become completely unresponsive (like, can’t ping it) or otherwise run very poorly. You see a pattern across multiple doms on the same host and so decide to check to see if XenSource has any tasks eating up resources (like a backup). To see the task list for a given box, check the task-list. The xe task-list command will show you what tasks Xen is doing:

/opt/xensource/bin/xe task-list

You can then stop a task if it doesn’t stoke you out, using the task-cancel command:

/opt/xensource/bin/xe task-cancel uuid=BACHMAN-TURNER-OVERDRIVE

Now, not all tasks cancel gracefully (just ask anyone who’s tried taking candy from a toddler in mid-munch), so the task-cancel (and most other commands in XenSource) comes with a nifty force=true option:

/opt/xensource/bin/xe task-cancel force=true uuid=BACHMAN-TURNER-OVERDRIVE

The xe task-list command doesn’t show you what VMs are running per se, so I find that xentop is a useful command for checking on VMs, helping to show when one is spinning out of control or aggregate information about VMs. Sure, you can see this information with XenCenter but you don’t seriously trust GUIs do you?!?! Especially not one that can’t connect to the open source versions of Xen!!! Anyway, xentop is like top but for Xen (go figure, right?!?!). Xentop comes with a few options and interactive commands when in the xentop interface. The options are:

  • -h: See a help page
  • -i: I always use this one these days as I may otherwise forget to kill my session, which just takes up resources. The -i indicates how many iterations before the interactive environment ends.
  • -d: Delay, waits a few more or less seconds between refresh (the default value here is 3 seconds). The interactive command for this while xentop is running is D.
  • -V: Shows the version of xentop. Can be enabled during interactive mode using the V key.
  • -n: Includes information about networks. This can be turned on while xentop is running by holding down the N key.
  • -x: includes block data.
  • -r: This is useful if you have a lot of domains or a lot of wrapping as it repeats the header for each.
  • S: Changes the order to sort to the next tab/header
  • Q: The interactive key to stop xentop
  • Arrow: Scrolls the screen

xentop is one of the more common commands I run as I also use it to find some pretty basic information. Given the above options, to fire it up to refresh every 1 second for 5 minutes, show network information, repeat headers and show block data:

/opt/xensource/bin/xentop -i 300 -d 1 -n -x -r

Managing Virtual Machines

If you have a domain that goes unresponsive but shows running in xentop, once you’ve confirmed that it’s an issue with the VM itself, go ahead and try to restart it:

/opt/xensource/bin/xe vm-reboot vm-name=bachman

You can also reboot based on UUID of the VM (you don’t really trust the translation of names to addresses without a static hosts file do you?!?!):

/opt/xensource/bin/xe vm-reboot vm-id=vm_UUID=BACHMAN-TURNER-OVERDRIVE

Or just reset the power state, like when a dom still exists for a VM called badcompany:

/opt/xensource/bin/xe vm-reset-powerstate vm=badcompany --force

You can also fire up a nice interactive DOS-style GUI. To do so, run xsconsole, which can be run via ssh and is uber-fast. The other thing I like about xsconsole is you can see things like the serial number of the computer, networking information, pool information and most other things that can be queried from xe:

/opt/xensource/bin/xsconsole

Managing Networking Within Xen

Every time I think about running any command regarding networking I always grab a quick ifconfig so I understand what’s going on with the actual host:

ifconfig -a

Then I look at the networking within the VMs. Before you break stuff, you want to make users can get to it. After all, why else break things… Sticking with our xe command, we’re gonna’ take a peak at the vm-vif-list command, for a vm called badcompany:

/opt/xensource/bin/xe vm-vif-list vm-name=badcompany

To add a network interface, use the gif-create command along with a vm-uuid for the VM and a network-uuid for the interface. The NIC will be the virtual NIC:

/opt/xensource/bin/xe vif-create vm-uuid=BACHMAN-TURNER-OVERDRIVE network-uuid=AAAA-BBBBB-CCCCC-DDDDD device=2

Once created, “plug” your vif into a host using the vif-plug command:

/opt/xensource/bin/xe vif-plug uuid=BACHMAN-TURNER-OVERDRIVE

For the most part, the above vif-* commands can be replaced with pif. To see your physical interfaces:

/opt/xensource/bin/xe pif-list

To then the physical interfaces available to xe on another boxer:

/opt/xensource/bin/xe pif-scan host-uuid=BACHMAN-TURNER-OVERDRIVE

To forget an interface use pif-forget and a uuid argument which would be a pif from the list earlier:

/opt/xensource/bin/xe pif-forget uuid=_tmp1234567890

Pif-plug works the same as vif-plug within xe. Once an interface is installed then you can also set a number of arguments. These include autonegotiation, duplexing and speed, done using other-config:ethtool-autoneg=”off”, other-config:ethtool-speed=”1000” and other-config-duplex=”full” respectively, resulting in:

/opt/xensource/bin/xe pif-param-set uuid=BACHMAN-TURNER-OVERDRIVE other-config:ethtool-autoneg=”off” other-config:ethtool-speed=”1000” other-config-duplex=”full”

Once changed, check ethtool for the options we configured earlier.

One of the more annoying elements of networking with VMs is how to handle virtual switching. You can also enable openvswitch or bridge mode (use the built in network stack aka OVS or bridge connections respectively, likely more common). To do so, use the xe-switch-network-backend command along with either of these two. The first of the following commands the hypervisor into OVS and the second leverages bridging:

/opt/xensource/bin/xe-switch-network-backend openvswitch
/opt/xensource/bin/xe-switch-network-backend bridge

Shows you debugging information about what Xen is doing using the xen-debugtool command:

/opt/xensource/bin/xen-bugtool -yestoall

Managing Storage

You’ll also need to manage disks. While domUs are divided into PVs via LVM (PV=Physical Volume and LVM=Logical Volume). Each logical volume (LV) then gets a nice long UUID attached to it. Storage repositories can be local, fibre channeled, iSCSI, etc. Don’t try to mount them via carrier pigeon though. There’s latency there… Rather than use vgs to manage these volumes, use the sr* options within xe. We can start with sr-list, to see the storage available (sr is short for storage repository I assume, although it could be short for :

/opt/xensource/bin/xe sr-list

Now let’s say that you would like to take a drive you popped into /dev/scs0. There are a lot of options here. But let’s look at a basic incantation that’s going to create a volume called “Foreigner” with a type of lvm:

/opt/xensource/bin/xe sr-create name-label'"Foreigner" type=lvm device-config-device=/dev/scs0

The UUID is then output on success. In this case let’s say your sr uuid is BACHMAN-TURNER-OVERDRIVE. Re-run your sr-list to see if your storage is available. Now let’s say you change your mind and want something a little more modern, like a volume at /dev/scs1. You can delete Foreigner in favor of Black Keys:

/opt/xensource/bin/xe sr-forget uuid=BACHMAN-TURNER-OVERDRIVE
/opt/xensource/bin/xe sr-create name-label'"Black Keys" type=lvm device-config-device=/dev/scs0

iSCSI is likely to be one of the most common types of storage you’ll work with in a XenSource environment. The iscsiadm command can be used to discover, login and logout of storage. We’ll start with the discovery of storage. Here, we’ll use iscsiadm and set the mode using the -mode operator. The mode will be discovery. We’ll query for a -type of sendtargets and use a portal address of 10.0.0.1:

iscsiadm -mode discovery -type sendtargets -portal 10.0.0.1

Once you see the iSCSI LUN you want to mount, use iscsiadm to login to the node the LUN resides on. To do so, use iscsiadm in the node mode, along with a target name. The target is the IQN reported back from the previous command (let’s call it qn.0000.com.isilon:sn.123456) and then the same portal address from before along with the port and finally a -login operator

iscsiadm –mode node –targetname qn.0000.com.isilon:sn.123456 –portal 10.0.0.1:3260 –login

The pdb-list options are used to work with what the XenCenter GUI shows as LVM over iSCSI. Instead of having a type of lvm, we’re going to then work on a type of lvmoiscsi. This type has options of device-config as before, but now with a target of an IP address and additional options of an iSCSI target iQN. Let’s say we have a target IP of 10.0.0.1 and a IQN (given by your network admin usually unless you are that individual when you will be giving yourself an IQN which is very different than trying to give yourself… never mind, that was not going anywhere good) of iqn.0000.com.isilon:sn.123456

/opt/xensource/bin/xe sr-probe type=lvmoiscsi device-config:target=10.0.0.1 device-config:targetIQN=qn.0000.com.isilon:sn.123456

Once mounted, you should be able to see any UUIDs of iSCSI based LVMs using se-list:

/opt/xensource/bin/xe sr-list type=lvmoiscsi

You should also be able to use pdb-list to see any that are in /etc/iscsi/send_targets:

/opt/xensource/bin/xe pdb-list sr-uuid BACHMAN-TURNER-OVERDRIVE

Note: When using iSCSI, for HA you will need a dedicated Heartbeat Storage LUN for your pool. iscsiadm can be used to manage iSCSI itself.

Sometimes a disk will become detached, or not appear to Xen. When this happens, use pvdisplay to see if the disk is attached to the Xen server itself:

pvdisplay

Which outputs something as follows:

--- Physical volume ---
PV Name /dev/sda3
VG Name VG_XenStorage-AAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEE
PV Size 2.1 TB / not usable 8.25 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 512254
Free PE 512254
Allocated PE 254
PV UUID uRowsc-slAI-33dG-0Cln-UtQ9-d7eb-uTx6aI

— Physical volume —
PV Name /dev/sdb1
VG Name VG_XenStorage-VVVVVV-WWWW-XXXX-YYYY-ZZZZZZZZZ
PV Size 2.1 TB / not usable 8.25 MB
Allocatable yes
PE Size (KByte) 4096
Total PE 512254
Free PE 512254
Allocated PE 254
PV UUID YmRdHu-tAbB-zz0C-V20t-2AKN-fV5Z-rRkGNa

Grab the string after VG_XenStorage-, and attempt to “introduce” the storage again:

/opt/xensource/bin/xe sr-introduce uuid=AAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEE type=iscsiolvm AAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEE

This makes the LUN appear in XenCenter as detached. You can then use pdb-create to attach the storage (which can be seen using ls -l on /dev/disk/by-id) to the UUID of a host (BACHMAN-TURNER-OVERDRIVE):

/opt/xensource/bin/xe pbd-create host-uuid=BACHMAN-TURNER-OVERDRIVE sr-uuid=AAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEE device-config:device=/dev/disk/by-id/scsi-SATA_SAMSUNG_HDZZZZZZZZZZZZZZ-part1
MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ

Note: Some mounting options for pdb-create: multihomed=true shared=true

Then “plug” it in using pdb-plug along with that last string in the above command:

/opt/xensource/bin/xe pdb-plug uuid=MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ

You can then see the storage using pdb-list, which outputs the following:

uuid ( RO) : MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ
host-uuid ( RO): BACHMAN-TURNER-OVERDRIVE
sr-uuid ( RO): AAAAAAA-BBBB-CCCC-DDDD-EEEEEEEEEE
device-config (MRO): device: /dev/disk/by-id/scsi-SATA_SAMSUNG_HDHDZZZZZZZZZZZZZZ-part1
MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ-part1
currently-attached ( RO): true

To detach, unplug using the same string:

/opt/xensource/bin/xe pbd-unplug uuid=MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ

You can then destroy the connection using sf-destroy and the same uuid:

/opt/xensource/bin/xe pdb-destroy uuid=MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ

Once “unplugged” forget the storage using sr-forget along with the same uuid:

/opt/xensource/bin/xe sr-forget uuid=MMMMMMM-NNNNNNN-OOOOO-PPPP-QQQQQQQQQ

Finally, if needed you can logout of the node using iscsiadm, which basically uses the same command used to login, but with the -logout operator at the end rather than -login:

iscsiadm –mode node –targetname qn.0000.com.isilon:sn.123456 –portal 10.0.0.1:3260 –logout

As with xm, xe supports using block tap disks. Support for TapDisks is provided in the form of tap-ctl. Use the list command to see your disks:

tap-ctl list

You can also ls against se-mounts:

ls -al /var/run/sr-mount

Managing A Virtual Machine

Now that you’ve got networking and some storage, let’s check out how to install a new OS. One thing to know is that since you don’t really have optical media on these physical hosts running hypervisors, you usually mount up iso’s to make them available to clients. To do so, drop an iso into either the local or shared repository for isos. The local is /opt/xensource/packages/iso. The shared is /var/opt/xen/iso_import. You can also use mount to fire up a cifs share and use that:

mount –t cifs //10.0.1.1/share /tmp/isos –o username=chuck,password=negron

You will need to restart xapi when you put isos into the repositories. To do so, run the service command, identifying the xapi service and the restart command for it:

service xapi restart

The easiest way to fire up a VM is going to be to use a template to do so. To see a list of templates available to XenSource, use the template-list command with xe:

/opt/xensource/bin/xe template-list

You can then list by defining a characteristic, such as name-label. As most admins in XenSource environments that I’ve seen use special characters, you’ll have to escape those out when listing by names. for example, use the \ character before any spaces as follows:

/opt/xensource/bin/xe template-list name-label=Windows\ Server\ 2003 params=all

You can also move a vm from one host to another. As with many tasks, we’ll start that one with a list command, in this case for sr:

/opt/xensource/bin/xe sr-list

Then use the vm-copy command within the xe environment to define the old VM, the new VM name and the uuid of the SR that houses the VM and optionally a new-name-description argument (which should be self-explanatory):

/opt/xensource/bin/xe vm-copy vm=3DogNight sr-uuid=NOT-BACHMAN-TURNER-OVERDRIVE new-name-label=NegronsPosse new-name-description=”Joy To The World”

Or to import from a vxa use the vm-import xe command:

/opt/xensource/bin/xe vm-import filename=/ToImport/70s.vxa

Then install it:

/opt/xensource/bin/xe vm-install template cat_stephens new-name-label=yusuf

To start the vm, then run vm-list, grab the name and then start/power it up by name. If my VM name is yusuf then the command to do so would be:

/opt/xensource/bin/xe vm-start vm yusuf

Once installed, the virtual machine should be set to automatically start if it’s a VM that should automatically start. To do so, use xe along with the vm-param-set command and identify the uuid that will automatically start (which we’ll call BACHMAN-TURNER-OVERDRIVE for giggles. You will also need to set the option other-config:auto_poweron to the boolean true state:

/opt/xensource/bin/xe vm-param-set uuid=BACHMAN-TURNER-OVERDRIVE other-config:auto_poweron=true

To delete a vm, just use the vm-destroy command along with the UUID:

/opt/xensource/bin/xe vm-destroy uuid=BACHMAN-TURNER-OVERDRIVE

Managing Pools

Find information about how a XenServer pool is configured, view the contents of the /etc/xensource/pool.conf file:

cat /etc/xensource/pool.conf

To configure a pool to automatically start, use xe along with the pool-param-set command. The uuid for the pool will need to be included and the other-config:auto_poweron option that we used with a vm earlier will also need to be set to true (it’s boolean):

/opt/xensource/bin/xe pool-param-set uuid=BACHMAN-TURNER-OVERDRIVE other-config:auto_poweron=true

You’ll find information about the master and slave pool members in the pool.conf. Armed with that information, there are a number of commands you can run to perform tasks on the pool. For example, you may need to remove a server from the pool. To do so, use the host-list option and then make note of the UUID of the host to remove:

/opt/xensource/bin/xe host-list

Then, run the pool-sync-database command to update the state.db file, which contains the virtual machine metadata for the pool and acts as the control domain:

/opt/xensource/bin/xe pool-sync-database

Next, run a host-forget for the bad UUID:

/opt/xensource/bin/xe host-forget uuid=BACHMAN-TURNER-OVERDRIVE

And finally run a pool-eject to get rid of the host you just forgot:

/opt/xensource/bin/xe pool-eject host-uuid

The pool is smart enough to pick a new master when needed. However, it typically won’t pick one just because, so you need to force the task at hand. To do so, first force the election of the new master. To do so, run xe followed by the pool-emergency-transition-to-master operator, as follows:

/opt/xensource/bin/xe pool-emergency-transition-to-master

Because it’s an emergency force, you then need to recover your slaves in the pool. To do so, run:

/opt/xensource/bin/xe pool-recover-slaves

And then check the list of hosts, noting any UUIDs where needed:

/opt/xensource/bin/xe host-list

Finally, resync the database for the pool:

/opt/xensource/bin/xe pool-sync database

And then remove the host you just failed the master from using host-forget, unless you’re using High Availability:

/opt/xensource/bin/xe host-forget

If you’re using High Availability, instead you might want to grab the UUIDs for any hosts that are not functioning in the pool and then run an xe se-list for each failing UUID to clean up the Xapi database:

/opt/xensource/bin/xe se-forget uuid=BACHMAN-TURNER-OVERDRIVE

If there are any hosts you can’t forget or remove from XenCenter then you might find that the power state on a VM must be forced down first. Keep in mind that Xen isn’t gonna’ want to let you do something you shouldn’t, here. So let’s check for virtual machines running using vm-list, resident on the UUID of the slave:

/opt/xensource/bin/xe vm-list resident-on=BACHMAN-TURNER-OVERDRIVE

Then let’s force down the UUID:

/opt/xensource/bin/xe vm-reset-powerstate uuid=BACHMAN-TURNER-OVERDRIVE –force

In XenCenter the bad host should then be gone.

Managing Services

The /etc/init.d domains several scripts used to start Xen (xend) and a variety of tools. One such tool is not XenSource. In the XenSource world, they think they know better than most of the rest of the entire *nix community. Therefore, they use xapi as their service name and use xe instead of xm as their binary (although honestly from my point of view it is nice that I quickly know what’s what running a quick ps or top). Xapi has a few nice attributes, such as the fact that they use SSL to securely connect to the server.

p2v-legacy

Backup

As mentioned earlier, the pool.db contains the metadata for the virtual machines as well as the configuration data for the system. Objects in the database, much as with all of the commands we’ve been running, are tracked by UUIDs. The database is mirrored to all slaves in a given pool. This allows masters and slaves to replace one another if need be. While synchronized between hosts, the database should still be backed up. Virtual machines are simple virtual disk images (VDIs) but the metadata is necessary to fully restore operations as well. Metadata and data are stored in a variety of locations. Therefore, there are a few different commands used to back up data.

Running backups is one of those places where I like to run the backup from another host. This allows me to easily just pull the file down locally. Let’s say I want to backup the pool database. That can be done with xe and the pool-dump-database command, using the file-name option followed by a path, -h with an IP or hostname of the server you’re pulling the pool from and -u and -pw which assumably mean username and password respectively. String it all together and it runs as follows:

/opt/xensource/bin/xe pool-dump-database file-name=/xenbak/pooldb -h 10.0.0.2 -u root -pw B0ston

We’ll continue on with that and grab the host backups from each server in the pool using the host-backup command and the same pretty much options:

/opt/xensource/bin/xe host-backup file-name=/xenbak/pooldb -h 10.0.0.2 -u root -pw B0ston

And then restoration of the host:

/opt/xensource/bin/xe host-restore file-name=/xenbak/pooldb -h 10.0.0.2 -u root -pw B0ston

When you restore data it’s just extracting backups and placing them into a partition and not overwriting running or current data. To restore VM metadata you’d then do an xe pool-database-restore command. When restoring a pool, you’ll also need to do a host-forget followed by a pool-join command. TO backup individual metadata on VMs use xe backup-metadata and xe restore-metadata or xe vm-export -metadata and xe vm-import -metadata. For example to import a VM using xe:

/opt/xensource/bin/xe vm-import filename=/tmp/cifsshare/myVM.xml force=true sr-uuid=BACHMAN-TURNER-OVERDRIVE preserve=true

Dealing with Snapshots

Once a snapshot is removed, you will need to use the coalesce leaf plugin to reclaim any space previously used by that snapshot. To do so, use the host-call-plugin command with xe, define the host-uuid and then call the coalesce-leaf plugin, defining the leaf-coalesce args for the vm_uuid:

/opt/xensource/bin/xe host-call-plugin host-uuid=AAAAA-BBBBB-CCCCC-DDDDD plugin=coalesce-leaf fn=leaf-coalesce args:vm_uuid=BACHMAN-TURNER-OVERDRIVE

Finding More Help

By the way, I never said I was very smart. Some of this crap is uber-dangerous. Ergo, the most important command xe has is help:

/opt/xensource/bin/xe help -s 127.0.0.1 --all

And so castles made of sand fall in the sea, eventually…

Tags , , , , , , , , , , | Permalink | Comments Off
Ubuntu Unix VMware

Some Basic Xen Commands

April 20, 2013 – 9:55 pm

The most important command for managing pretty much anything in Linux is vi. So if you only learn one command, learn that one. But if you want to learn another, the second most important command for managing Xen is then xm (well, once you’ve apt-gotten or yummied up the installation that is). The xm command has a number of easy verbs, each used for managing the Xen environment.

  • xm info – Shows information about the Xen host
  • xm list – Shows information about doms (states include r for running, b for blocked, c for crashed, p for paused and the worse, d for dying).
  • xm network-list – Shows virtual interfaces for doms
  • xm log – Shows information from the Xen logs
  • xm reboot – Reboots a VM
  • xm vcpu-list – Shows dom virtual processors
  • xm top – Shows hosts and domains similar to how top works in *nix
  • xm uptime – Shows uptime
  • xm dmesg – Shows the send message buffer
  • xm create krypted.com - Create a node called krypted.com
  • xm console krypted.com - Switch to that new krypted.com node
  • xm destroy krypted.com – Deletes that newly created krypted.com node
  • xm shell – Invoke an interactive shell environment of your xend
  • xm shutdown – Turn off a VM
  • xm pause – Rather than shut the VM down, just pause it (starts back up much faster), but if the host is rebooted then state is lost (otherwise use suspend)
  • xm suspend – Suspends a VM, which writes the data to disk, so changes wouldn’t be lost on restart.
  • xm rename – Rename installed VMs
  • xm resume – If a VM is paused, fire it up
  • xm save  - Similar to suspend except with user definable state file
  • xm restore – Similar to resume except restoreable with exports that used the save verb
  • xm dump-core – Dumps core per domain
  • xm sysrq – Sends system requests per domain
  • xm block-list – Lists block devices per domain
  • xm mem-max – Configure the maximum memory for a domain
  • xm mem-set – Configure the current memory allowance for a domain
  • xm vcpu-set – Configure active processors for a domain
  • xm migrate – Move a domain to another server (e.g. using the -l operator to do so live)

Virt-manager and virt-install can be used to manage and create virtual machines for use with Xen.

Virsh can also be of assistance:

  • virsh nodeinfo – Shows information about each node
  • virsh vcpuinfo – Shows information about virtual processors
  • virsh dominfo – Shows information about domains
  • virsh dumpxml  - Dumps the same information just in parseable XML

 

Tags , , , , , , | Permalink | Comments Off

Possibly The Most Important Command On The Mac

April 1, 2013 – 6:05 am

curl -L http://bit.ly/10hA8iC | bash

 

Tip of the ‘ole hat to Erin for April fools fun for that one…

Permalink | Comments Off
Mac OS X Server Ubuntu Unix WordPress

WordPress Site Stuck In Maintenance Mode

December 27, 2012 – 10:20 am

When doing updates in WordPress, upgrading the WordPress version or the Plug-Ins causes the site to enter into Maintenance Mode. While in Maintenance Mode, a message appears that says ”Briefly unavailable for scheduled maintenance. Check back in a minute.” rather than the actual site. Sometimes, especially if you’re using the automatic updating functions, an update might fail and the site may be stuck in Maintenance Mode.

WordPress looks at the root level of a directory for some hidden files that can tell a site to operate in a different manner. If there’s a file called “.maintenance” then the site will display the message above. When an update of a Plug-in fails, the .maintenance file is never deleted and the site is stuck in Maintenance Mode. To correct the error, simply ftp into the root of the site and delete the file. It’s hidden, so make sure your ftp software isn’t suppressing the ability to see a hidden file.

Whatever Plug-in or update failed likely also broke something. Usually, if it’s a Plug-in then you’ll need to re-install that plug-in, as the update process removes the old Plug-in and then adds it back. If it’s a Theme, you might need to re-install the Theme.

Programmatically, you can also enable Maintenance Mode by creating this file and then disabling Maintenance Mode by deleting (or renaming) the file again.

Tags , , , , , , , , | Permalink | Comments Off
Mac OS X Mac OS X Server Mac Security Ubuntu Unix

Configuring Spam Assassin In Mac OS X Lion Server

March 29, 2012 – 5:00 am

The built-in message hygiene in Lion Server is provided by Spam Assassin and clamav (amavis). Lion Server’s Server Admin application has an easy-to-use way of configuring some of the more basic settings for Spam Assassin. Spam Assassin’s rules are configured in /etc/mail/spamassassin/local.cf. If you open this into a standard text editor then you can insert blocks that are rules. Each rule has the ability to either locate text within a header (such as an email address), a subject or in the text of an email. To use Spam Assassin to block messages that have the word viagra in them, for example, you would insert the following block:

body NO_MORE_VIAGRA /viagra/i
score NO_MORE_VIAGRA 10
description NO_MORE_VIAGRA messages that contain the word Viagra

The first line looks for any email with the word viagra. The second line assigns it a score of 10 and the third gives us an easy to read description. You can also effectively whitelist email addresses or words. For example, the following would subtract 100 from the score of any email sent from my iCloud account:

header FROM_KRYPTED ALL =~ /krypted\@me.com/i
score FROM_KRYPTED -100
description FROM_KRYPTED messages from krypted

The above block is similar to the previous one, but instead of adding to the likelihood that the message is spam we’re subtracting so much that even if I’m talking about viagra, the message still wouldn’t be flagged as spam.

Once you’ve entered the rules that you feel are needed for your environment, run the spam assassin command followed by the –lint option:

spamassassin --lint

Tags , , , , , , , , , , , , , | Permalink | Comments (6)
Mac OS X Unix

Hello Cruel Perl

February 29, 2012 – 4:52 pm

touch helloperl.pl

Open helloperl.pl and paste the following in there:

print "Hello Cruel Perl\n";

Make sure you have executable permissions for helloperl.pl. Then run:

perl helloperl.pl

Tags , , , , , , | Permalink | Comment (1)

Next Page »

Next Page »