Category Archives: VMware

Active Directory Mac OS X Mac OS X Server Microsoft Exchange Server Network Infrastructure Ubuntu Unix VMware Windows Server

Stashbox: Turning a Mac Mini Into A Logstash and Kibana Server

You have a lot of boxes. You would like to be able to parse through the logs of all those boxes at the same time, searching for a given timestamp across a set of machines for a specific string (like a filename or a port number). elasticsearch, logstash and kibana are one way to answer that kind of need. This will involve downloading three separate packages (which for this article, we’ll do in /usr/local) and creating a config file.

First, install the latest Java JDK. This is available at jdk8-downloads-2133151.html.

The following is going to download the latest version of logstash and untar the package into /usr/local/logstash (I like nesting that logstash-1.4.0 inside logstash so when the next version comes out I can have it there too, I have plenty of space so keeping a couple versions back helps in the event I need some old binary and can’t get to it ’cause they revved out the version I wrote a script against at some point):

curl -O https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz
mkdir /usr/local/logstash
tar zxvf logstash-1.4.0.tar.gz -C /usr/local/logstash

Once we have log stash, we’ll grab elastic search similarly:

curl -O https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.1.tar.gz
mkdir /usr/local/elasticsearch
tar zxvf elasticsearch-1.0.1.tar.gz -C /usr/local/elasticsearch

Then we’ll untar kibana in the same manner:

curl -O https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
mkdir /usr/local/kibana
tar zxvf kibana-3.0.0.tar.gz -C /usr/local/kibana

Next we’ll make a very simple config file that we call /usr/local/stashbox.conf that listens on port 514 for syslog:

input {
tcp {
port => 514
type => syslog
}
udp {
port => 514
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}

Next, we’ll enable elastic search:

/usr/local/elasticsearch/elasticsearch-1.0.1/bin/elasticsearch

And finally, in a different window we’ll call logstash with that file as the config file:

/usr/local/logstash/logstash-1.4.0/bin/logstash -f /usr/local/stashbox.conf

Having each of these open in different Terminal windows allows you to see logs in stdout. Next, point a host at your new syslog box. You can use http://krypted.com/windows-server/use-syslog-on-windows for installing Windows clients or http://krypted.com/mac-security/redirect-logs-to-a-syslog-server-in-os-x/ for  a Mac. Once done, let’s get Kibana working. To do so, first edit the config.js.

vi /usr/local/kibana/kibana-3.0.0/config.js

Locate the elastic search setting and put the name of the host running logstash in there (yes, it can be the same as the actual logstash box as long as you install a web server on the logstash box). Then save the changes.

Now move the contents of that kibana-3.0.0 folder into your web directory. Let’s say this is a basic OS X Server, that would be:

cp -R /usr/local/kibana/kibana-3.0.0/* /Library/Server/Web/Data/Sites/Default/

You can then check out your Kibana site at http://localhost or http://localhost/index.html#/dashboard/file/logstash.json for the actual search pages, which is what I’ve bookmarked.

Screen Shot 2014-04-10 at 10.37.51 PM

For example, to see the impact of periodic scripts in System Logs:

Screen Shot 2014-04-12 at 9.07.44 AM

 

Mac OS X VMware Windows Server Windows XP

Create A Server 2012 VM In VMware Fusion

Our friends at VMware continue to outdo themselves. The latest release of Fusion works so well with Windows Server 2013 that even I can’t screw it up. To create a virtual machine, simply open VMware Fusion and click New from the File menu.

Screen Shot 2014-04-06 at 3.43.26 PM
Click “Choose a disc or disc image.”

Screen Shot 2014-04-06 at 3.43.58 PM

Select your iso for Server 2012 and click on Open (if you have actual optical media it should have skipped this step and automatically sensed your installation media). Click Continue back at the New Virtual Machine Assistant screen.

Screen Shot 2014-04-06 at 3.45.26 PM

Click Continue when the Assistant properly shows the operating system and version.

Screen Shot 2014-04-06 at 3.50.07 PM

Enter a username, password and serial number for Windows Server if you want Fusion to create these things automatically and just complete an installation. If not, uncheck Easy Install (but seriously, who doesn’t like easy). Also, choose the version of Windows Server (note that there’s no GUI with the Core options). Click Continue.

Screen Shot 2014-04-06 at 3.50.55 PM

At the Finish screen, you can click Customize Settings if you would like to give the new virtual machine more memory or disk. Otherwise, just click Finish.

Screen Shot 2014-04-06 at 3.52.00 PM

When prompted, choose where the new virtual machine will live and click Save. The VM then boots into the Setup is starting screen. You will be prompted for a Core vs. a GUI install (I know, you picked that earlier). I choose a GUI, then click Next.

Screen Shot 2014-04-06 at 3.53.28 PM

When the setup is complete, login, run Software Update and you’re done!

VMware Windows Server

Hyper-V: Convert .vhd Files to .vhdx

The vhdx format provides support for 2 terabyte drives, can difference dynamic disks faster and provides more options for sector sizes. When upgrading into Server 2012, you can migrate your vhd files to vhdx files using Hyper-V Manager. To do so:

  1. Open Hyper-V Manager
  2. Click on the Action pane from the Action menu
  3. Click Edit Disk…
  4. At the Edit Virtual Disk Wizard click Next
  5. Provide the name of the location for the vhf file
  6. Click Next at the Location pane
  7. Click Convert at the Choose Action pane
  8. Click Next
  9. Click the vhdx format at the Choose Disk Format
  10. Click Next until the wizard is complete

You can also use the convert-vhd powershell cmdlet to covert these vhd files. It’s an easy powershell command to use. Simply  specify the source as your first positional parameter and the target as the second, as follows.

convert-vhd V:\myvm.vhd V:\myvm.vhdx

The fact that you can manage these from a script also opens up the ability to automate the conversion for a number of files concurrently.

VMware Xsan

Resolve Error 1006.0005 For Qlogic Switches

Error 1006.0005 can appear on a Qlogic fibre channel switch when using ACL zones. If you don’t need ACL zones, then the easiest thing to do here is to swap the offending zone back to a soft zone. To do so, open the Qlogic Switch and use the Edit menu to select “Edit Zoning …”

Screen Shot 2014-03-05 at 2.12.57 PM

From the zone editor, right-click on the zone to change and click on Set Zone Type.

Screen Shot 2014-03-05 at 2.17.24 PM

From the Set Zone Type pop-up, click on the option for Soft.

Screen Shot 2014-03-05 at 2.18.37 PM

Save the zoning and provided that you can actually use soft zones you are done. Now, what if you can’t use soft zoning? In that case, I find that this error specifically comes up when you have a device in a soft and ACL-based zone. To rectify that, either switch the soft zone to ACL or define the port in the ACL zone and the WWN in the soft zone.

VMware

vSphere: The Datastore Browser

When you’re moving virtual machines around, you’ll frequently use a tool such as vMotion. But what happens when you’re trying to load new virtual machines into VMware from the .vmdks on a client system or trying to archive a virtual machine that isn’t actually destined for another host? You can use nfs or ssh to access an ESX host, but there’s an even simpler way: the Datastore Browser.

To use the Datastore Browser, first login to the vSphere Client. If you’ll be archiving a virtual machine, from there, I would stop the virtual machine. Then click on the virtual machine in the sidebar and click on Summary to see the Resources available to the VM. In storage, right-click on the datastore that houses the virtual machine and click on Datastore Browser.

Screen Shot 2014-01-23 at 4.32.17 PM

The Datastore Browser opens. From here, you can browse assets, including the vmdk files, vmx files and logs. Click on one (or shift-click on many) and then click on the download button in the toolbar (alternatively click none and click on the upload button if you’d like to upload something).

Screen Shot 2014-01-23 at 4.32.36 PM

The download option brings up a browser so you can choose where to drop your assets off. Once done, you can deprovision storage or simply delete assets as needed.

iPhone Mac OS X Mac OS X Server Mac Security Mass Deployment Microsoft Exchange Server Network Infrastructure Ubuntu Unix VMware

Quick nmap Hacks

The nmap application is a pretty easy-to-use tool that can be used to port scan objects in a network environment. To obtain mmap in an easy-to-use package installer, for OS X check out the download page at http://nmap.org/download.html#macosx (use the same page to grab it for Windows or *nix as well). Once downloaded run the package/rpm/whatever.

Before I scan a system, I like to pull the routing table and eth info to determine how scans are being run, which can be run by using the mmap command anong with the —iflist option:

nmap —iflist

Basic Scanning
To then scan a computer, just use the mmap command followed by the host name or even throw a -v option in there to see more information (you can use a hostname or an IP):

nmap -v www.apple.com

Use the -6 option if scanning via IPv6:

nmap -v -6 8a33:1a2c::83::1a

Can drop the -v for less info on these, but I usually like more than less. Shows ports, states, services (for the ports) and a MAC address for each IP being scanned.

You can also scan a range of IPs. I usually take the lazy way for this, by using a wildcard. I can replace an octet to scan all objects in that octet. For example, to scan all systems running on the 192.168.210 class B:

nmap 192.168.210.*

You can scan a subnet, which can cover more or less than one octet worth of IPs, by including the net mask:

nmap 192.168.210.0/24

You can also just list a range, which is much easier in some cases, using the —exclude option to remove an address that will be angry if port scanned:

nmap 192.168.210.1-100 —exclude 192.168.210.25

Or to do a few hosts within that range:

nmap 192.168.210.1,10,254

Of you can even use the following to read in a list of addresses and subnets where each is on its own line:

nmap -iL ~/nmaplist.txt

By default, mmap is scanning all ports. However, if you know what you’re looking for, scans can be processed much faster if you constrain it to a port or range of ports. Use the -p option to identify a port and then T: for only TCP or U: for only UDP, or neither to do both. Additionally, you can scan a range of ports or separate ports using the same syntax used for identifying multiple hosts. For example, here we’ll scan 53, 80, 110, 443 and 143:

nmap -p 53,80,110,143,443

DO OS detection using the -A option:

nmap -A www.apple.com

For true remote OS detection, use -O with —osscan-guess:

mmap -v -O —osscan-guess mail.krypted.com

We can also output to a text file, using the -o option (or of course > filename but -o is more elegant here unless you’re parsing elsewhere in the line):

mmap -v -o ~/Desktop/nmapresults.txt -O —osscan-guess mail.krypted.com

Firewalls
Next, we’ll look at trying to bypass pesky annoyances like stageful packet inspection on firewalls. First, check whether there is actually a firewall using -s:

nmap -sA www.apple.com

Scan even if the host is protected by a firewall:

nmap -PN www.apple.com

Just check to see if some devices are up even if behind a firewall:

nmap -sP 192.168.210.10-20

Run a scan using Syn and ACK scans, run mmap along with the either -PS or -PA options (shown respectively):

nmap -PS 443 www.apple.com
nmap -PA 443 www.apple.com

Try to determine why ports are in a specific state:

nmap —reason www.apple.com

Show all sent/recvd packets:

nmap —packet-trace www.apple.com

Try to read the header of remote ports to determine a version number of the software:

nmap -sV www.apple.com

Security Scanning
Next, we can look at actually using nmap to test the attacking waters a little bit. First, we’ll try and spoof another MAC address, using the —spoof-mac options. We’ll use the 0 position after that option to indicate that we’re randomly generating a Mac, although we could use a real MAC in place of the 0:

nmap -v -sT —spoof-mac 0 www.apple.com

Next, let’s try to add a decoy, which allows us to spoof some IPs and use that as decoys so our target doesn’t suspect our IP as one that’s actually scanning them (note that our IP we’re testing from is 192.168.210.210):

nmap -n -192.168.210.1,192.168.210.10,192.168.210.210,192.168.210.254

Then, send some crazy packets (not an official term like magic packets, just my own term for throwing a curve ball at things and testing for the viability of syn-flood or Xmas packet attacking):

nmap -sX www.apple.com

Configure a custom mtu:

nmap —mtu 64 www.apple.com

Fragment your packets:

nmap -f www.apple.com

Note: None of Apple’s servers were damaged in the writing of this article. I did a find/replace at the end, when I realized I didn’t want all of you hitting www.krypted.com.

Mac OS X Mac OS X Server Mac Security Ubuntu Unix VMware

Show Line Numbers When Viewing A File

The nl command is used to show line numbers when viewing a file (unless you use the -b option along with an n, which seems to be one of the more pointless things to ever do at the command line, but then what do I know…). So if you’d like to see the line numbers for a file called xsbackup.sh:

nl xsbackup.sh

The output would look like this:

1 #
2 #!/bin/bash
3 #
4 # Script Name:

Or at least, that’s how I used to do it. For decades I never noticed that cat had a -b option. So if you’d like to use cat to see line numbers in your script, just run the command, along with a -b.

cat -b xsbackup.sh

Cat also allows output lines (-b skips lines with whitespace). Cat also has a squeeze option for sequential lines. Overall, not absolutely everything in nl, but enough that I doubt I’ll need to use nl much in the future.

Mac OS X Mac OS X Server Mac Security Network Infrastructure Network Printing Ubuntu Unix VMware

Use Netstat To Locate What Process Is Using A Port

You’re installing software on some host. The installation goes well and then you go to access the information you need or connect to the service from another host. Wait, what’s that? Port is already in use? Crap. We’ve all been there. The quick and dirty answer: netstat. Let’s say you’re trying to use port 8080:

netstat -tuln | grep 8080

Let’s say the response is httpd. OK, let’s see where that’s located using whereis:

whereis httpd

And what kind of file is httpd:

file /usr/sbin/httpd

Which responds with:

/usr/sbin/httpd: Mach-O 64-bit executable x86_64

I guess we knew that since it had a port open, but what type of executable is this httpd you speak of, pray tell?

whatis httpd

httpd(8) – Apache Hypertext Transfer Protocol Server
Apache2::Resource(3pm) – Limit resources used by httpd children
CGI::Carp(3pm) – CGI routines for writing to the HTTPD
httpd(8) – Apache Hypertext Transfer Protocol Server

Oooohhhhh, I see now…

Mac OS X Mac OS X Server Ubuntu Unix VMware

Sort ls Entries By Modification Times

I find that when I’m trying to quickly sort directory listings in a terminal screen, the last thing I want is to have to open . to get a Finder screen and then sort that way. Luckily, I’m not the only one who needs to sort data by time stamp from time to time. To do so using ls, simply use the -alt options:

ls -alt ~/Desktop

Or to see the information in reverse:

ls -altr ~/Desktop

Mac OS X Mac OS X Server Mac Security Mass Deployment Ubuntu Unix VMware Xsan

5 Ways To Manage Background Jobs In A Shell Environment

When running commands that are going to take awhile, I frequently start them with the nohup command, disown the command from the current session or queue them for later execution. The reason is that if I’m running them from a Terminal or SSH session and the session is broken I want to make sure they complete. To schedule a job for later execution, use at. For example, if I want to perform a simple command, I can schedule it in a minute by running it as an echo piped to at:

echo "goldengirlsfix.sh" | at now + 2 minutes

Note, if using 1 minute, you’ll need that to be singular. But you can also disown the job. To do so, end a command with an & symbol. So, running a command or script that will take awhile with an ampersand at the end displays the job number for the command and then you can disown it by running disown followed by -h at the end. for example:

du -d 0 &
disown -h

If you choose not to disown the job, you can check running jobs using the jobs command at any time:

jobs

Nohup runs a command or script in the background even after a shell has been stopped:

nohup cvfsck -nv goldengirls &

The above command runs the command between nohup and the & symbol in the background. By default, you’ll then have the output to the command run in the nohup.out file in your home directory. So if your username were krypted, you could tail the output using the following command:

tail -f /Users/krypted/nohup.out

You can also use screen and then reconnect to that screen. For example, use screen with a -t to create a new screen:

screen -t sanconfigchange

Then run a command:

xsanctl sanConfigChanged

Then later, reconnect to your screen:

screen -x

And you can control-n or control-a to scroll through running background processes this way, provided each is in its own screen.

Finally, in AIX you can actually use the bg command. I used to really like this as I could basically move an existing job into the background if I’d already invoked it from a screen/session. For example, you have pid 88909 running and you want to put it into the background. You can just run bg 88909 and throw it into the background, allowing you to close a tty. But then if you’d like to look at it later, you can always pop it back using, you guessed it, fg. This only worked in AIX really, but is a great process management tool.