Category Archives: Unix

Ubuntu Unix

Installing the Docker for OpenStack Heat

Docker is an engine that automates deploying applications as highly portable, self-sufficient containers, independent of hardware, language, framework, packaging system and hosting provider. Heat is the main project used when it comes to OpenStack orchestration. There is a Docker plugin for Heat. To install this plugin, you’ll need to use the stable/icehouse branch (which seems like what’s made the tool so mature rather than simply being available for Nova) to install Heat via apt-get install. Once downloaded, extract the contrib/docker folder and delete the tests directory. Then copy the contrib/docker folder to the OpenStack controller. Here we’ll put it at /usr/lib/heat directory. This results in the path of /usr/lib/heat/docker/docker. Next, install python-pip:

apt-get install python-pip

Then check the installer requirements:

pip install -r requirements.txt

Then edit the heat config, likely at /etc/heat/heat.conf. Here, provide the plugin in the plugin_dirs section as:

/usr/lib/heat/docker/docker

Then reboot and check the resource type listing:

/usr/lib/heat/heat resource-type-list

If you see DockerInc::Docker::Container, you can deploy Docker containers.

Mac OS X Mac OS X Server Mac Security Mass Deployment Unix Windows Server Xsan

Make iMovie Work With Network Volumes

I work with a lot of network storage and video world stuff. While most in the editorial world prefer FinalCut, Avid, Adobe and other tools for video management, I do see the occasional task done in iMovie. By default, iMovie doesn’t support using assets stored on network volumes. However, you can make it. To do so, just use defaults to write com.apple.iMovieApp with a boolean allowNV key marked as true:

defaults write com.apple.iMovieApp allowNV -bool TRUE

imovie

Mac OS X Server Mac Security Microsoft Exchange Server Unix Windows Server

Heartbleed in Comics

Active Directory Mac OS X Mac OS X Server Microsoft Exchange Server Network Infrastructure Ubuntu Unix VMware Windows Server

Stashbox: Turning a Mac Mini Into A Logstash and Kibana Server

You have a lot of boxes. You would like to be able to parse through the logs of all those boxes at the same time, searching for a given timestamp across a set of machines for a specific string (like a filename or a port number). elasticsearch, logstash and kibana are one way to answer that kind of need. This will involve downloading three separate packages (which for this article, we’ll do in /usr/local) and creating a config file.

First, install the latest Java JDK. This is available at jdk8-downloads-2133151.html.

The following is going to download the latest version of logstash and untar the package into /usr/local/logstash (I like nesting that logstash-1.4.0 inside logstash so when the next version comes out I can have it there too, I have plenty of space so keeping a couple versions back helps in the event I need some old binary and can’t get to it ’cause they revved out the version I wrote a script against at some point):

curl -O https://download.elasticsearch.org/logstash/logstash/logstash-1.4.0.tar.gz
mkdir /usr/local/logstash
tar zxvf logstash-1.4.0.tar.gz -C /usr/local/logstash

Once we have log stash, we’ll grab elastic search similarly:

curl -O https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.0.1.tar.gz
mkdir /usr/local/elasticsearch
tar zxvf elasticsearch-1.0.1.tar.gz -C /usr/local/elasticsearch

Then we’ll untar kibana in the same manner:

curl -O https://download.elasticsearch.org/kibana/kibana/kibana-3.0.0.tar.gz
mkdir /usr/local/kibana
tar zxvf kibana-3.0.0.tar.gz -C /usr/local/kibana

Next we’ll make a very simple config file that we call /usr/local/stashbox.conf that listens on port 514 for syslog:

input {
tcp {
port => 514
type => syslog
}
udp {
port => 514
type => syslog
}
}
filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date {
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
}
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}

Next, we’ll enable elastic search:

/usr/local/elasticsearch/elasticsearch-1.0.1/bin/elasticsearch

And finally, in a different window we’ll call logstash with that file as the config file:

/usr/local/logstash/logstash-1.4.0/bin/logstash -f /usr/local/stashbox.conf

Having each of these open in different Terminal windows allows you to see logs in stdout. Next, point a host at your new syslog box. You can use http://krypted.com/windows-server/use-syslog-on-windows for installing Windows clients or http://krypted.com/mac-security/redirect-logs-to-a-syslog-server-in-os-x/ for  a Mac. Once done, let’s get Kibana working. To do so, first edit the config.js.

vi /usr/local/kibana/kibana-3.0.0/config.js

Locate the elastic search setting and put the name of the host running logstash in there (yes, it can be the same as the actual logstash box as long as you install a web server on the logstash box). Then save the changes.

Now move the contents of that kibana-3.0.0 folder into your web directory. Let’s say this is a basic OS X Server, that would be:

cp -R /usr/local/kibana/kibana-3.0.0/* /Library/Server/Web/Data/Sites/Default/

You can then check out your Kibana site at http://localhost or http://localhost/index.html#/dashboard/file/logstash.json for the actual search pages, which is what I’ve bookmarked.

Screen Shot 2014-04-10 at 10.37.51 PM

For example, to see the impact of periodic scripts in System Logs:

Screen Shot 2014-04-12 at 9.07.44 AM

 

iPhone Mac OS X Mac OS X Server Mac Security Mass Deployment Microsoft Exchange Server Network Infrastructure Ubuntu Unix VMware

Quick nmap Hacks

The nmap application is a pretty easy-to-use tool that can be used to port scan objects in a network environment. To obtain mmap in an easy-to-use package installer, for OS X check out the download page at http://nmap.org/download.html#macosx (use the same page to grab it for Windows or *nix as well). Once downloaded run the package/rpm/whatever.

Before I scan a system, I like to pull the routing table and eth info to determine how scans are being run, which can be run by using the mmap command anong with the —iflist option:

nmap —iflist

Basic Scanning
To then scan a computer, just use the mmap command followed by the host name or even throw a -v option in there to see more information (you can use a hostname or an IP):

nmap -v www.apple.com

Use the -6 option if scanning via IPv6:

nmap -v -6 8a33:1a2c::83::1a

Can drop the -v for less info on these, but I usually like more than less. Shows ports, states, services (for the ports) and a MAC address for each IP being scanned.

You can also scan a range of IPs. I usually take the lazy way for this, by using a wildcard. I can replace an octet to scan all objects in that octet. For example, to scan all systems running on the 192.168.210 class B:

nmap 192.168.210.*

You can scan a subnet, which can cover more or less than one octet worth of IPs, by including the net mask:

nmap 192.168.210.0/24

You can also just list a range, which is much easier in some cases, using the —exclude option to remove an address that will be angry if port scanned:

nmap 192.168.210.1-100 —exclude 192.168.210.25

Or to do a few hosts within that range:

nmap 192.168.210.1,10,254

Of you can even use the following to read in a list of addresses and subnets where each is on its own line:

nmap -iL ~/nmaplist.txt

By default, mmap is scanning all ports. However, if you know what you’re looking for, scans can be processed much faster if you constrain it to a port or range of ports. Use the -p option to identify a port and then T: for only TCP or U: for only UDP, or neither to do both. Additionally, you can scan a range of ports or separate ports using the same syntax used for identifying multiple hosts. For example, here we’ll scan 53, 80, 110, 443 and 143:

nmap -p 53,80,110,143,443

DO OS detection using the -A option:

nmap -A www.apple.com

For true remote OS detection, use -O with —osscan-guess:

mmap -v -O —osscan-guess mail.krypted.com

We can also output to a text file, using the -o option (or of course > filename but -o is more elegant here unless you’re parsing elsewhere in the line):

mmap -v -o ~/Desktop/nmapresults.txt -O —osscan-guess mail.krypted.com

Firewalls
Next, we’ll look at trying to bypass pesky annoyances like stageful packet inspection on firewalls. First, check whether there is actually a firewall using -s:

nmap -sA www.apple.com

Scan even if the host is protected by a firewall:

nmap -PN www.apple.com

Just check to see if some devices are up even if behind a firewall:

nmap -sP 192.168.210.10-20

Run a scan using Syn and ACK scans, run mmap along with the either -PS or -PA options (shown respectively):

nmap -PS 443 www.apple.com
nmap -PA 443 www.apple.com

Try to determine why ports are in a specific state:

nmap —reason www.apple.com

Show all sent/recvd packets:

nmap —packet-trace www.apple.com

Try to read the header of remote ports to determine a version number of the software:

nmap -sV www.apple.com

Security Scanning
Next, we can look at actually using nmap to test the attacking waters a little bit. First, we’ll try and spoof another MAC address, using the —spoof-mac options. We’ll use the 0 position after that option to indicate that we’re randomly generating a Mac, although we could use a real MAC in place of the 0:

nmap -v -sT —spoof-mac 0 www.apple.com

Next, let’s try to add a decoy, which allows us to spoof some IPs and use that as decoys so our target doesn’t suspect our IP as one that’s actually scanning them (note that our IP we’re testing from is 192.168.210.210):

nmap -n -192.168.210.1,192.168.210.10,192.168.210.210,192.168.210.254

Then, send some crazy packets (not an official term like magic packets, just my own term for throwing a curve ball at things and testing for the viability of syn-flood or Xmas packet attacking):

nmap -sX www.apple.com

Configure a custom mtu:

nmap —mtu 64 www.apple.com

Fragment your packets:

nmap -f www.apple.com

Note: None of Apple’s servers were damaged in the writing of this article. I did a find/replace at the end, when I realized I didn’t want all of you hitting www.krypted.com.

Mac OS X Mac OS X Server Mac Security Ubuntu Unix VMware

Show Line Numbers When Viewing A File

The nl command is used to show line numbers when viewing a file (unless you use the -b option along with an n, which seems to be one of the more pointless things to ever do at the command line, but then what do I know…). So if you’d like to see the line numbers for a file called xsbackup.sh:

nl xsbackup.sh

The output would look like this:

1 #
2 #!/bin/bash
3 #
4 # Script Name:

Or at least, that’s how I used to do it. For decades I never noticed that cat had a -b option. So if you’d like to use cat to see line numbers in your script, just run the command, along with a -b.

cat -b xsbackup.sh

Cat also allows output lines (-b skips lines with whitespace). Cat also has a squeeze option for sequential lines. Overall, not absolutely everything in nl, but enough that I doubt I’ll need to use nl much in the future.

Mac OS X Mac OS X Server Mac Security Network Infrastructure Network Printing Ubuntu Unix VMware

Use Netstat To Locate What Process Is Using A Port

You’re installing software on some host. The installation goes well and then you go to access the information you need or connect to the service from another host. Wait, what’s that? Port is already in use? Crap. We’ve all been there. The quick and dirty answer: netstat. Let’s say you’re trying to use port 8080:

netstat -tuln | grep 8080

Let’s say the response is httpd. OK, let’s see where that’s located using whereis:

whereis httpd

And what kind of file is httpd:

file /usr/sbin/httpd

Which responds with:

/usr/sbin/httpd: Mach-O 64-bit executable x86_64

I guess we knew that since it had a port open, but what type of executable is this httpd you speak of, pray tell?

whatis httpd

httpd(8) – Apache Hypertext Transfer Protocol Server
Apache2::Resource(3pm) – Limit resources used by httpd children
CGI::Carp(3pm) – CGI routines for writing to the HTTPD
httpd(8) – Apache Hypertext Transfer Protocol Server

Oooohhhhh, I see now…

Mac OS X Mac OS X Server Mac Security Mass Deployment Ubuntu Unix Xsan

compgen like a boss

I’ve traditionally used the apropos command to find new commands. But you can also use the compgen command, which looks at the completion matches for given words, to find a list of commands that you can run, simply use compgen with a -c option:

compgen -c

You can parse information for a single command:

compgen -c | grep apropos

You can also use -a for aliases, -b for bash built-ins and -k for bash keys, as well as `-A function` for functions. You can then string ‘em together:

compgen -abckA function

I won’t paste the output but I’ll let you pipe it to grep to compgen like a boss. Enjoy!

Mac OS X Mac OS X Server Ubuntu Unix

Get Ze Fuser!

A quick and easy way to figure out what process ID and user has a file open is to use fuser. While fuser sounds like maybe some rad command that could emit lasers out of your laptop and fuse together your frenemies, it’s not. It also can’t put that bowl you just dropped off the table that you made when you were 6 and your mother still gushes over. I told you not to use it as an ash try she’ll say. Also, I don’t use it often as lsof can do such things as well. But sometimes you don’t wanna’ wait that long. So let’s look at using fuser. First, run fuser along with a file:

fuser /Library/Logs/CrashPlan/backup_files.log.0

The output will just be the filename back to you, followed by a : then a pid. Now try it with the -u option:

fuser -u /Library/Logs/CrashPlan/backup_files.log.0

Holy crap, a user is included. Again, it’s not something I use a lot but it’s easy to parse so if you’re running a lot of lsof or other calls, then you can save a little expense within your scripts here.

Unix Xsan

One Liner Script To Check If Xsan Is Installed

The following will tell you whether Xsan has been installed on a client system. Here we’re checking if the file exists using the [] for a file (I always quote paths that aren’t variables when doing this type of thing) and and then echoing a response that it does.

[ -f "/Library/Preferences/Xsan/uuid" ] && echo "Xsan is installed"

If the file exists, we could also perform some other tasks or use an else and make changes, like copying an authorization and fsnameservers file into the directory when installing StorNext clients on OS X. The way I would likely do this, if I were saying if the uuid file doesn’t exist, do a task would be:

[ | -f "/Library/Preferences/Xsan/uuid" ] && echo "Xsan is not installed"

In the above example, placing the pipe in front acts as a negative operator, so these two lines are basically the opposite of one another.