Tag Archives: ssh

cloud Mac OS X

One Liner To Install gcloud for Managing App Engine Instances

I had previously been using the gcutil command. But I cheated a little with the one liner promise to get the new tool, gcloud, installed:

curl https://dl.google.com/dl/cloudsdk/release/install_google_cloud_sdk.bash | bash ; unzip google-cloud-sdk.zip ; ./google-cloud-sdk/install.sh

The installation shell script is interactive and will ask if you want to update your bash profile. Once run, kill your terminal app and the new invocation will allow you to log into App Engine using the gcloud command followed by auth and then login:

gcloud auth login

Provided you’re logged into Google using your default browser, you’ll then be prompted to Accept the federation. Click Accept.

Screen Shot 2014-01-03 at 11.14.21 PM

The gcloud command can then be used to check your account name:

gcloud config list

To then set a project as active to manage it, use the set option (or unset to not manage it any longer:

gcloud config set project kryptedmuncas

You can then use components, sql or interactive verbs to connect to and manage instances. Each of these commands are interfacing with the API, so if you ever find that you’ve exceeded what this simple command provides for, you can always hit the API directly as well. I found that the interactive command was my favorite as I could figure out what limitations I had using interactive and then try and figure out how to accomplish tasks with commands from there.

Mac OS X Mac OS X Server Mac Security Ubuntu Unix

Working With Files Whose Name Starts with a “-“

Recently I needed to create a bunch of files that had names starting with a dash. If you simply run touch followed by the filename, if the filename starts with a dash, it will throw an error that there’s an illegal option. Therefore, you must escape out the dash by passing a double dash in front of it. For example, to create a file with a dash in the name, use the following syntax:

touch -- -man

Likewise, to remove that file:

rm -- -man

You can also use the double dash when passing commands to ssh rather than have ssh interpret them as arguments, making it useful for command and control scripting. For example, to ssh into a host and repair a volume:

ssh cedge:mypassword@krypted.com -- cvfsck goldengirls --nv

The double dash indicates there are no more command line options for the command in front of it.

Mac OS X Server

Enable SSH, ARD, SNMP & the Remote Server App Use In OS X Server (Mavericks)

SSH allows administrators to connect to another computer using a secure shell, or command line environment. ARD (Apple Remote Desktop) allows screen sharing, remote scripts and other administrative goodness. SNMP allows for remote monitoring of a server. You can also connect to a server using the Server app running on a client computer. To enable all of these except SNMP, open the Server app (Server 3), click on the name of the server, click the Settings tab and then click on the checkbox for what you’d like to enter.

Screen Shot 2013-10-05 at 9.18.55 AM

All of these can be enabled and managed from the command line as well. The traditional way to enable Apple Remote Desktop is using the kickstart command. But there’s a simpler way in OS X Mavericks Server (Server 2.2). To do so, use the serveradmin command.

To enable ARD using the serveradmin command, use the settings option, with info:enableARD to set the payload to yes:

sudo serveradmin settings info:enableARD = yes

Once run, open System Preferences and click on Sharing. The Remote Management box is then checked and the local administrative user has access to ARD into the host.

Screen Shot 2013-10-05 at 9.15.00 AM

There are also a few other commands that can be used to control settings. To enable SSH for administrators:

sudo serveradmin settings info:enableSSH = yes

When you enable SSH from the serveradmin command you will not see any additional checkboxes in the Sharing System Preferences; however, you will see the box checked in the Server app.

To enable SNMP:

sudo serveradmin settings info:enableSNMP = yes

Once SNMP is enabled, use the /usr/bin/snmpconf interactive command line environment to configure SNMP so you can manage traps and other objects necessary.

Note: You can’t have snmpd running while you configure SNMPv3. Once SNMPv3 is configured snmpd can be run. 

To allow other computers to use the Server app to connect to the server, use the info:enableRemoteAdministration key from serveradmin:

sudo serveradmin settings info:enableRemoteAdministration = yes

To enable the dedication of resources to Server apps (aka Server Performance Mode):

sudo serveradmin settings info:enableServerPerformanceMode = yes

Mac OS X Mac OS X Server Mac Security Ubuntu

Generating New SSH Keys

Sometimes when we’re doing work, we end up changing an SSH key. You then access the host using something like this:

ssh krypted@10.10.10.10

When accessing the host you then get a warning similar to the following if the key changed:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
aa:bb:cc:dd:ee:ff:00:11:22:33:00:11:22:33:44:55.
Please contact your system administrator.
Add correct host key in /home/remi/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /home/remi/.ssh/known_hosts:1
In case you know the host identification has changed, you can safely discard this warning.

To fix you can edit the ~/.ssh/known_hosts file and remove the offending line (usually the number in the Offending ECDSA line) or just regenerate the key using ssh-keygen, which will automatically edit the known_hosts file for you:

ssh-keygen -R 10.10.10.10

Mac OS X Mac OS X Server Mac Security Xsan

Update the Promise Vtrak Firmware Using SSH

Updating the firmware on Promise arrays is straight forward enough from the WebPAM. But what happens if a firmware update goes funky and you can’t get into the WebPAM any longer (ah, the joys of beta testing)? Well, you can always download an older firmware and reload it provided you can ssh or telnet into the host. Download from http://www.promise.com/support/download.aspx?m=93&region=en-global for your given model.

Then, you need the firmware accessible to the Promise chassis via tftp. A simple tftp GUI tool is available at http://ww2.unime.it/flr/tftpserver. Once configured, log into the Promise array and then use the ptiflash command to update the firmware. In the following command we’ll use the -s option to identify the IP address of our tftp server and then the -f option to identify the name of the file (note that I’ve shortened the ptif file for this X30 to be just fw.ptif so I don’t fat finger the multiple hyphens in a ridiculously long file name that I can’t autocomplete):

ptiflash -t -s 192.168.69.30 -f fw.ptif

If the server can’t access the file note that you have a tftp client binary that works much like the ftp binary built into OS X to test that you can access the server and the file from the IP address the X30 is using. If the file is accessible, when prompted to update the flash, enter y and press enter.

The update process is going to take about 15 to 20 minutes. If running the latest versions of the X30 firmware I recommend using Firefox.

Mac OS X Mac OS X Server Mass Deployment

Enable Server Side File Tracking in OS X Mountain Lion Server

Mobile Home Directory synchronizing in OS X Server environments is used to synchronize the home folder of clients with a copy that lives on the server, so users can roam between computers with their desktop, documents and preferences following them from machine to machine. Server Side File Tracking creates and keeps a copy of the sync database on client machines and servers, comparing the two databases when synchronizing rather than scanning directories for all the synced files each time a synchronization occurs. In environments with synchronizing Mobile Home Directories, Server Side File Tracking (SSFT) can help reduce the amount of time required for syncs. Server Side File Tracking is disabled by default in OS X Mountain Lion Server and cannot be enabled from the Server app. To enable Server Side File Tracking (aka – FileSyncAgent), use the following command:

sudo serveradmin settings info:enableFileSyncAgent = yes

To then turn it back off, if you so choose:

sudo serveradmin settings info:enableFileSyncAgent = no

Logs are then stored in ~/Library/Logs/FileSyncAgent/FileSyncAgentVerbose.log if you need further information. Note that TCP port 2336 needs to be open for the FileSync Agent to connect over ssh on port 2336 to the server; however, ssh doesn’t need to be enabled on the standard port 22 but mobile users must have access to the SSH SACL.

Mac OS X Server Mac Security Mass Deployment

Enabling ARD, SSH & SNMP On Mountain Lion Server Using serveradmin

The traditional way to enable Apple Remote Desktop is using the kickstart command. But there’s a simpler way in OS X Mountain Lion Server. To do so, use the serveradmin command.

To enable ARD using the serveradmin command, use the settings option, with info:enableARD to set the payload to yes:

sudo serveradmin settings info:enableARD = yes

Once run, open System Preferences and click on Sharing. The Remote Management box is then checked and the local administrative user has access to ARD into the host.

The Server app will also have the “Enable screen sharing and remote management” option checked.

There are also a few other commands that can be used to control settings. To enable SSH for administrators:

sudo serveradmin settings info:enableSSH = yes

To enable SNMP:

sudo serveradmin settings info:enableSNMP = yes

To enable the dedication of resources to Server apps (aka Server Performance Mode):

sudo serveradmin settings info:enableServerPerformanceMode = yes

Mass Deployment

One Teletype to Bind Them (Or, Clustered SSH for OS X)

When working at scale, and particularly with hosts that need to have the same configuration or you want to perform the same queries on, the issue becomes how do I ‘reach out and touch’ my fleet? Without centralized infrastructure backed by a messaging broker or a heavier process that leaves hooks in systems and/or requires its own domain specific language, sometimes you can get by with… plain ol’ ssh. Apple Remote Desktop can take us a lot of the way there, and one of the announced features of Mountain Lion is that screen sharing gets another piece of ARD’s pie, the ability to drag-and-drop files to transfer them to the remote machine. But when trying to use features other than screen control, ARD has been found to be hit-or-miss (or misreporting the functionality of hosts) in some circumstances.

csshX in action

‘Scripty’ folks look at these issues and craft tools to meet the challenge-slash-obscure-use case. Perl has long been relied upon for network-aware utilities, and csshX is a tool for managing a ‘cluster’ of  ssh sessions on the Mac. You can download or checkout the code from its googlecode site, and it has a man page that can be accessed when calling the binary directly with the -m switch. Options include telling it the login and/or password to use, feeding it a text file of hosts to access, or merely list hosts by DNS name or IP with spaces in between. Even if user names or passwords are different, fully-functional windows open as it attempts ssh connections to each host, with a red window you can use to control them all once you’ve authenticated to the ssh sessions.
From that point on, the world is your proverbial jerry-rigged oyster! To mimic ARD’s file transfers you could scp back to your machine (as kludges go, smileyface,) and another random tip: using the emacs readline functionality to jump to the beginning of a line with Ctrl-a still works, even though csshX uses that for a special purpose (as does the terminal multiplexer screen,) simply hit Ctrl-a again and the program will understand you wanted to send that to the remote sessions. Enjoy!

Mac OS X Mac OS X Server MobileMe

Sync'ing iTunes Libraries

I recently spent a few days trimming down the amount of space consumed by my home folder. In so doing I discovered a number of things I could be doing better with regards to utilization of my drive space. So I decided to offload most of my media (photos, movies, etc) off my laptop and onto my Mac Mini server. I also decided that one thing I’d like to live on both is iTunes.

Note: Before you do anything in this article you should verify you have a good back up. Also, both machines will end up needing to be Authorized for your iTunes account.

There are a lot of ways to keep two iTunes libraries in sync. There are also a number of 3rd party tools that can help you do so. I tested all the tools I could find and decided I’d rather just script it myself. Scripting a synchronization operation in Mac and Linux always seems to come down to a little rsync action. Given that rsync is a little old in Mac OS X, I started out by updating rsync to the latest (3.0.7) using the steps provided on bombich.com (I added using /tmp):

mkdir /tmp/rsyncupdate
cd /tmp/rsyncupdate
curl -O http://rsync.samba.org/ftp/rsync/src/rsync-3.0.7.tar.gz
tar -xzvf rsync-3.0.7.tar.gz
curl -O http://rsync.samba.org/ftp/rsync/src/rsync-patches-3.0.7.tar.gz
tar -xzvf rsync-patches-3.0.7.tar.gz
cd rsync-3.0.7
curl -o patches/hfs_compression.diff http://www.bombich.com/software/opensource/rsync_3.0.7-hfs_compression_20100701.diff
curl -o patches/crtimes-64bit.diff https://bugzilla.samba.org/attachment.cgi?id=5288
curl -o patches/crtimes-hfs+.diff https://bugzilla.samba.org/attachment.cgi?id=5966
patch -p1 <patches/fileflags.diff
patch -p1 <patches/crtimes.diff
patch -p1 <patches/crtimes-64bit.diff
patch -p1 <patches/crtimes-hfs+.diff
patch -p1 <patches/hfs_compression.diff
./prepare-source
./configure
make
sudo make install
sudo rm -Rf /tmp/rsyncupdate
/usr/local/bin/rsync –version

Provided the version listed is 3.0.7 then we have a good build of rsync and can move on with our next step, getting a target volume mounted. In this case, I have a volume shared out called simply Drobo (I wonder what kind of RAID that is?!?!). Sharing was done from System Preferences -> Sharing -> File Sharing -> click + -> Choose Drobo and then assign permissions. The AFP server is on an IP address of 192.168.210.10. For the purposes of this example, the username is admin and the password is mypassword. So we’ll do a mkdir in /Volumes for Drobo:

mkdir /Volumes/Drobo

Then we’ll mount it using the mount_afp command along with a -i option:

mount_afp “afp://admin:mypassword@192.168.210.10/Drobo” /Volumes/Drobo

Now that we have a mount we’ll need to sync the library up. In this case, the Music directory on the Drobo has a symlink from ~/Music. This was created by copying my Music folder to the drobo and then rm’ing it (fails when trying from Finder):

rm -Rf ~/Music

Then using ln to generate the symlink:

ln -s ~/Music /Volumes/Drobo/Music

Now sync the files. I’m not going to go into all of the options and what they do, but make sure you have permissions to both the source and the target (using the username and password from the user whose data your changing helps):

/usr/local/bin/rsync -aAkHhxv –fileflags –force –force-change –hfs-compression –delete –size-only ~/Music/iTunes /Volumes/Drobo/Music

Note: If you get a bunch of errors about operations failing then consider disabling the Ignore ownership on this volume setting for any external media you may be using.

Now fire up iTunes on the target machine and make sure it works. At this point, I could also share out the Music folder from my laptop and sync back as well, which would effectively allow me to make changes on both machines. However, for now, I only want to make changes on the laptop and not the desktop so there’s no need for a bidirectional sync.

Once the sync is complete, we can tear down our afp mount:

diskutil unmount /Volumes/Drobo

Now that we can sync data, we still need to automate the process as I’m not going to want to type all this every time I run it. First up, I’m going to create a .sh file (let’s just say /scripts/synciTunes.sh):

touch /scripts/synciTunes.sh

Then I’m going to take the commands to mount the drive, sync the data and then unmount the drive and put them in order into the script:

/bin/mkdir /Volumes/Drobo
mount_afp “afp://admin:mypassword@192.168.210.10/Drobo” /Volumes/Drobo
/usr/local/bin/rsync -aAkHhxv –fileflags –force –force-change –hfs-compression –delete –size-only ~/Music/iTunes /Volumes/Drobo/Music
/usr/sbin/diskutil unmount /Volumes/Drobo

Once created, the script should be run manually and provided it succeeds then it can be automated (ie – creating a LaunchDaemon). If it works after a little while, then you can consider synchronizing your iPhoto and anything else if you so choose. Also, I ended up actually using ssh pre-shared key authentication and doing rsync over ssh. That allows you not to put the password for a host on your network into an unencrypted form in a script. You could do some trickeration with the password, but you might as well look into pre-shared keys if you’re going to automate this type of thing to run routinely. Finally, I also later ended up removing the iTunes Genius files as I started to realize they were causing unneeded data to sync and they would just rebuild on the other end anyway. Hope this helps anyone else looking to build an iLife server of their own!

Home Automation Mac OS X

Running SSH on AppleTV

Sometimes it can be really useful to have an SSH connection into your AppleTV. If I need to explain why then you probably won’t want to do it. Unless of course, you’re just after getting something like Boxee running, which we’ll look at as well. Before we get into doing anything to your AppleTV, when we’re done I do not know how Apple will feel about your warranty moving forward, so do this stuff at your own risk (but that’s pretty much true for many articles on this site)…

So first up, let’s install SSH. To get started, plug in a jump drive you don’t mind reformatting. Then run the df command and look at which filesystem that the jump drive was mounted as. In most cases this should be /dev/disk1s1 or /dev/disk2s1 or something like that. Note this location and while you’re at it, double-check that the data is trivial to you and that you really don’t mind reformatting the jump drive.

Next, let’s download atvusb-creator, a little utility that will generate a new patchstick based on that jump drive (a patchstick being the term applied to usb sticks that will hax0r an AppleTV). Once downloaded, run the tool. Select ATV-Patchstick in the Choose an Installation dialog, and then select the version of the AppleTV OS you have (if you’re fully software updated then as of the date of this writing that would be 3.x). Next, choose ssh tools from the 3rd field in the Installation Options section, making sure that the box is checked. If you are just trying to get XBMC or Boxee running then you can check the boxes for those as well at this point.

ATV USB Creator Screenshot

ATV USB Creator

Next, set the USB Target Device field to be the filesystem you selected earlier and then click the Create Using button and wait for the process to finish. Once the patchstick has been created, plug it into your AppleTV and reboot the unit. You’ll see a bunch of code, similar to starting Mac OS X into verbose mode. When the screen tells you that you’re done, unplug the patchstick and reboot the device. Upon reboot it will be running SSH with a username and password of frontrow. If you’re not using a static IP address then if you open iTunes and connect to the device you’ll have an entry in your arp table for it. You can run arp and find the IP fairly easily. Once found, use the SSH command to connect to the device. For example, if mine is on an IP address of 10.0.0.100 then I would use the following command to connect to it:

ssh frontrow@10.0.0.100

Now you have an AppleTV running SSH. Even though this article isn’t meant to be about Boxee or XBMC, you can then install those by going to the new Launcher menu and then to Downloads and downloading those applications (otherwise if you try to access them you’ll get an error that the .app bundle can’t be found). Once those are in place it should open pretty easily.

Now that you’re running SSH, let’s look at one of the uses. I want a web browser on the AppleTV (even though typing a URL in it is pretty painful unless you install a keyboard too). For this instance, I’m going to use CouchServer, ’cause I like the way the keyboard works and because there’s a silverlight that kinda’ sorta’ works with it. First, download the files for CouchSurfer here. Then copy the files that were downloaded up to the device (assuming the filename is CouchSurfer-Lite.tar) from your client computer:

scp ~/Desktop/CouchSurfer-Lite.tar frontrow@10.0.0.100:~

Next, SSH into the AppleTV and extract the tar file:

tar -xvpf CouchSurfer-Lite.tar

Then move the extracted data into the PlugIns directory (which will display the appliance similar to how Launcher would be displayed at this point:

sudo mv CouchSurfer.frappliance /System/Library/CoreServices/Finder.app/Contents/PlugIns/

(your password will be frontrow in case you have hard core add and have forgotten it already)

We’re gonna’ give ownership to wheel:

sudo chown -R root:wheel /System/Library/CoreServices/Finder.app/Contents/PlugIns/CouchSurfer.frappliance

Then reboot the AppleTV. Upon reboot, you will then have a shiny new web browser making your AppleTV even more like a full fledged Mac with Front Row. Now you’re in pretty good shape. You’ve pretty much put more stuff on your AppleTV than you can possibly use, but you still probably just want NetFlix to work on it. For that, you’ll need to get Silverlight working with CouchSurfer and just browse to the movies in the web browser at Netflix.com as the Boxee implementation for AppleTV doesn’t yet work with NetFlix and there aren’t any native Plug-Ins that work with it yet either (that I’m aware of). Also, if you’re going to use any of the 3rd party media browsers, keep in mind that they’re sitting on top of the OS layer and that their resource utilization seems pretty poor compared to the native media browser on the device (given the abstraction there, it seems logical it would be so no complaints).

BTW, another fun little app (to help make your AppleTV more like your iPad):

http://code.google.com/p/weatherfront

And the most intriguing one that I haven’t actually gotten to work yet (haven’t had time to get past the second or third step – busy) is:
http://www.appletvhacks.net/2007/04/02/install-asterisk-on-apple-tv/#more-41

What I’d like to see – the ability to run my AppleTV as a Zwave controller… Or iPad… Or Newton… :)