krypted.com

Tiny Deathstars of Foulness

I was recently working on a new project developing against Twitter using their JSON interface. Turns out that the Twitter app has an awesome little feature to assist with such a task, a Console. To see the menu for the Console, enable the Develop menu, by putting a true boolean ShowDevelopMenu key into the com.twitter.twitter-mac.plist: defaults write com.twitter.twitter-mac ShowDevelopMenu -bool true Once enabled, use the Develop menu to open Console. Here, you can select various buttons and see the GET, POST, PUT or DELETE sent. as well as the entities sent. To disable the Develop menu: defaults write com.twitter.twitter-mac ShowDevelopMenu -bool false

August 26th, 2012

Posted In: Mac OS X

Tags: , , , , , , , , , ,

CrashPlan Pro Server is a pretty cool tool with a lot of great features that can be used to back up client computers. There are a lot of things that CrashPlan Pro is good at out of the box, but there are also a lot of other things that CrashPlan Pro wasn’t intended for that it could be good at, given a little additional flexibility. The REST API that CrashPlan Pro uses provides a little flexibility and as with most APIs I would expect it to provide even more as time goes on. I often hear people run away screaming when REST comes up, thinking they’re going to have to learn some pretty complex scripting. And while the scripting can be complex, it doesn’t necessarily have to be. You can find a lot of handy information about the options available in the REST API at http://support.crashplanpro.com/doku.php/api. The very first example command that CrashPlan gives is the following:
http://:4280/rest/users?status=Active
Now, to use this in a very simple script, let’s look at it with curl. You are going to need to authenticate, so we’re going to inject that into the URL in much the same was that we would with something like, let’s say, WebDAV, SSH or FTP. If the server name were foundation.lan, the user name was daneel and the password was seldonrulez then the curl command would actually look like so (you could use the -u operator to inject the authentication information, but as you’ll see later I’d like to make those a bit less complex):
curl http://daneel:seldonrulez@foundation.lan:4280/rest/users?status=Active
Note: The default port for the web administration in CrashPlan Pro is 4280. This is simply going to output a list of Active users on the server. The reason it’s going to output only Active users is that we asked it to (reading from left to right after the rest is shown in the URL) query users, using the status attribute and specifying only to show us users whose status matches as Active. We could just as easily have requested all users by using the following (which just removes ?status=Active):
curl http://daneel:seldonrulez@foundation.lan:4280/rest/users
Each user has a unique attribute in their id. These are assigned in an ascending order, so we could also query for the user with an ID of 3 by simply following the users with their unique ID:
curl http://daneel:seldonrulez@foundation.lan:4280/rest/users/3
We could also query for all users with a given attribute, such as orgId (note that these attributes are case sensitive unlike many other things that start with http). For example, to find users with an orgID of 3:
curl http://daneel:seldonrulez@foundation.lan:4280/rest/users?orgId=3
The API doesn’t just expose looking at users though. You can look at Organizations (aka orgs), targets (aka mountPoints), server statistics (aka serverStats) and Computers (aka computers). These can be discovered by running the following command:
curl -i http://daneel:seldonrulez@foundation.lan:4280/rest/
To then see each Organization:
curl http://daneel:seldonrulez@foundation.lan:4280/rest/orgs
And to see each Computer:
curl http://daneel:seldonrulez@foundation.lan:4280/rest/computers
You can also perform compound searches fairly easily. For example, let’s say that we wanted to see
curl http://daneel:seldonrulez@foundation.lan:4280/rest/computers?userId=3&status=Active
These basic incantations of curl are simply getting information, which programmatically could also be specified using a -X operator (or –request if you like to type a lot) to indicate the type of REQUEST we’re sending (continuing on with our Code42 Sci-fi inspired example):
curl -X GET -H ‘Content-type: application/json’ http://daneel:seldonrulez@foundation.lan:4280/rest/orgs
The important thing about being able to indicate the type of REQUEST is that we can do more than GET: we can also POST and PUT. We also used the -H operator to indicate the type of data, which we’re specifying as application/json (per the output of a curl -i command against the server’s REST API URL). POST is used to create objects in the database whereas PUT is used update objects in the database. This could result in:
curl -i -H ‘Content-Type: application/json’ -d ‘{“username”: “charlesedge”, “password”: “test”, “firstName”: “Charles”, “lastName”: “Edge”, “orgId”: “3”}’ http://daneel:seldonrulez@foundation.lan:4280/rest/users
Once you are able to write data, you will then be able to script mass events, such as create new users based on a dscl loop using groups, remove users at the end of a school year (PUT {“status”: “Deactivated”}), mass change orgIds based on other variables and basically fully integrate CrashPlan Pro into the middleware that your environment might already employ.
Perl, Python, Ruby and PHP come with a number of options specifically designed for working with REST, which makes more complicated scripting much easier (such as with php’s curl_setopt); however, these are mostly useful if you already know those languages and the point of this article was to stay in shell scripting land. This allows you knock out simple tasks quickly, even if the good people at Code 42 didn’t think to add the specific features to their software that you might have in mind. Once you start to get into scripting more complex events, look to the Python examples at the bottom of the API Architecture page to get ya’ kickstarted!

November 4th, 2010

Posted In: cloud, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Ubuntu, Unix

Tags: , , , , , , , , , , ,

Earlier we looked at using s3cmd to interact with the Amazon S3 storage cloud.  Now we’re going to delve into using Another S3 Bash Interface.  To get started, first download the scripts and then copy the hmac and s3 commands into the ec2 folder created in previous walkthroughs. To use the s3 script, you need to store your Amazon secret key in a text file and set two environment variables. The INSTALL file included with the package has all the details. The only tricky part I ran into, and from the comments on Amazon, other people ran into, is how to create the secret key text file. Now go into your environment variables in ~/.bash_profile and add S3_ACCESS_KEY_ID (your S3 access key id) from the S3 site on Amazon Web Service and S3_SECRET_ACCESS_KEY (the name of the file with your S3 secret key). If the file that stores this key is called ~/SUPERSECRET then to create it, copy the key to your clipboard from the AWS site and then run echo -n and send the contents of the pasted line to a file:
echo -n MYPASTEDKEY > ~/SUPERSECRET
The -n switch tells echo to not include a new line character and results in a text file of exactly 40 bytes. Once I got the key file created correctly, the s3 script started working, and I was able to upload, download, and list objects in S3. Next, we’ll list your buckets:
s3 buckets
Then we’ll list the contents of a bucket called images:
s3 list images
Next, we’ll upload a file called emerald.png from the desktop of our computer:
s3 put images emerald.png ~/Desktop/emerald.png
Now let’s try and get the same file and just leave it somewhere else so we can compare the two:
s3 get images emerald.png ~/Documents/emerald.png
Now let’s get rid of the file:
s3  rm images emerald.png
And then to just remove all the files from the bucket:
rmrf images
If you notice, this toolkit is very similar to the s3cmd kit that we looked at earlier.  It’s a little more limited, but I thought it might come with less of a learning curve or be easier to script against depending on what you need.

May 21st, 2009

Posted In: Consulting, Network Infrastructure, sites, Unix, VMware

Tags: , , , , , ,