Consulting,  Network Infrastructure,  sites,  Unix,  VMware

S3 Command Line Part II

Earlier we looked at using s3cmd to interact with the Amazon S3 storage cloud.  Now we’re going to delve into using Another S3 Bash Interface.  To get started, first download the scripts and then copy the hmac and s3 commands into the ec2 folder created in previous walkthroughs.

To use the s3 script, you need to store your Amazon secret key in a text file and set two environment variables. The INSTALL file included with the package has all the details. The only tricky part I ran into, and from the comments on Amazon, other people ran into, is how to create the secret key text file. Now go into your environment variables in ~/.bash_profile and add S3_ACCESS_KEY_ID (your S3 access key id) from the S3 site on Amazon Web Service and S3_SECRET_ACCESS_KEY (the name of the file with your S3 secret key). If the file that stores this key is called ~/SUPERSECRET then to create it, copy the key to your clipboard from the AWS site and then run echo -n and send the contents of the pasted line to a file:

echo -n MYPASTEDKEY > ~/SUPERSECRET

The -n switch tells echo to not include a new line character and results in a text file of exactly 40 bytes. Once I got the key file created correctly, the s3 script started working, and I was able to upload, download, and list objects in S3.

Next, we’ll list your buckets:

s3 buckets

Then we’ll list the contents of a bucket called images:

s3 list images

Next, we’ll upload a file called emerald.png from the desktop of our computer:

s3 put images emerald.png ~/Desktop/emerald.png

Now let’s try and get the same file and just leave it somewhere else so we can compare the two:

s3 get images emerald.png ~/Documents/emerald.png

Now let’s get rid of the file:

s3  rm images emerald.png

And then to just remove all the files from the bucket:

rmrf images

If you notice, this toolkit is very similar to the s3cmd kit that we looked at earlier.  It’s a little more limited, but I thought it might come with less of a learning curve or be easier to script against depending on what you need.