Once upon a time I had to learn to script in bash. I’m still learning, as with most people, but I’m feeling pretty comfortable. I often have people ask me what is the easiest way to learn scripting and I find myself telling people to use the history command. Much of what people need beyond simply looking at their bash history involves variable substitution, loops and regular expressions. Tackled separately this makes a palatable experience.
So then what makes object-oriented or interpretive languages such as perl or python so much more difficult? Is it the lack of a bash history? Let’s try and exercise and see about that.
Open a Terminal window and type the word python.
You’ll see a line about version numbers, etc and then you will then be placed into an interactive python environment, called the interpreter. It looks like this:
At the prompt, simply type
You will then see an error that looks something like: “I am zeh interpretah, you vill do az I say!” Actually it looks a bit more like:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'foo' is not defined
I like to call that Python’s middle finger. That means you did something wrong. Ya, I told ya’ to do something wrong. Why would I start out that way? To show that it doesn’t matter. You should still have the interpreter looking at’ ya, so no big deal. If it helps the old ego pretend I did something wrong, which my wife can tell you is something that’s never ever ever happened before (or not). Now let’s actually do something: set a variable. Type some string followed by an equals sign and then the content you want the variable to have. Numbers and integers go without quotes:
>>> charles = 0
strings are single quoted), as follows (respectively):
>>> charles = 'n00b'
Now that you’ve set a variable print the output by typing just the string that is the name of the variable:
Of course, we could have just typed print n00b to see what the interpreter thinks of me:
>>> print n00b
Note: Variables do not have to be defined and disappear automatically when their scope has also disappeared.
Now let’s talk about functions. Python comes with a number of functions built in. These start with a word and have something (or nothing) in parenthesis, which are arguments for the function (if you’ve got them). A function that Python comes with is exit(). To exit Python with no arguments, just say exit():
Exit without the parenthesis doesn’t work. That’s because even if there are no arguments, exit is a function and functions need parenthesis. There are a lot of functions that can be used, although to keep the interpreter zippy they’re not all active at the same time. Functions can be imported in groups, where each group is in a file and each file is a module. To import a module, use the import command, followed by the module to be imported:
Now let’s look at Python’s help pages using the built-in help function. We’re going to use help to show us methods supported for the redline module we imported earlier:
Or to see a specific function’s help within a module (assuming it’s been documented):
One easy function to use is len(), which calculates the length of a string. To see the length of that charles variable:
I know what you’re thinkin’: 4 seems high (otherwise I might be doing something other than writing this, right?!?!). Moving on. Time to make our own function. Let’s make a basic loop function. To do so, at the interpreter still, type the def to define a function followed by the name and the variables you want to send to it. Below we’ll begin creating a function called bestrobot, creating a variable that will be used, which we’re just gonna’ call input. Notice that the declaration ends with a colon (:). From here on the whitespace is pretty important. All items that begin with a single tab will be a part of this function until the next double carriage return. Hit tab and then let’s do a simple if/then. In this case best will be boolean, simplifying the if then. Notice each of those have a colon at the end as well, with an additional indent:
... if best:
... print 'daneel is the best'
... print 'you should get daneel'
If we fire this up:
Then we should see:
daneel is the best
Next, let’s build a array. This is a pretty straight forward task. Let’s define a list of directory services. To do so, we’ll set a variable name, then in brackets define each item in the list:
>>> dirservers = ['Open Directory', 'Active Directory', 'eDirectory']
That’s not too bad. Next up, let’s do a loop, a staple in any scripters toolbox likely to be whipped out in the first 2 minutes of any scripting exercise. This is much like the if, where the for is followed by a condition, then a colon and on the subsequent line there will be a task assigned to occur:
>>> for dirservers in list:
... print dirservers
Now, because these are in brackets, we can edit the list. But if they were in parenthesis instead then they would be immutable meaning that they couldn’t be edited during runtime (way faster tho). Also, again, the above white space defines the end of the set of commands that loops. Personally, I find allowing the white space to manage where the if/for separates from the else and where the function ends to be pretty straight forward. Arrays are one thing, but I find counters to be a pretty standard staple of why to use for loops. To set a counter that iterates 10 times, you don’t have to do a bunch of crazy stuff, you can just use the range() function to set a loop to count up:
>>> for num in range(10):
Next, let’s look at writing a script into a file. All python scripts should Python scripts should end with a py file extension (*.py). Python scripts should also start with the following line, identifying the interpreter:
The next part of the file is used to import modules. Earlier, we interactively loaded the readline module. This script is going to accept a positional parameter, made possible using the sys module:
Now we’re going to fire up a function:
would be the first parameter that sys brings in, 2 would be the second, etc. 0 is the actual name of the script. Below will be a simple print statement followed by a little text and then our positional parameter:
print 'Ello ', sys.argv
Then call the function at the end:
Put it all together and viola, ur first little python proggie, which I’ll call didyousayello.py:
print 'Ello ', sys.argv
Keep in mind Python scripts will need to be executable by the user account that is trying to execute the script (as with any script). Also, keep in mind that upon reading this that your computer has now been programmed to giggle at you in my voice if you attempt to run a script without executable privileges. Also keep in mind that this setting cannot be undone, even with a secure unerase of the whole drive and reinstall. Since secure unerase leaves behind a little data though the laughing will cut in and out, like VoIP calls to tech support silos in strange foreign lands such as Singapore, Beijing and Minneapolis, where apparently it snows a lot. Where was I? Right, run your script, passing that first argument to the script:
python didyousayello.py Emerald
krypted February 21st, 2011
Posted In: Mac OS X, Mass Deployment, Unix
array, basic for loop, define function, if/then, immutable, import sys, python, python variable
Amazon S3 now allows administrators to host simple web sites. Previously, you could host images, videos and other files using S3 buckets, but now you can host full sites. To do so you will need only configure a webroot and some error documents. To get started:
- Log into the Amazon S3 Management Console
- Right-click on an Amazon S3 bucket
- Open the Properties panel
- Configure your webroot
- Click on the Website tab
- Configure error documents in the Website tab
- Click Save
Pretty easy, right? But what if you need to configure the php.ini file or add MIME types, etc. Notice that at the start of this I said “simple.” I’m sure more features are to follow, but for now S3 is mostly appropriate for very simplistic sites.
krypted February 19th, 2011
Posted In: cloud
amazon s3, cloud, hosting, web, web hosting, website
There are a lot of environments that attach Windows client computers to an Xsan or StorNext filesystem. In the past I’ve looked at using different versions of StorNext to communicate with Xsan, but in this article we’re actually going to take a look at Quantum’s StorNext FX2 client software. Before getting started, you’ll want to have the StorNext media, have the serial number added to the metadata controllers, have the HBA (fibre channel card) installed, have the fibre patched into the HBA, have the IP addresses for the metadata controllers documented and have a copy of the .auth_secret file obtainable from the metadata controllers once they’ve been properly licensed.
To get started, first install the HBA drivers. This will be different for each brand of card, but in most cases it will be a simple installer. Once the installer is run and the system rebooted (not all HBA’s required reboots), you will see a screen similar to the following, indicating the Generic SCSI Array Device is installed and then starting to recognize the LUNs that comprise the volume. Make sure not to configure a new filesystem for the LUNs, especially if they are already in use.
Once you can see the hardware infrastructure you are ready to install the software. If using a 64 bit version of Windows then contact StorNext for the installers, otherwise you should be able to use those downloaded from the website. Always start with the latest drivers rather than using those distributed on the media with the StorNext license.
Once you have the installer ready, click on it and you will see the StorNext Installation screen. Here, provided it is a new installation, the first item wills ay Install StorNext. As you can see from the following screen, you also have the options in the future to Upgrade, Reinstall, Remove and Configure StorNext from the StorNext Installation screen.
At the StorNext Component selection screen, you will be able to select whether to install the Help Files (FAQs) and/or the StorNext FX2 client. Here, you can probably leave both enabled and simply click on the Next>> button.
You should now see your LUNs. Open the Windows shell environment and cd to the C:Program FilesStorNextbin directory. Run cvlabel -l and verify all of the LUNs that are needed for the volumes you will be working with are present. If they are not, check your zoning and physical infrastructure.
Still using the Windows shell environment, cd into the C:Program FilesStorNextconfig directory. Edit the fsnameservers file and enter the IP addresses for your metadata controllers in the order they appear in the fsnameservers list on your metadata controllers.
You will also need to copy the .auth_secret file from the metadata controllers to the client computer. By default, this will be copied into the C:Program FilesStorNextconfig directory. Next, reboot.
Provided that we can see our LUNs, we should also be able to see our volumes through cvadmin. To verify, we can use the cvadmin command in much the same way we do so in StorNext for Linux or Xsan:
cvadmin -e select
Provided all of the volumes appear and each has a valid controller (see those with an * below), we can then go into the StorNext Client Configuration tool to complete the client configuration. To do so, open Client Configuration from Start -> Program Files -> StorNext. Once open, you should see the volumes shown in cvadmin and they should be listed as available.
Windows accesses volumes through what are known as drive mappings. A drive map is an alphabetical representation of a location. These locations can be folders within a file system, network volumes or direct attached volumes. the configuration is complete, Windows will treat the drive letter being configured as a local volume. Select Tools Properties and you will then be able to map a drive to the appropriate drive letter. It is also possible to mount an volume into a given directory; however, this is somewhat rare comparably.
Repeat this process until all volumes are mapped and click on Apply. Now you can mount your new volumes. To do so click on them back at the main screen and you will then be prompted to mount. Click Yes.
That’s pretty much it. Have fun.
krypted February 17th, 2011
Posted In: Windows Server, Xsan
cvadmin, cvlabel, install windows client, mount volume, StorNext, windows xsan, Xsan
The CIRCL (Computer Incident Response Center Luxembourg)
has cited my Enterprise iOS book in a paper title Security of iOS Based Devices
It’s only a few pages so a pretty quick read. But what is interesting about it is that in the second edition of the book I’ve already started to replace the Applications chapter (which I wasn’t really happy with in the first place) with a chapter on securing the devices. These days, with a two year old around, I’ve been thinking about adding something on physical security, but I think that might just be superfluous (and ever changing)… Also, in preparation for 2nd edition, I would like to add case studies. So if your organization or company is doing cool stuff with iOS based goodness (and more importantly you can get sign-off to talk about it publicly), please let me know!
Oh, and for those waiting, the paper on bare metal imaging iOS is almost done. Just waiting for my new 64 port USB hub to show up so I can test flashing 64 at once! 🙂
krypted February 15th, 2011
Posted In: iPhone, Mass Deployment
ios, iOS Security, iPhone, ipod touch, Luxembourg, Securing iPad
In an earlier post
I looked at querying feeds and removing objects from Podcast Producer in a somewhat broad manner. To delete a single podcast (not a feed), you need to first find the ID for the podcast, then delete the corresponding podcast bundle (pdb). Removing assets is one of the only processes with regards to Podcast Producer that isn’t intuitive, and it’s just waiting for someone to wrap these steps into a nice pretty GUI…
To find the ID of the podcast, first let’s query the sql database for the title:
sqlite3 /Volumes/pcp/PodcastProducer/Server/db.sqlite3 'SELECT * FROM episodes' | grep "TITLE"
Note the date and the ID (the big long string) and then browse to /Volumes/pcp/PodcastProducer/Content and then the folder that corresponds to the date obtained from the sqlite3 command. Once opened find the ID (or “big long string”) from the sqlite3 command. Then remove the Podcast Data Bundle that corresponds to that ID:
rm -rf .pdb
Once the entry has been deleted then resynchronizing the library will get rid of it completely. This can be done using the pcastconfig command along with the –sync_library option:
krypted February 11th, 2011
Posted In: Mac OS X Server
pcast, pcastaction, podcast, Podcast Producer, podcasting, sqlite3
I’ve done a number of articles on using Linux to provide services such as OpenLDAP for Mac OS X, but never on using the LDAP implementation in Mac OS X Server to provide authentication services for Linux. Well, it isn’t that difficult to do, but it is worth pointing out how to do it. To get started, we’re going to use Red Hat. You’ll want to have a known IP address for Open Directory and if SSL is required you’ll want to install the certificate on the Red Hat box before moving forward.
There are two tools that can be used to hook Red Hat into an LDAP environment. The first is an old school blue screen that you tab through called authconfig. You can get to authconfig using the /usr/bin/authconfig file. Once you run the command you’ll be able to tab through the fields, enabling LDAP authentication and providing a server name or IP as well as the Base DN, which can be obtained in the Open Directory settings from Server Admin. Additionally, if you will be using Kerberos, the Realm can also be obtained from Server Admin.
When you’ve configured the settings, tab to Ok and press enter to enable the changes. This will update the /etc/ldap.conf file on the client and kinit if needed. Another tool, with similar functionality but a more modern looking interface is Authentication Configuration, located at /usr/bin/system-config-authentication. This works identically, although it looks a bit prettier.
Once you have saved, the ldap.conf file should reflect the changes, as with authconfig:
A common case for this type of configuration would be an Xsan environment, where the Open Directory Master and Replica are also metadata controllers and where you want to have RHEL clients on the SAN, leveraging the same directory services so the UID and groups are all matched up. Additionally, the common schema files for Red Hat are mostly included with Open Directory (core, cosine, inetorgperson, nis), but for autofs, you’ll need to load that into Open Directory manually (if you use it). It can be found on any Red Hat system in /etc/openldap/schema/redhat/autofs.schema.
krypted February 8th, 2011
Posted In: Mac OS X Server, Ubuntu, Unix
I find a very common task that I need to do is find a string in a file and replace it with another string. Or better, find all instances of a given string and replace them with a new string. I figure others will need to do this as well. This is also an interesting example of how Mac OS X is not “the same” as Linux.
The sed command can be used to quickly perform a find and replace inside of a file. The following example will use the -i option to do so in-place, defining no extensions to -i using the double quotes (“”), then using the /s to instruct a substitution of the pattern to match followed by the pattern to replace (in this case we’re finding like and replacing it with love in celebration of the upcoming Valentines Day) and doing so globally, or for all instances using the /g option. The find and replace will be performed inside of the file called /Users/krypted/test:
sed -i "" 's/like/love/g' /Users/krypted/test
Because things can go wrong during the edit, such as an OS crash, it is always smart to make a backup of your file before you change it. By appending a .backup to the -i option, sed will get instructed to make a backup:
sed -i.backup -e 's/like/love/g' /Users/krypted/test
By default in Linux, the command would work with or without the double quotes (“”) that I used to define the extension; however, as mentioned Mac OS X is not Linux; you have to define an extension or the command will error out. Overall, finding and replacing or substituting text within a file is a very common systems administration task and the above command is a very simplistic approach to performing such a task.
krypted February 7th, 2011
Posted In: Mac OS X, Ubuntu
find and replace, interactive, Mac OS X, scripting, sed, text substitution
For those who don’t yet get a subscription to MacTech Magazine, I’ve been a long time lurker and suggest you do the same. I recently submitted an article to them for the Enterprise Desktop Alliance
and it will be in the February issue. There’s also one appearing in the March issue. They’re both fairly long and cover the topic of moving specific services off of Mac OS X Server and onto the Windows platform.
I don’t recommend that any organization go out and start ripping out all their Mac OS X Servers because Apple dropped the Xserve. But I am a guy who really likes having a lot of options in front of me. While I’ve published a number of articles here on migrating services to Linux, it makes sense that given the deep Windows chops many environments have with their Windows and Active Directory administrators that many would rather just stick with that. So I hope you enjoy the articles and subscribe to MacTech if you don’t already (btw, MacTech is the only print magazine that I personally subscribe to)!
krypted February 4th, 2011
Posted In: Active Directory, Mac OS X Server, Mass Deployment
Absolute Manage, Active Directory, ExtremeZ-IP, MacTech, Windows Server
Seems like the answer to a lot of Samba problems are to toggle streams on or off according to what kind of problems are being encountered. So I thought I’d do a couple of quick .command files that can be sent to clients to do it. This way, they can be sent to users having problems to test enabling or disabling the option.
Click to download
krypted February 1st, 2011
Posted In: Mac OS X, Mac OS X Server, Mac Security
fix, script, SMB, streams