Imaging can be a complicated task. Many imaging environments have a lot of scripts, packages, base images and other aspects of automation. The more of these that you have, the more potential combinations you have for the state of a system once they’ve been run. This gets complicated when you want to make sure that each possible combination of images will have a consistent result when installed. For example, take something simple, like a property list. Each possible combination of packages, scripts images and even managed preferences might have a different impact on that poor property list.

A simple defaults command can often give administrators the ability to see what settings have been applied to a system. For simple tasks that can be programatically trapped for, this might be enough. You can argue that everything can be programatically discovered, but I would argue that is an immature way to look at such things. The example I used in the Enterprise Mac Administrator’s Guide was looking at Microsoft Word. Let’s say that you install 24 fonts. When you open Word, you end up needing to check and make sure that each font loads. You can verify fonts test clean, but you can’t verify that they show up in Word. Microsoft gives you the ability to list fonts that appear in the font menu programatically. But you would literally need to open Word and look in the fonts menu on each regression of each system image in order to tell if the fonts actually appear in the list and if so, whether they look right. One small example. In many environments, each application will have 10 or more tests, with a lot of applications potentially being deployed – and the list grows as failures are discovered year over year…

Now let’s say you’ve got 80 packages in your self service library. At this point, in order to tell that every combination is cool, you’d need to manually open Word on a lot of image combinations in order to verify that your fonts load properly. EggPlant is a tool that enables administrators to run a script that clicks on things for us and reports back on what happens when we do. Now, AppleScript can do some of this as well. But eggPlant gets very in depth in terms of scripting logic and is cross platform. EggPlant leverages a Vine server that gets installed on clients. Screen shots are then taken through a Vine client and compared to screen shots you make and save in eggPlant to compare to the ones that are captured from Vine. That’s a bit of overs implication, but at the end of the day, bolt some simple scripting on top of that and you’ve got a pretty advanced solution. EggPlant now also supports iOS by jail breaking devices to load the Vine server.

EggPlant is one of the best tools for this kind of thing for the Mac platform. It’s more popular amongst those who do a lot of web testing, but it shoehorns nicely into post-flight imaging regression testing. I had a good conversation with @tvsutton at the Penn State Mac Admins conference about eggPlant and he mentioned a tool called Sikuli that I hadn’t used before, so hat tip to him for the rest of this article. While eggPlant has way more logic in it, for simple regression testing, you can leverage Sikuki in its place, which wouldn’t cause some of the potential conflicts that running a localized Vine server might cause on client systems.

Sikuli is a simple tool that allows administrators to script graphical events. You can effectively automate anything you see on a screen using the same type of screen shots; however, with Sikuli, you dump the screen shots directly into the script via the scripting interface, known as the Sikuli IDE.

The first step to using Sikuli is to download the IDE from their site and run the installer. Once installed, open Sikuli.

The options along the left side are some common commands that can be run, in serial from top to bottom. Not all of the possible methods are exposed in the GUI sidebars, but many of the more common ones are. Let’s go ahead and manually enter switchApp followed by an application you’d like to open with your script. In my example, I’m going to use Server.app:

switchApp("Server.app")

While the command says switchApp, it might as well say openApp, as the application will open when you run the command. Go ahead and see for yourself (assuming you have a Lion Server and can therefore open Server.app. Now, let’s look at some graphical interfacing, go ahead and click on click in the sidebar, which will dim the screen and bring up crosshairs for taking a screenshot. Here, select some part of the screen. I’m using Users for my example:

Once you take the screenshot, it should appear in the parenthesis after click. Next, we’re going to send a screenshot command to the computer itself. To do so, we’re going to use the type command, which conveniently types content into the screen. You can also use this for dialog boxes and other such things, but we’re going to just use it to send a screenshot command to the server. To do so, we’re going to tell it to type the 3 key, along with KeyModifier.SHIFT and KeyModifier.CMD:

type("3", KeyModifier.SHIFT + KeyModifier.CMD)

Now your screen will look something like this:

Running the workflow should result in a screenshot on your desktop, of the users on the system. Next, let’s just repeat this process by clicking on all the screens, grabbing a little screenshot of each and producing results on the desktop. All we’re going to do is cut/copy/paste the actual commands and then loop through each of the services and screens in Server Admin until we’ve gotten a screenshot of each:

Now is when things can start getting a little more interesting. We could have done everything we did up to now using Automator. But Sikuli has some built in logic. Here, we’re going to use the wait option in the Find section of the sidebar to go ahead and wait each screen that can be latent to show content. After we click the services, we’ll wait for a pattern on the screen not specific to any given system but that only appears when the settings appear, as follows:

This is a basic configuration of Sikuli, with a simple task, take some screenshots of the Server application. It can then be anchored to an imaging workflow by invoking the workflow from the command line. The command line is just a shell script sitting in the Sikuli-IDE, which is by default dropped into Applications. Because you won’t need it for standard systems, you can use it in special regression testing workflows from your imaging solution. Save the workflow you’ve been running as some title (I’ll use ServerApp) with the .sikuli extension. Then, to run it from the command line, fire up sikuli along with the actual project and a -r operator to tell it to run:

/Applications/Sikuli-IDE.app/sikuli-ide.sh ~/Desktop/ServerApp.sikuli -r

You can also use the sikuli-ide.jar or sikuli-script.jar to get an even lighter install, documented at http://sikuli.org/docx/faq/010-command-line.html#cmdoption-t on the sikuli.org site. On that page, it also explains how to pass arguments to Jython’s sys.argv using the –args option as well as using -stderr. The scripting environment from there becomes jython, a mashup language that takes python and java and uses a little duct tape to hold them together. Overall, it’s an interesting concept. There isn’t a lot of logic, unless you’re willing to script things. But if you are willing to script things then you can do pretty much anything you might want. For example, you can have it play bejeweled for you, so you can actually get some work done (although perhaps you’d rather play bejeweled and script your meetings…):