Apple,  Machine Learning,  Machine Learning And Artificial Intelligence,  Programming,  Swift

New App TestFairy Writes Unit Tests With AI

I just posted a new app to GitHub called TestFairy. It’s at https://github.com/krypted/TestFairy. It automatically generates unit tests for either a file or a highlighted piece of swift code. It does this using the OpenAI API, so leverages a LLM to write test code. It’s pretty straight forward. Simply highlight the code and in the Editor Menu, click TestFairy, then Generate Tests.

I had hoped to post it for free to the App Store, but I didn’t want to distribute my OpenAI API key, or write an intermediary microservice that housed my key that an app uses. The use of an API key is considered an end-around to the In-App Purchase flow, so even though it isn’t my API or a key you pay me for, it’s still something I would effectively need to charge a subscription for to fund the API calls with OpenAI (assuming anyone actually uses it).

I had always planned to put the source out there so it’s on GitHub, as well as a compiled/notarized/archive app bundle, so it should run fairly easily. One thing about extensions, though, is they need to be enabled to run. This type of Extension is an Xcode Source Editor extension. Those are enabled in System Settings->Privacy & Security->Extensions->Xcode Source Editor and then check the box.

Once enabled, open the Test Fairy app and enter the API key. Then just hide that window (the extension will exit memory when the app is closed).

If you see the app in the Xcode Editor menu then it is either that the extension was not enabled or that the API key isn’t present. If you get an error when you invoke the extension – the issue is likely that the API key doesn’t actually have access to perform the operation in question (I’ve noticed free OpenAI accounts don’t work properly).

I really wanted to promote quality code (for myself if noone else) with this little project. I spend a fair amount of time writing unit tests for stuff that goes to production and also thought it would be cool to write an Xcode Source Editor extension, as I hadn’t done so before. I still spend time working on unit tests. Some that OpenAI comes up with aren’t very good – and a targeted piece of code doesn’t identify failures with larger atomic operations that cross files.

I’ve posted a number of apps over the years and never charged for them, so the idea of jumping straight from that to a subscription app isn’t really what I wanted to do. I used a fully open license when I published this, though. So if someone else wanted to take it, add that subscription model, and pipe requests through your own Bard/GPT-J/OpenAI gateway endpoint, more power to ya’. I might just be your first customer!