krypted.com

Tiny Deathstars of Foulness

When you’re building and manipulating apps in the Apple App Stores, it helps to be able to pull and parse pieces of data. Here, we’ll look at two strategies that you can use to do so. It’s worth noting that the purpose of this was to use the URL of an app from an MDM and then be able to script updating metadata about the app, given that vendors often change names of the display name of an app (e.g. Yelp is actually called “Yelp: Discover Local Favorites on the App Store”).

First, we’ll grab a URL. This one is for Self Service:

https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8

If you don’t know the URL then you can get it based on the ID by parsing the json from:

curl https://itunes.apple.com/lookup?id=718509958

Of course, if you know the id, you can probably just assume that https://itunes.apple.com/us/app/id718509958?mt=8 will work as well, since if you remove the name it has always worked for me (although I’ve never seen that in a spec so I can’t guarantee it will always be true). Then, we can curl it, but the output is a bit not lovely:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8'

So then we’ll want to just grab the pieces of information we want, which could be done using a variety of scripting techniques. Below, we’ll use grep:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | grep -o "<title>[^<]*" | cut -d'>' -f2-

And here, we’ll use perl:

curl -s 'https://itunes.apple.com/us/app/yelp/id284910350?mt=8' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si'

And there you go, you have the title. The title is easy, because it’s a simple title tag. But let’s look at the description:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | awk '/meta name="description"/{;print }'

The output would be similar to the following 

<meta name="description" content="Read reviews, compare customer ratings, see screenshots, and learn more about Self Service Mobile. Download Self Service Mobile and enjoy it on your iPhone, iPad, and iPod touch." id="ember75894226" class="ember-view">

From there it’s pretty simple to extract the exact field you want and the metadata from that field. If you are obtaining names and descriptions for a large number of apps then you’d simply move the path into a variable as follows so you can put it into your loop:

curl -s $appurl | grep -o "<title>[^<]*" | cut -d'>' -f2-

I haven’t covered finding items in the App Store if you don’t know the ID of an app, but there’s a /search endpoint at iTunes.apple.com that will respond to a variety of parameters you can pass:

curl https://itunes.apple.com/search?term=yelp&country=us&entity=software

This wasn’t necessary for my use case. But it’s worth noting. And if you’ll be doing a lot of that, I’d recommend checking out the affiliates portal at https://affiliate.itunes.apple.com/resources/documentation/itunes-store-web-service-search-api/. Additionally, if you’re actually trying to automate the App Store instead, there are a few tools out there to do so, including https://github.com/mas-cli/mas or if you want to extract packages there’s https://github.com/maxschlapfer/MacAdminHelpers/tree/master/AppStoreExtract

March 2nd, 2018

Posted In: Apple Configurator, Apps, iPhone, Mac OS X

Tags: , , , , , , , ,

A number of commands available for finding positions that you want in a line and extracting only a certain amount of text can be pretty cumbersome in terms of learning curve. This isn’t to say that once you get the hang of them that they’re terribly complicated but it can take a little while to get the hang of them. And when you need something fast, you might want an easy command for extracting text from lines. In these cases, consider cut. The cut command doesn’t do regular expressions (I guess you could argue that its ability to use a delimiter can be used as a regular expression) and so it’s really easy to use. Basically, you feed cut some data and then tell it which characters in the line that you want to keep. It then gets rid of the rest. The easiest use of this is to look at a list of data. For example, let’s saw we have a file called test.txt with the following contents:
abc123 abc124 abc134 abc234 abd234 acd234
Now we’re going to cat the file (which just reads the file contents) and then pipe the output of reading that file into a cut command (which is done by simply adding a pipe character at the end of the first part of the command. Then we’re going to use the -c option of cut (which looks at character positions) to simply grab the first three positions (1-3) of the lines. The command would end up looking as follows:
cat test.txt | cut -c 1-3
And the output would look as follows (this output could then be redirected into a new file btw):
abc abc abc abc abd acd
You can also specify multiple ranges of characters (or single characters for that matter). For example, to see only characters 1-2 and 5-6:
cat test.txt | cut -c 1-2,5-6
Overall, cut is a very easy to use tool, with a limitation that your pattern that you are looking to maintain must be consistent in terms of the character position that you are using in each line. It also uses every line in a file; however, to go another step and look for all positions in a line only if the line has a pattern that it can match you could simply add a grep in the middle. For example, if you’re looking for each line of our sample text file that has the number 4 then you could do:
cat test.txt | grep 4
This would show you only the last five lines of the file since those are the only lines that have that number in them. You could then pipe the output of that file into your cut and, let’s say, look for characters 1-3 and 6 in the output:
cat test.txt | grep 4 | cut -c 1-3,6
Your result would then be the following:
abc4 abc4 abc4 abd4 acd4
Finally, there are going to be times when you’re not looking for a specific character position in a line but instead a character position or a pattern that begins with another pattern. For this you’re going to end up needing to use a more advanced tool, such as awk or if you’re feelin’ frisky (maybe I’m speaking for myself there) regex. These tools will have a steeper learning curve, but ultimately be far more useful.

January 17th, 2010

Posted In: Mac OS X, Mac OS X Server, Mac Security, Unix

Tags: , , , , , , , , ,

A regular expression “engine” is a piece of software that can process regular expressions, trying to match the pattern to the given string. Usually, the engine is part of a larger application and you do not access the engine directly. Rather, the application will invoke it for you when needed, making sure the right regular expression is applied to the right file or data.

June 15th, 2007

Posted In: Unix

Tags: ,