When you’re building and manipulating apps in the Apple App Stores, it helps to be able to pull and parse pieces of data. Here, we’ll look at two strategies that you can use to do so. It’s worth noting that the purpose of this was to use the URL of an app from an MDM and then be able to script updating metadata about the app, given that vendors often change names of the display name of an app (e.g. Yelp is actually called “Yelp: Discover Local Favorites on the App Store”).

First, we’ll grab a URL. This one is for Self Service:

https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8

If you don’t know the URL then you can get it based on the ID by parsing the json from:

curl https://itunes.apple.com/lookup?id=718509958

Of course, if you know the id, you can probably just assume that https://itunes.apple.com/us/app/id718509958?mt=8 will work as well, since if you remove the name it has always worked for me (although I’ve never seen that in a spec so I can’t guarantee it will always be true). Then, we can curl it, but the output is a bit not lovely:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8'

So then we’ll want to just grab the pieces of information we want, which could be done using a variety of scripting techniques. Below, we’ll use grep:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | grep -o "<title>[^<]*" | cut -d'>' -f2-

And here, we’ll use perl:

curl -s 'https://itunes.apple.com/us/app/yelp/id284910350?mt=8' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si'

And there you go, you have the title. The title is easy, because it’s a simple title tag. But let’s look at the description:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | awk '/meta name="description"/{;print }'

The output would be similar to the following 

<meta name="description" content="Read reviews, compare customer ratings, see screenshots, and learn more about Self Service Mobile. Download Self Service Mobile and enjoy it on your iPhone, iPad, and iPod touch." id="ember75894226" class="ember-view">

From there it’s pretty simple to extract the exact field you want and the metadata from that field. If you are obtaining names and descriptions for a large number of apps then you’d simply move the path into a variable as follows so you can put it into your loop:

curl -s $appurl | grep -o "<title>[^<]*" | cut -d'>' -f2-

I haven’t covered finding items in the App Store if you don’t know the ID of an app, but there’s a /search endpoint at iTunes.apple.com that will respond to a variety of parameters you can pass:

curl https://itunes.apple.com/search?term=yelp&country=us&entity=software

This wasn’t necessary for my use case. But it’s worth noting. And if you’ll be doing a lot of that, I’d recommend checking out the affiliates portal at https://affiliate.itunes.apple.com/resources/documentation/itunes-store-web-service-search-api/. Additionally, if you’re actually trying to automate the App Store instead, there are a few tools out there to do so, including https://github.com/mas-cli/mas or if you want to extract packages there’s https://github.com/maxschlapfer/MacAdminHelpers/tree/master/AppStoreExtract

There’s a macOS tool called AssetCacheLocatorUtil located at /usr/bin/AssetCacheLocatorUtil. The output is in… stderr. Because stderr is so fun to work with (note that sed -i only works with stdin). So, to update the caching server(s) you are using and only print the IP address of those, you’d do the following: /usr/bin/AssetCacheLocatorUtil 2>&1 | grep guid | awk '{print$4}' | sed 's/^\(.*\):.*$/\1/' | uniq If you use Jamf Pro and would like to use this as an extension attribute, that’s posted here: https://github.com/krypted/cachecheck. I didn’t do any of the if/then there, as I’d usually just do that on the JSS.

You search for items in macOS using compound conditions in a number of ways. One way is with awk. Here, we’re going to grab the output of a simple ls command. That gets piped into an awk statement. Then we’re going to look at the expression to evaluate. Basically, we’re going to say anything that contains com. as well as apple and .plist. Because it’s ls, we’re looking for names of files that match those patterns. Each pattern is listed in brackets. And then there’s the {print} to lay out the action of printing to the files that match the pattern to the screen: ls |awk '/[com.][apple][.plist]/ {print}' Note: I know you’re not supposed to use ls in scripts. Don’t care. If it were dates and such, I’d of used stat…

One of my favorite things about grabbing things with scripts is just how many ways (and sometimes how needfully or needlessly convoluted you can make them) to grab the same pieces of information. For example, something as simple as what hosts you use to resolve names on a Mac. There are a number of ways to grab what DNS server a device is using in macOS. So when you’re running a script you might choose to grab DNS information one way or another, according to what you’re after. Some of this might seem more complicated than it should be. And that’s correct…

resolv.conf

The /etc/resolv.conf file is updated automatically to look at what servers are used to resolve names used for DNS. The easiest way to see theses to simply cat it and grep for nameserver: cat /etc/resolv.conf | grep nameserver

scutil

The next way we’ll grab DNS information is using scutil. Here, we use the –dns option, which outputs a lot of DNS stuffs, including all the built-in resolvers: scutil --dns To just grab the name servers: scutil --dns | grep nameserver We can also simplify the output to just the servers with awk: scutil --dns | grep nameserver | awk '{print$3}'

networksetup

The second way is using networksetup. This command has an option to get a DNS server in (shocker) -getdnsservers. However, you have to list the interface for each. So below we’ll dump all interfaces into an array using -listallhardwareports and then read them in using a for loop and querying the name servers. interfaces=( "$(networksetup -listallhardwareports | grep Hardware | cut -c 16-900)" ) for i in "${interfaces[@]}" do networksetup -getdnsservers $i done The one tricky thing in this one is I initially forgot to quote the interfaces as they went into the array, which meant each word of the interface was an item in the array and therefore the -getdnsservers option failed. Once I quoted, it was all happy. The other thing I can point out is I used cut instead of sed because it was easier to quote; however, it seems unlikely the name can be more than 890 characters, so I think it’s fine…

dig

You can also use dig. Here, you’ll query for a name without using an @ option, but omit everything but the line with the server that responded: dig google.com | grep SERVER: The output is kinda’ fug:
;; SERVER: 4.2.2.2#53(4.2.2.2)
For simpler output, we’ll use sed to constrain the output to just what’s between the parenthesis: dig google.com | grep SERVER: | sed 's/^.*(//;s/)$//'

nslookup

nslookup is a tool similar to dig, used for querying names. We’ll basically do the same thing as above, just using awk as it’s just a standard position in a line: nslookup google.com | grep Server: | awk '{print$2}'

system_profiler

Then there’s system_profiler, the command line interface for System Profiler. Here, we can query the SPNetworkDataType. This is going to produce a lot of output, so we can limit it to just the DNS servers using grep to constrain to just the lines we want and awk for just the columns in those lines, as follows: system_profiler SPNetworkDataType | grep "Domain Name Servers:" | awk '{print$4}'

hosts

@knapjack added to use hosts. I had to use verbose mode to pull the local name server as follows: host -v -t ns google.com | grep Received | awk '{print $5}'

ipconfig

Thanks to the lovely Allister (@sacrilicious), we also have ipconfig to add to the list: /usr/sbin/ipconfig getpacket en0 2> /dev/null | grep name_ | cut -d' ' -f3- There are tons of ways to find things in macOS. Do you have a way to find a DNS server that I didn’t think of here?

Namespace conflicts can be interesting. Especially with multiple local domains. To grab the path of a directory domain of a currently logged in user (when running as the user) using a script, you can run the following: dscl . -read /Users/`whoami` | grep AppleMetaNodeLocation | awk '{print $2}' You can then replace the string we’re using with grep if you’d like to pull a different attribute from the user record, you’d use the following: dscl . -read /Users/`whoami` | grep UniqueID | awk '{print $2}'

Recently I was working on a project where we were isolating IP addresses by country. In the process, I found an easy little tool built right into OS X called ip2cc. Using ip2cc, you can lookup what country an IP is in. To do so, simply run ip2cc followed by a name or ip address. For example, to lookup apple.com you might run: ip2cc apple.com Or to lookup Much Music, you might run: ip2cc muchmusic.ca The output would be: IP::Country modules (v2.28) Copyright (c) 2002-13 Nigel Wetters Gourlay Database updated Wed May 15 15:29:48 2013 Name: muchmusic.com Address: 199.85.71.88 Country: CA (Canada) You can just get the country line: ip2cc apple.com | grep Country: To just get the country code: ip2cc apple.com | grep Country: | awk '{ print $2 }' Finally, ip2cc is located at /usr/bin/ip2cc so we’ll complicate things just a tad by replacing the hostname with the current IP (note that private IPs can’t be looked up, so this would only work if you’re rocking on a wan ip or feeding it what a curl from a service like whatismyip brings back): ip2cc `ipconfig getifaddr en0` | grep Country: | awk '{ print $2 }'

The number of characters n a line of text can be a difficult thing to calculate in a given app. If you have data in a text file, you can use awk to view the number of characters in a given line of the file. This is very helpful, for example, if you have code that you’ve put into documentation that exceeds the character maximum and therefore wraps. When going to print, you need to split these lines up. Here, we’re going to use the awk command to review all lines that exceed 56 characters: awk 'length($0) > 56' ~/Repo/Chapter1.xml

Yesterday I showed a way to get the serial number from a Mac OS X machine. However, as a couple of people pointed out, Apple will soon be adding another character to the serial number. This means that rather than use cut I should have used awk to allow for either serial number length. To grab the serial this way:
ioreg -l | grep IOPlatformSerialNumber | awk ‘{print $4}’
Or without the quotes:
ioreg -l | grep IOPlatformSerialNumber | awk ‘{print $4}’ | sed ‘s/”//g’