krypted.com

Tiny Deathstars of Foulness

When you’re building and manipulating apps in the Apple App Stores, it helps to be able to pull and parse pieces of data. Here, we’ll look at two strategies that you can use to do so. It’s worth noting that the purpose of this was to use the URL of an app from an MDM and then be able to script updating metadata about the app, given that vendors often change names of the display name of an app (e.g. Yelp is actually called “Yelp: Discover Local Favorites on the App Store”).

First, we’ll grab a URL. This one is for Self Service:

https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8

If you don’t know the URL then you can get it based on the ID by parsing the json from:

curl https://itunes.apple.com/lookup?id=718509958

Of course, if you know the id, you can probably just assume that https://itunes.apple.com/us/app/id718509958?mt=8 will work as well, since if you remove the name it has always worked for me (although I’ve never seen that in a spec so I can’t guarantee it will always be true). Then, we can curl it, but the output is a bit not lovely:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8'

So then we’ll want to just grab the pieces of information we want, which could be done using a variety of scripting techniques. Below, we’ll use grep:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | grep -o "<title>[^<]*" | cut -d'>' -f2-

And here, we’ll use perl:

curl -s 'https://itunes.apple.com/us/app/yelp/id284910350?mt=8' | perl -l -0777 -ne 'print $1 if /<title.*?>\s*(.*?)\s*<\/title/si'

And there you go, you have the title. The title is easy, because it’s a simple title tag. But let’s look at the description:

curl -s 'https://itunes.apple.com/us/app/self-service-mobile/id718509958?mt=8' | awk '/meta name="description"/{;print }'

The output would be similar to the following 

<meta name="description" content="Read reviews, compare customer ratings, see screenshots, and learn more about Self Service Mobile. Download Self Service Mobile and enjoy it on your iPhone, iPad, and iPod touch." id="ember75894226" class="ember-view">

From there it’s pretty simple to extract the exact field you want and the metadata from that field. If you are obtaining names and descriptions for a large number of apps then you’d simply move the path into a variable as follows so you can put it into your loop:

curl -s $appurl | grep -o "<title>[^<]*" | cut -d'>' -f2-

I haven’t covered finding items in the App Store if you don’t know the ID of an app, but there’s a /search endpoint at iTunes.apple.com that will respond to a variety of parameters you can pass:

curl https://itunes.apple.com/search?term=yelp&country=us&entity=software

This wasn’t necessary for my use case. But it’s worth noting. And if you’ll be doing a lot of that, I’d recommend checking out the affiliates portal at https://affiliate.itunes.apple.com/resources/documentation/itunes-store-web-service-search-api/. Additionally, if you’re actually trying to automate the App Store instead, there are a few tools out there to do so, including https://github.com/mas-cli/mas or if you want to extract packages there’s https://github.com/maxschlapfer/MacAdminHelpers/tree/master/AppStoreExtract

March 2nd, 2018

Posted In: Apple Configurator, Apps, iPhone, Mac OS X

Tags: , , , , , , , ,

This New Years Day, Learn The Jot Command The jot command is one I haven’t used in awhile. But it’s still useful. Let’s take a look at a few of the things you can do with it. First, let’s just print lines into a new file called “century.txt” which we can do by running with the number of increments followed by the starting number, and then redirecting the output into the file name:

jot 100 1 > ~/Desktop/century.txt

Or to do integers instead, simply put the decimals:

jot 100 1.00 > ~/Desktop/century.txt

Or in descending order,

jot – 100 1 > ~/Desktop/century.txt

Now we could change the output to be just 50 to 100, by incrementing 50 (the first position) and starting at 50 (the second):

jot 50 50

The jot command is able to print sequential data, as we’ve seen. But we can also print random data, using the -r option. Following that option we have three important positions, the first is the number of iterations, but the next two are the lower and upper boundaries for the numbers, respectively. So in the below command we’ll grab 10 iterations (or ten random numbers) that are between 1 and 1000:

jot -r 10 1 1000

Now if we were to add a -c in there and use a and z as the upper and lower bounds, we’d get… letters (this time we’re just gonna’ ask for one letter)!

jot -r -c 1 a z

Something I find useful is just to shove random data into a file infinitely. And by useful I mean hopefully not left running overnight on my own computer (been there, done that). To do this, just use a 0 for the number of iterations:

jot -r -c 0

Something that is actually useful is the basic ASCII set:

jot -c 128 0

We can also append data to a word using -w. So let’s say we want to print the characters aa followed by a through z. In the below we’ll define that with -w and then we’ll list those two characters followed by %c which is where the character substitution goes and then the number of iterations followed by the lower bound:

jot -w aa%c 26 a

You can also do stuttering sequences, useful for the occasional tango dancer, so here we’ll do a 5/3 countdown:

jot – 100 0 -.5

Or we could create a one meg file by creating 1,024 bytes:

jot -b 0 1024 > onemegfile.txt

Oh wait, that file’s two megs. Get it? 😉

And running strings teaches you that you can’t bound random (a good lesson for the New Year). Anything you use jot for?

Happy New Years!

January 1st, 2018

Posted In: bash, Mac OS X, Mac OS X Server

Tags: , , , , ,

Server comes with a command called RoomsAdminTool located at /Applications/Server.app/Contents/ServerRoot/usr/bin/RoomsAdminTool. This tool can list available rooms using a -l flag:

RoomsAdminTool -l

You can also create new rooms, using the following format, where krypted is the name of the room, the persistent option means the room is, er, persistent. The description option indicates a description used for the room.

RoomsAdminTool -n krypted -c persistent yes description "This room is for friends of krypted only”

To then delete the room, use the -d option:

RoomsAdminTool -n krypted -d

Add the -v to do it all verbosely. There are lots of other options as well, as follows (from the man page): Valid Configuration Keys and Values:
KEYVALID VALUESDESCRIPTION
descriptionstringA short description for the room
passwordstringDefine a password for room entry. An empty string implies no password required.
membersOnlyyes | noOnly room members are allowed to enter the room.
subjectLockedyes | noAre non-moderators and non-admins prevented from setting the room subject
logFormatDisabled | Text | XHTMLDisable room logging, or enable it using Text or XHTML.
maxUsersinteger; 0 for unlimitedSet the maximum allowed occupants for the room.
moderatedyes | no Make the room "moderated".
nonAnonymousyes | noIf "yes", only moderators/owners can discover occupants' real JIDs.
persistentyes | noPersistent rooms stay open until they are explicitly destroyed and their configuration survives service restarts, unlike non-persistent rooms.
privateMessagesAllowedyes | no Whether or not occupants can exchange private messages within the room.
roomPublicyes | no Defines whether the room be discovered by anyone
subjectstringSet a room subject/topic
usersCanInviteyes | no Defines whether occupants can invite other users to enter the room
addOwnervalid JabberIDMake the specified user a room owner (ex.: admin@krypted.com). Rooms can have multiple owners.
removeOwnervalid JabberIDRemove the specified user from the room owner list
addAdminvalid JabberIDMake the specified user a room admin
removeAdminvalid JabberIDRemove the specified user from the room admin list
addMembervalid JabberIDMake the specified user a room member
removeMembervalid JabberIDRemove the specified user from the room member list
addOutcastvalid JabberIDMake the specified user a room outcast (banned from public rooms)
removeOutcastvalid JabberIDRemove the specified user from the room outcast list
Ultimately, if you’d like to do Student Information System (SIS) integration, or wait for an AD/OD group and then programmatically generate rooms, this is how you’d do it. Also, it’s worth noting that Messages (and so Jabber if you’re running your own server) is a very basic instant messaging tool. There are more modern ways of interacting with others these days, including Slack and Confluence. Additionally, the Messages app can just use the phone number of people to let address books become a way of managing groups you’d like to message. These do not require a dedicated server, but most strategies will require a monthly fee that’s typically per-user.

September 9th, 2017

Posted In: Mac OS X Server

Tags: , , , , ,

Wordpress has an app. That means there’s an API to normalize communication using a predictable programmatic interface. In this case, as with many others, that’s done using a standard REST interface to communicate. The easiest way to interact with any API is to just read some stuff from the server via curl. You can feed curl the URL to the API by using your URL followed by /wp-json – as follows, assuming a URL of http://www.krypted.com: curl http://www.krypted.com/wp-json To view header information: curl -s -D - http://www.krypted.com -o /dev/null In the below example we’ll ask for a list of posts by adding /wp/v2/posts to the URL: curl http://www.krypted.com/wp-json/wp/v2/posts You’ll see a list of some posts in the output along with a little metadata about the posts. You can then grab an ID and ask for just that post, using a post ID of 48390: curl http://www.krypted.com/wp-json/wp/v2/posts/48390 You can also see revisions that have been made to a post by appending the URL with /revisions curl http://www.krypted.com/wp-json/wp/v2/posts/48390/revisions You can see comments with the comments route: curl http://www.krypted.com/wp-json/wp/v2/comments Or pages with the pages route: curl http://www.krypted.com/wp-json/wp/v2/pages Or users with the users route: curl http://www.krypted.com/wp-json/wp/v2/users Or media that has been uploaded with the media route: curl http://www.krypted.com/wp-json/wp/v2/media And the output of each can be constrained to a single item in that route by providing the ID of the item, which shows additional metadata about the specified item. And there are routes for categories, tags, etc. There’s also some good stuff at https://github.com/WP-API such as https://github.com/WP-API/Basic-Auth which is a plugin that allows you to auth against the API. curl --user admin:krypted http://www.krypted.com/wp-json/users/me Not only can you look at user information, you can also add and remove posts. You would add by doing a -X followed by a POST and then feeding a file with the –data option curl --user admin:password -X POST http://www.krypted.com/wp-json/posts --data @post.json The output would then include the ID of your new post to wordpress. In the following example, we’ll get rid of the post we were looking at earlier using -X and DELETE in the URL, assuming a username of admin, a password of krypted, and a post ID of 48390: curl --user admin:krypted -X DELETE http://www.krypted.com/wp-json/posts/48390 If successfully deleted the response would be as follows:
{ “message”:”Deleted post” }
To dig in deeper, check out http://v2.wp-api.org/reference/posts/ where the whole schema is documented. You can also use the https://github.com/WP-API GitHub site to access a command called wp (as well as PHP, node, and java clients) that can be run at the command line for simple scripting interfaces. This could allow you to, for example, simply backup posts to json files, etc. Also, it’s worth noting that various plugins will require their own interface (note there’s no themes or plugins route), such as woocommerce, interfacing with http://gerhardpotgieter.com/2014/02/10/woocommerce-rest-api-client-library/ or https://woocommerce.github.io/woocommerce-rest-api-docs/.

July 14th, 2017

Posted In: WordPress

Tags: , , , , , , , ,

To tell curl that you can read and write cookies, first we’ll start the engine using an empty cookie jar, using the -b option, which always reads cookies into memory: curl -b newcookiejar http://krypted.com If your site can set cookies you can then read them with the -L option curl -L -b newcookiejar http://krypted.com The response should be similar to the following:
Reading cookies from file
Curl also supports reading cookies in from the Netscape cookie format, used by defining a cookies.txt file instead: curl -L -b cookies.txt http://krypted.com If the server updates the cookies in a response, curl would update that cookie in memory but unless you write something that looks for a new cookie, the next use will read the original cookie again. To create that file, use the -c option (short for –cookie-jar) as follows: curl -c cookie-jar.txt http://krypted.com This will save save all types of cookies (including session cookies). To differentiate, curl supports junk, or session cookies using the –junk-session-cookies options, or -j for short. The following can read these expiring cookies: curl -j -b cookie-jar.txt http://krypted.com Use that to start a session and then that same -c to call them on your next use. This could be as simple as the following: CURL=/usr/bin/curl COOKIEJAR=cookie-jar.txt SITE=http://krypted.com $CURL -j -b $COOKIEJAR $site You can also add a username and password to the initial request and then store the cookie. This type of authentication and session management is used frequently, for example in the Munkireport API, as you can see here: For converting, the -b detects if a file is a Netscape formatted cookie file, parses, then rewrites using the -c option at the end of a line: curl -b cookie.txt -c cookie-jar.txt http://krypted.com

February 20th, 2017

Posted In: Mac OS X, Mac Security

Tags: , , , , , ,

When you’re regression testing, you frequently just don’t want any delays for scripts unless you intentionally sleep your scripts. By default Safari has an internal delay that I’d totally forgotten about. So if your GUI scripts (yes, I know, yuck) are taking too long to run, check this out and see if it helps: defaults write com.apple.Safari WebKitInitialTimedLayoutDelay 0 With a script I was recently working on, this made the thing take about an hour less. Might help for your stuffs, might not. If not, to undo: defaults delete com.apple.Safari WebKitInitialTimedLayoutDelay Enjoy.

February 1st, 2017

Posted In: Mac OS X Server, Mac Security

Tags: , , , , , , , ,

Dropping network connections can be incredibly frustrating. And finding the source can be a challenge. Over the years, I’ve found a number of troubleshooting methods, but the intermittent drop can be the worse to troubleshoot around. When this happens, I’ve occasionally resorted to scripting around failures, and dumping information into a log file to find the issue. For example, you may find that when a network connection fails, you have a very strong signal somewhere, or that you have a very weak signal on all networks. I’ve found there are three pretty simple commands to test joining/unjoining, and using networks (beyond the standard pings or port scans on hosts). The first is the airport command, along with –disassociate. This just unjoins all networks: sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport --disassociate The second is a quick scan. Here, I’ve grep’d out the network I’m after (aka SSIDofNetwork – a very likely wireless network name), but when looking for environmental issues, you might choose to parse this into a csv and output all networks: sudo /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport -s | grep SSIDofNetwork Finally, you can join a network. You might have to escape out special characters in a password and it’s never wise to put a password into a script, etc. But, quick and dirty, this will join that SSIDofNetwork network: sudo networksetup -setairportnetwork en0 "SSIDofNetwork" mysecretpassword Anyway, loop it, invoke it however you invoke it, etc. Hope this helps someone, and if you have other tricks you’ve found helpful, feel free to throw them in the ‘ole comments!

How Users Feel About Intermittent Networking Issues

August 26th, 2016

Posted In: Mac OS X, Mac OS X Server, Mac Security, Network Infrastructure, Programming

Tags: , , , , , , ,

Posted a new swift command line tool to accept serial number data from an Apple device and respond with warranty information about a device at https://github.com/krypted/swiftwarrantylookup. This is based on pyMacWarranty, at https://github.com/pudquick/pyMacWarranty. Screen Shot 2016-03-15 at 10.34.41 PM

March 16th, 2016

Posted In: Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Swift

Tags: , , , , , , , , ,

One of the options thats a tad bit hidden in OS X is the Secure Erase option, which runs a multi-pass erase on a volume. Additionally, there’s no option to Secure Erase free space on a volume. But you can still securely erase whatever you’d like (other than you boot volume obviously), when needed. To do so, use the diskutil command along with the secureErase option. Screen Shot 2016-01-07 at 7.44.07 AM The format of the command to secureErase freespace is:
diskutil secureErase freespace [level] [device]
The levels are as follows (per the man page as not all of these are specified in Disk Utility):
  1. Single-pass zero-fill erase
  2. Single-pass random-fill erase
  3. US DoD 7-pass secure erase
  4. Gutmann algorithm 35-pass secure erase
  5. US DoE algorithm 3-pass secure erase
So for example, let’s say you had a volume called Seldon and you wanted to do a standard Single-pass zero-fill erase. In this example you would use the following:
diskutil secureErase freespace 0 /Volumes/Seldon
If you were to automate the command then you would want to dump the output into a log file. For example:
diskutil secureErase freespace 0 /Volumes/Seldon > /var/log/secureeraselog.tmp
You can also secureErase a volume itself. To erase a volume called /Volumes/Seldon, use the same structure of the command, but this time without the freespace option: diskutil secureErase 0 /Volumes/Seldon The latest update to Disk Utility removes a lot of options from the GUI, but overall, I have yet to find a scenario where a task I need to perform isn’t still available, if only from the command line.

January 7th, 2016

Posted In: Mac OS X, Mac OS X Server, Mac Security, Mass Deployment

Tags: , , , , , , , ,

Pretty much every script I’m working on these days must be run as root. Checking what user is running something is pretty straight forward, as there’s a built-in shell variable for $USER that contains the user running a script. To see this real quick, simply run the following: echo $USER You can then put this into your scripts. I’ve been using the same block of code for decades, which can be run in a script by itself if you’d like to paste this into one. if [[ $USER != "root" ]]; then echo "This script must be run as root" else echo "You are root" exit 1 fi Note: Keep in mind that the built-in $USER variable is case sensitive. Obviously, most people won’t keep the lines that contain the else and you are root echo statements. You can just remove these or replace them with the meat of your script that requires elevated privileges to run. Enjoy.

December 21st, 2015

Posted In: Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Unix

Tags: , , , , , , , ,

Next Page »