krypted.com

Tiny Deathstars of Foulness

Apple has a number of different logging APIs. For the past few releases, Apple has tried to capture everything possible in logs, creating what many administrators and developers might consider to be a lot of chatter. As such, an entirely new interface needed to be developed to categorize and filter messages sent into system logs.

Writing Logs

The logger command is still used to create entries in system logs. However, if you are then using tail to view /var/log/system.log then you will notice that you no longer see your entry being written. This is because as the logs being created in macOS have gotten more complex, the tools to read and write those logs has gotten more complicated as well.

Let’s take a simple log entry. Below, we’ll write the string “Hello Logs” into the system log. To do so, use the –i option to put the process id of the logger process and –s to write to the system log, as well as to stderr. To make the entry easier we’ll tag it with –t followed by the string of the tag. And finally, we’ll quote the entry we want written into the log. This is basically the simplest form of an entry:

logger -is -t krypted "Hello Logs"

Once written, use the log command to read your spiffy new entries. This isn’t terribly different than how things worked previously. If you’re a developer, you will need to note that all of the legacy APIs you might be using, which include asl_log_message, NSLog, and syslog, have been redirected to the new Unified Logging system, provided you build software for 10.12 (you can still build as before for 10.11, iOS 9, tvOS 10, and watchOS 3 and below). These are replaced with the os_log, os_log_info, os_log_debug, os_log_error, os_log_fault, and os_log_create APIs (which correspond to various levels of logs that are written).

Reading Logs

Logs are now stored in the tracev3 formatted files in /var/db/diagnostics, which is a compressed binary format. As with all binary files, you’ll need new tools to read the files. Console has been updated with a new hierarchical capability and the ability to watch activities, subsystems, etc.

The log command provides another means of reading those spiffy new logs. To get started, first check out the man page:

man log

That “Hello Logs” string we used earlier is part of a message that you can easily view using the ‘log show’ command. In the below example, we’ll just run a scan of the last 3 minutes, using the –last option, and then providing a –predicate. We’ll explain those a bit later, but think of it as query parameters – here, we’ll specify to look for “Hello Logs” in eventMessage:

log show --predicate 'eventMessage contains "Hello Logs"' --last 3m

Filtering the log data using “eventMessage CONTAINS “Hello Logs”” shows us that our entry appears as follows:

Timestamp                       Thread     Type        Activity             PID

2017-03-23 23:51:05.236542-0500 0x4b83bb   Default     0x0                  88294  logger: Hello Logs

——————————————————————————————————————–

Log      – Default:          1, Info:                0, Debug:             0, Error:          0, Fault:          0

Activity – Create:           0, Transition:          0, Actions:           0

How do you find out what to use where? Here’s an example where I’m going to try to find all invalid login attempts. First, I’m just going to watch the logs. Many will prefer the “log stream’ command. I’m actually going to just use show again, because I like the way it looks more. I’m also going to use log with the syslog –style so it’s easier to read (for me at least), and then here I’m just looking at everything by specifying a space instead of an actual search term:

log show --style syslog --predicate 'eventMessage contains " "' --info --last 24h

Looking at the output, you can see an entry similar to the following:

2017-03-23 14:01:43.953929-0500  localhost authorizationhost[82865]: Failed to authenticate user <admin> (error: 9).

Oh, I’ve got to just search for Failed to authenticate user” and I’ll be able to count invalid login attempts. To then take this and place it into a command that, for example, I could build an extension attribute using, I can then just find each entry in eventMessage that contains the string, as follows:

log show --style syslog --predicate 'eventMessage contains "Failed to authenticate user"' --info --last 1d

As with many tools, once you have a couple of basic incantations, they become infinitely simpler to understand. These few commands basically get you back to where you were with tailing logs. If you want to get that –f functionality from tail, to watch the logs live, just swap show with stream. The most basic incantation of this would just be ‘log stream’ without bothering to constrain the output:

log stream

Running this is going to spew so much data into your terminal session. So to narrow down what you’re looking for, let’s look at events for Twitter:

log stream --predicate 'eventMessage contains "Twitter"'

You can also view other logs and archives, by calling a file name:

log show system_logs.logarchive

Organization and Classification

The new logging format also comes with Subsystems. If you’re a developer you’ll be able to file your messages into, for example, a com.yourname.whatevers domain space, so you can easily find your log messages. You can also build categories, and of course, as we noted previously, tag. So there are about as many ways to find log entries as you can possibly ask for. Apple has a number of subsystems built into macOS. I put together a list of Apple subsystems into a class that you should be able to throw into your python projects at https://gist.github.com/krypted/495e48a995b2c08d25dc4f67358d1983.

You also have different logging levels. These include the basic levels of Default, Info, and Debug. You also have two special levels available: Fault and Error. All of this is to add hierarchical logs (which makes tracing events a much more lovely experience) and protecting privacy of end users (think sandbox for logs). I’d recommend watching the WWDC session where Unified Logging was introduced at https://developer.apple.com/videos/play/wwdc2016/721 if you’re interested in learning more about these types of things, especially if you’ll be building software that makes use of these new logging features.

The one thing that’s worth mentioning for the Mac Techs out there, is how you would go about switching between logging levels for each subsystem. This is done with the ‘log config’ command. Here, I’ll use the –mode option to set the level to debug, and then defining the substyem to do so with using the –subsystem option:

log config --mode "level:debug" --subsystem com.krypted

If you have a particularly dastardly app, the above might just help you troubleshoot a bit. As mentioned earlier, we also have these predicates, which you can think of as metadata in the searching context. These include the following:

  • category: category of a log entry
  • eventMessage: searches the activity or message
  • eventType: type of events that created the entry (e.g. logEvent, traceEvent)
  • messageType – type or level of a log entry
  • processImagePath: name of the process that logged the event
  • senderImagePath: not all entries are created by processes, so this also includes libraries and executables
  • subsystem: The name of the subsystem that logged an event

Comparisons and Searches

OK, now let’s make things just a tad bit more complicated. We’ll do this by stringing together search parameters. Here, we have a number of operators available to us, similar to what you see in SQL. These include:

  • && or AND to indicate two matches
  • || or OR indicates one of the patterns matches
  • ! or NOT searches for items that the patterns don’t match for, which is useful for filtering out false positives in scripts
  • = to indicate that one search matches a pattern or is equal to
  • != to indicate that the search is not equal to
  • > is greater than
  • < is less than
  • => means greater than or equal to
  • =< means less than or equal to
  • CONTAINS indicates a string matches a given pattern with case sensitivity
  • CONTAINS[c] indicates a string matches a given pattern without case sensitivity
  • BEGINSWITH indicates a string begins with a given pattern
  • ENDSWITH indicates that a string ends with a given pattern
  • LIKE indicates a pattern is similar to what you’re searching for
  • MATCHES indicates that two text strings match
  • ANY, SOME, NONE, IN are used for pattern matching in arrays
  • NULL indicates a NULL response (for example, you see “with error (NULL)” in logs a lot)

To put these into context, let’s use one in an example. Thus far my most common as been a compound search, so matching both patterns. Here, we’ll look at the WirelessProximity subsystem for Bluetooth and we’ll look at how often it’s scanning for new devices, keeping both patterns to match inside their own parenthesis, with all patterns stored inside single quotes, as follows:

log show --style syslog --predicate '(subsystem == "com.apple.bluetooth.WirelessProximity") && (eventMessage CONTAINS[c] "scanning")' --info --last 1h

Developers and systems administrators will find that the Apple guide on predicate programming, available at https://developer.apple.com/library/prerelease/content/documentation/Cocoa/Conceptual/Predicates/AdditionalChapters/Introduction.html, to be pretty useful if you’re doing lots of this kind of work.

Note: sysdiagnose, a tool long used for capture diagnostics information to include in bug reports, is still functional, and now includes Unified Logging information, so Apple developers can get a complete picture of what’s going on in systems.

Conclusion

Ultimately, the new Unified Logging is a bit more complicated than the previous options for both creating and reading logs. But once you get used to it, you’ll log it – I mean, love it. I find that I use less grep and awk and get more concise results. I also like the fact that the same code is useable with all four platforms, so learn once and re-use across devices. There’s a lot of information out there, but I had to go hunting around. Hopefully having a number of links and a the structure used in this article makes it easier to learn how to use all these new new little toys! Good luck!

July 26th, 2017

Posted In: Mac OS X, Mac OS X Server, Mac Security

Tags: , , , , ,

Some apps have defaults domains that don’t work the same as other apps and you need to use the -app option in defaults. This option is available for most apps, and sometimes I’ll use it to specifically crawl around for a specific setting I’m looking for. But for other apps, you need to interact with them there. So let’s look at Eclipse. Here, we can do a read with -app followed by the path:

defaults read -app /Applications/eclipse/Eclipse.app/

The output would be as follows:

{
NSNavLastRootDirectory = “~/smb/smb”;
NSNavPanelExpandedSizeForOpenMode = “{712, 426}”;
NSScrollAnimationEnabled = 0;
WebKitJavaEnabled = 0;
}

Now, let’s say you had a specific setting, like fixing an anti-aliasing issue:

defaults write -app /Applications/eclipse/Eclipse.app AppleAntiAliasingThreshold 19

#thanksaloteclipseupdaters

June 4th, 2017

Posted In: Java, Mac OS X, Mac OS X Server, Mac Security

Tags: ,

June 2nd, 2017

Posted In: Mac OS X, Mac OS X Server, Mac Security, MacAdmins Podcast

Tags: , , , , ,

From Anthony Reimer:

The MacDeployment Conference in Calgary, Alberta, Canada has extended the “early bird” registration rate through this Thursday. Only CAD$179+GST gets you a 2-day conference with a diverse speaker group from across North America. (One day rate also available.) For more information and to register, go to http://macdeployment.ca

May 25th, 2017

Posted In: Mac OS X, public speaking

Tags: ,

May 24th, 2017

Posted In: Mac OS X

Over the users I’ve written a good bit about pushing a workload off to a virtual machine sitting in a data center somewhere. The Google CloudPlatform has matured a lot and I haven’t really gotten around to writing about it. So… It’s worth going into their SDK and what it looks like from a shell using some quick examples.

For starters, you’ll need an account with Google Cloud Platform, at cloud.google.com and you’ll want to go ahead and login to the interface, which is pretty self-explanatory (although at first you might have to hunt a little for some of the more finely grained features, like zoning virtual instances.

The SDK

The SDK will include the gcloud command, which you’ll use to perform most tasks in the Google CloudPlatform. To install the SDK, go to https://cloud.google.com/sdk/downloads and download the appropriate version for your computer. If you’re on a mac, most likely the x86_64 version.

Next, move the downloaded folder to a permanent location and run the install.sh inside it, which will kindly offer to add gcloud to your path.

./install.sh

Welcome to the Google Cloud SDK!
To help improve the quality of this product, we collect anonymized usage data
and anonymized stacktraces when crashes are encountered; additional information
is available at <https://cloud.google.com/sdk/usage-statistics>. You may choose
to opt out of this collection now (by choosing ‘N’ at the below prompt), or at
any time in the future by running the following command:
gcloud config set disable_usage_reporting true
Do you want to help improve the Google Cloud SDK (Y/n)?  y
Modify profile to update your $PATH and enable shell command
completion?
Do you want to continue (Y/n)?  y
The Google Cloud SDK installer will now prompt you to update an rc
file to bring the Google Cloud CLIs into your environment.
Enter a path to an rc file to update, or leave blank to use
[/Users/charlesedge/.bash_profile]:
Backing up [/Users/charlesedge/.bash_profile] to [/Users/charlesedge/.bash_profile.backup].
[/Users/charlesedge/.bash_profile] has been updated.
==> Start a new shell for the changes to take effect.
For more information on how to get started, please visit:
https://cloud.google.com/sdk/docs/quickstarts

Inside that bin folder, you’ll find the gcloud python script, which once installed, you can then run. Next, you’ll need to run the init, which links it to your CloudPlatform account via oauth. To do so, run gcloud with the init verb, which will step you through the process:

gcloud init

Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init –skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection…done.
Reachability Check passed.
Network diagnostic (1/1 checks) passed.

You must log in to continue. Would you like to log in (Y/n)? y

If you say yes in the above screen, your browser will then prompt you with a standard Google oauth screen where you’ll need to click Allow.

Now go back to Terminal and pick a “Project” (when you set up billing the default was created for you):

Pick cloud project to use:
[1] seventh-capsule-138123
[2] Create a new project
Please enter numeric choice or text value (must exactly match list
item):
1

The Command Line

Next, we’re gonna’ create a VM. There are several tables that lay out machine types. Let’s start by listing any instances we might have:

gcloud compute instances list

Listed 0 items.

Note: If you have a lot of these you can use  --regexp to filter them quickly.

Then let’s pick a machine type. A description of machine types can be found at https://cloud.google.com/compute/docs/machine-types. And an image. Images can be seen using the compute command with images and then list, as follows:

gcloud compute images list

Now, let’s use that table from earlier and make a custom machine using an ubuntu uri, a –custom-cpu and a –custom-memory:

gcloud compute instances create krypted1 –image https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1610-yakkety-v20170502 –custom-cpu 2 –custom-memory 5

You’ll then see that your VM is up, running, and… has an IP:

Created [https://www.googleapis.com/compute/v1/projects/seventh-capsule-138523/zones/us-central1-a/instances/krypted1].
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
krypted1 us-central1-a custom (2 vCPU, 5.00 GiB) 10.128.0.2 104.154.169.65 RUNNING

Now let’s SSH in:

gcloud compute ssh krypted1

This creates ssh keys, adds you to the hosts and SSH’s you into a machine. So viola. You’re done. Oh wait, you don’t want to leave it running forever. After all, you’re paying by the minute… So let’s list your instances:

gcloud compute instances list

Then let’s stop the one we just created:

gcloud compute instances stop krypted1

And if you’d like, tear it down:

gcloud compute instances delete krypted1

Overall, super logical, very easy to use, and lovely command line environment. Fast, highly configurable VMs. Fun times!

May 18th, 2017

Posted In: cloud, Mac OS X, Ubuntu, Unix

Tags: , , , , , , ,

Builtin commands are always kinda’ interesting. At first glance, it’s hard to know which commands are builtins. Luckily, there’s a command that I rarely use, called… command. If you run command with the -V flag it will tell you if the command is a builtin:

command -V cd

cd is a shell builtin

If you run a command that isn’t a builtin

command -V ls

ls is /bin/ls

Some builtins are in /bin (like echo). But not all builtins are in /bin. Some are in /usr/bin (like cd). Information about how to use builtins is built into the help command rather than standalone man pages. So, if you do help followed by the name of a command, you’ll get information about the command, and sometimes how to use the command:

help cd

cd: cd [-L|-P] [dir]
Change the current directory to DIR. The variable $HOME is the
default DIR. The variable CDPATH defines the search path for
the directory containing DIR. Alternative directory names in CDPATH
are separated by a colon (:). A null directory name is the same as
the current directory, i.e. `.’. If DIR begins with a slash (/),
then CDPATH is not used. If the directory is not found, and the
shell option `cdable_vars’ is set, then try the word as a variable
name. If that variable has a value, then cd to the value of that
variable. The -P option says to use the physical directory structure
instead of following symbolic links; the -L option forces symbolic links
to be followed.

There are also commands not in a path, which can be found using the which command:

which dsconfigad

/usr/sbin/dsconfigad

May 6th, 2017

Posted In: bash, Mac OS X

Tags: , , , ,

If you’re in need of MDM in Japanese or German, Jamf Now shipped support for those languages last week. To switch languages, click on your name once logged in, and then click on the language you would like to use.

Enjoy.

May 1st, 2017

Posted In: iPhone, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment

Tags: , , ,

I mentioned mdmclient when I gave the talk on the inner workings of Mobile Device Management, or MDM. There, I spent a lot of time on APNs and profiles, but just kinda’ spoke about mdmclient in terms of it being the agent that runs on macOS to provide mdm parity for the Mac. The mdmclient binary is located at /usr/libexec/mdmclient and provides pretty limited access to see how the Mac reacts to and interprets information coming from a device management provider.

I had been meaning to do a write-up on mdmclient and document what it can do since it first shipped. But as luck would have it, @Mosen on the Slacks beat me to the punch with a fantastic resource at https://mosen.github.io/profiledocs/troubleshooting/mdmclient.html. So here I’d like to focus on just 3 examples of using mdmclient. The first is to see what insight an MDM has to the applications installed (whether that information is actually committed to a database somewhere or not) using QueryInstalledApps:

/usr/libexec/mdmclient QueryInstalledApps

Here, we can see an array output of each bundle installed:

{BundleSize = 27457223;
Identifier = “com.hipchat.HipChat”;
Name = HipChat;
ShortVersion = “3.1.6”;
Version = “3.1.6”;}

Now, we can end up with duplicates, and so focus on just the unique Identifier keys, as follows:

/usr/libexec/mdmclient QueryInstalledApps | grep Identifier | uniq

The second iteration is to see installed profiles. The most basic of these, is to see user profiles, which can be obtained using QueryInstalledProfiles, as follows:

/usr/libexec/mdmclient QueryInstalledProfiles

Now, I could see using the profiles command with the -L option that I have a profile to configure office365 on my machine:

profiles -L

charlesedge[1] attribute: profileIdentifier: com.jamfsw.office365.a5f0e328-ea86-11e3-a26c-6476bab5f328
There are 1 user configuration profiles installed for ‘charlesedge’

So to see what that same information looks like, when queried from an MDM solution:
/usr/libexec/mdmclient QueryInstalledProfiles

QueryInstalledProfiles then returns:

({HasRemovalPasscode = 0;
IsEncrypted = 0;
PayloadContent = (
{PayloadDisplayName = “Charles Edge’s Office 365”;
PayloadIdentifier = “com.jamfsw.office365.a5f0e328-ea86-11e3-a26c-6476bab5f328.exchange.a5f2ccd9-ea86-11e3-b1e0-6476bab5f328”;
PayloadType = “com.apple.ews.account”;
PayloadUUID = “a5f2ccd9-ea86-11e3-b1e0-6476bab5f328”;
PayloadVersion = 1;});
PayloadDescription = “This will configure your Office 365 account for your Mac.”;
PayloadDisplayName = “Charles Edge’s Office 365”;
PayloadIdentifier = “com.jamfsw.office365.a5f0e328-ea86-11e3-a26c-6476bab5f328”;
PayloadOrganization = “JAMF Software”;
PayloadRemovalDisallowed = 0;
PayloadUUID = “a5f0e328-ea86-11e3-a26c-6476bab5f328”;
PayloadVersion = 1;
SignerCertificates = ();})

You can then take action based on this type of information, allow you to either fill a database for agent-based management, or simply take action if something is missing, etc.

QueryInstalledProfiles covers user profiles. To see system, you’ll need installedProfiles:

/usr/libexec/mdmclient installedProfiles | grep "Profile Name"

Run without the grep for a considerably more verbose amount of information.

Finally, let’s look at one more piece of information, which is the hash for the iTunes Store. That’s a point I’ve made a number of times, that the iTunes account email address is never provided to an MDM, once associated to a device or user on a device. Instead, there’s a hash of the account. These are important with VPP, as it allows for reversing (according to the MDM) which users have claimed which apps, or which users are using a given app, as well as how many devices they’re accessing those from. To see a hash, as an MDM sees it:

/usr/libexec/mdmclient QueryAppInstallation | grep iTunesStoreAccountHash

There’s a lot more you can do here, and I’m sure we’ll see a lot more over time. However, the work from @mosen combined with the opening up of the documentation on profiles and the mdm protocol helps to shed some light on how things work under the hood, and how we can use these features to provide greater programatic management for the Mac.

For example, to grab that iTuneshash from earlier, as a Jamf extension attribute you could use the following: https://github.com/krypted/ituneshash/blob/master/ituneshash.sh

April 28th, 2017

Posted In: Mac OS X, Mac OS X Server, Mac Security

Tags: , , , , , , ,

Here’s a new extension attribute at https://github.com/krypted/ituneshash/blob/master/ituneshash.sh for grabbing the hash ID used for iTunes Store accounts, useful with VPP:

#!/bin/sh
#
#
#
#Jamf Pro Extension Attribute to return the App Store Account Hash for iTunes
#Note that the return is null if one is not found
#
#
result=`/usr/libexec/mdmclient QueryAppInstallation | grep iTunesStoreAccountHash | sed '/.*\"\(.*\)\".*/ s//\1/g'`
echo "<result>$result</result>"

The output is something like:

<result>oBSmAAAa0nUAAACBHe5AaALlNBg=</result>

Which would bring the string into Jamf Pro

April 26th, 2017

Posted In: JAMF, Mac OS X, Mac OS X Server, Mac Security

Tags: , , , ,

Next Page »