krypted.com

Tiny Deathstars of Foulness

The Caching Server in OS X Server 5.2 (for Sierra) does content, apps, and software updates. The Software Update service is hidden by default indicating it will likely be removed from the Server app in a future update, although when is kinda’ up in the air. The Software Update service can still be enabled for now, which we’ll look at later. The Caching service on the Server app works like a proxy. When 10 of your users download that latest Nicholas Sparks book and movie, you only sacrifice your WAN pipe to download it once, and the other 9 people piggy-back off that. And when 10.12.1 ships, you only need to download it over the WAN once, and the other local users will pull off that spiffy Caching server sitting in your office. Pretty sweet, right?

So, how do you use this ultra-complicated service? It looks and feels kinda’ like an iPad app. Which is to say that as far as server stuffs go, this thing is pretty darn easy to use. To get started, open the Server app and then click on the Caching service in the sidebar of the Server app.

screen-shot-2016-09-22-at-12-54-08-pm

Here, click on the ON button. OMG, so hard. But wait, there’s more! Click on that Change Location button and you can select a larger volume for your updates that are cached. You’ll likely wanna’ do this because the entire series of the HBO drama OZ is kinda’ big (and yes, creepy, but really well written)…

If you do change the location, you’ll see a window to change the volume you’re caching to. That’s pretty much it. Other than the waiting for the updates to move. By default, the Caching service allows for unlimited space. Use the spiffy slider to reduce the total amount of space that the service can occupy on the hard drive. This can be a good thing if it happens to be your boot volume and there are other more mission critical services hosted on that thing.

Overall, this all seems pretty straight forward. So what else might you need to know. In case you get a corrupt asset, or in case your volume fills up, there’s a Reset button, to reset the cache.

screen-shot-2016-09-25-at-3-00-54-pm

The service can be controlled from the command line as well. To start it, use the serveradmin command along with the start verb and the service name (oddly, that’s caching).

sudo /Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin start caching

To stop the service, use the stop verb along with the service name:

sudo /Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin stop caching

To see a list of settings, use the settings verb with the service name:

sudo /Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin settings caching

The settings are as follows, mostly available in the Server app:

caching:ReservedVolumeSpace = 25000000000
caching:CacheLimit = 350000000000
caching:ServerRoot = "/Library/Server"
caching:ServerGUID = "DEE63BBB-9F32-428B-B717-E3941F82E2DC"
caching:DataPath = "/Library/Server/Caching/Data"
caching:LocalSubnetsOnly = yes
caching:Port = 0

One setting you might choose to change is the reserved volume space, as this can keep you from getting the service started on smaller volumes. In the above example, the setting is 250 gigs. To change that to 100 gigs:

sudo /Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin settings caching:ReservedVolumeSpace = 10000000000

A new setting in Server 5.2 for macOS Sierra is defining other servers that can access your Caching server. This is like providing a proxy for a proxy. Basically if your devices can cache updates onto the server from other servers then the updates are caching much faster than if your server caches the updates from Apple. This is called Peering Permissions. To define Peering Permissions, click on the Edit Peering Permissions… button.

screen-shot-2016-09-25-at-3-04-08-pm

At the Caching screen, click on Only Local Subnets if you want to let the server identify which subnets are local, or Only Some Networks to define which ranges of addresses have servers that can cache content and update from your server.

screen-shot-2016-09-25-at-3-05-53-pm

Click on the plus sign to add a network and then click on “Create a new network”

screen-shot-2016-09-25-at-3-06-55-pm

At the Create A New Network screen, provide a name and then the first and last IP

screen-shot-2016-09-25-at-3-07-57-pm

Click Create and then add all of the appropriate subnets. Click OK when you’re done. Restart the service and viola, you’re finished.

September 24th, 2016

Posted In: Mac OS X Server

Tags: , , , , , , , , ,

The Caching Server in OS X is a little bit of a black box. But, it’s not all that complicated, compared to some things in the IT world. I’d previously written about command line management of the service itself here. When you enable the caching service, the server registers itself as a valid Caching Server. Nearby devices then lookup the closest update server with Apple and register with that update server using a GUID:

/Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin settings caching:ServerGUID

Then, each time the device looks for an update, it does so against http://mesu.apple.com/assets/com_apple_MobileAsset_SoftwareUpdate/com_apple_MobileAsset_SoftwareUpdate.xml using the device model. Noticed this with this line in my proxy logs:

"GET http://mesu.apple.com/assets/com_apple_MobileAsset_SoftwareUpdate/com_apple_MobileAsset_SoftwareUpdate.xml HTTP/1.1" 200 - "-" "MobileAsset/1.0"

Let’s say that the device is an iPad 2,7, then the following information is used for the update, with a URL of http://appldnld.apple.com/iOS9.3.1/031-56322-20160331-F8B29F9E-F68D-11E5-AF11-0744ED25FABD/com_apple_MobileAsset_SoftwareUpdate/1c02ea51b4d2d50b04526c4ec29780b8e02dfe76.zip, which is created using the _BaseURL string followed by the _RelativePath string:

<dict>
<key>ActualMinimumSystemPartition</key>
<integer>1965</integer>
<key>Build</key>
<string>13E6238</string>
<key>InstallationSize</key>
<string>0</string>
<key>MinimumSystemPartition</key>
<integer>2017</integer>
<key>OSVersion</key>
<string>9.3.1</string>
<key>ReleaseType</key>
<string>Beta</string>
<key>SUDocumentationID</key>
<string>iOS931GM</string>
<key>SUInstallTonightEnabled</key>
<true/>
<key>SUMultiPassEnabled</key>
<true/>
<key>SUProductSystemName</key>
<string>iOS</string>
<key>SUPublisher</key>
<string>Apple Inc.</string>
<key>SupportedDeviceModels</key>
<array>
<string>P107AP</string>
</array>
<key>SupportedDevices</key>
<array>
<string>iPad2,7</string>
</array>
<key>SystemPartitionPadding</key>
<dict>
<key>1024</key>
<integer>1280</integer>
<key>128</key>
<integer>1280</integer>
<key>16</key>
<integer>160</integer>
<key>256</key>
<integer>1280</integer>
<key>32</key>
<integer>320</integer>
<key>512</key>
<integer>1280</integer>
<key>64</key>
<integer>640</integer>
<key>768</key>
<integer>1280</integer>
<key>8</key>
<integer>80</integer>
</dict>
<key>_CompressionAlgorithm</key>
<string>zip</string>
<key>_DownloadSize</key>
<integer>1164239508</integer>
<key>_EventRecordingServiceURL</key>
<string>https://xp.apple.com/report</string>
<key>_IsZipStreamable</key>
<true/>
<key>_Measurement</key>
<data>Rfrw7jNYWH8xNS67pXoq7NEhpUI=</data>
<key>_MeasurementAlgorithm</key>
<string>SHA-1</string>
<key>_UnarchivedSize</key>
<integer>1235575808</integer>
<key>__AssetDefaultGarbageCollectionBehavior</key>
<string>NeverCollected</string>
<key>__BaseURL</key>
<string>
http://appldnld.apple.com/iOS9.3.1/031-56322-20160331-F8B29F9E-F68D-11E5-AF11-0744ED25FABD/
</string>
<key>__CanUseLocalCacheServer</key>
<true/>
<key>__QueuingServiceURL</key>
<string>https://ns.itunes.apple.com/nowserving</string>
<key>__RelativePath</key>
<string>
com_apple_MobileAsset_SoftwareUpdate/1c02ea51b4d2d50b04526c4ec29780b8e02dfe76.zip
</string>
</dict>

You can then use these dictionaries to assemble this path for all items in the dictionary with “iPad 2,7” in the SupportedDevices key. You can also choose to assemble this path for all items with the OSVersion of a given string, such as 9.3.1 in this case. You could curl these updates down to a client, or request them through the caching service, which would cache them to the Caching Server, using the IP of the server (e.g. 10.1.1.2) http://10.1.1.2:55491/iOS9.3.1/031-56322-20160331-F8B29F9E-F68D-11E5-AF11-0744ED25FABD/1c02ea51b4d2d50b04526c4ec29780b8e02dfe76.zip?source=appldnld.apple.com

Found the above URL using a reverse proxy. This URL is generated based on an http request to the IP address of the caching service, followed by the port. The port is derived using the serveradmin command and query the settings for caching:Port as follows:

/Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin settings caching:Port

In this example, the URL is then

http://10.1.1.2:55491/

But the URL then splits the _BaseURL into two parts, taking appldnld.apple.com from the URL and appending ?source=appldnld.apple.com. So without the update, the URL would be the following:

http://10.1.1.2:55491?source=appldnld.apple.com

OK, so now we’ll pop the other part of that _BaseURL in there:

http://10.1.1.2:55491/iOS9.3.1/031-56322-20160331-F8B29F9E-F68D-11E5-AF11-0744ED25FABD?source=appldnld.apple.com

And then there’s one more step, which is throw the zip in there:

http://10.1.1.2:55491/iOS9.3.1/031-56322-20160331-F8B29F9E-F68D-11E5-AF11-0744ED25FABD/1c02ea51b4d2d50b04526c4ec29780b8e02dfe76.zip?source=appldnld.apple.com

Viola. Curl that and the caching server will download that update and make it ready for clients to access. Everything is hashed and secure in the directory listed using this command:

/Applications/Server.app/Contents/ServerRoot/usr/sbin/serveradmin settings caching:DataPath

April 18th, 2016

Posted In: Apple Configurator, Mac OS X, Mac OS X Server, Mac Security

Tags: , , , , , , ,

I’ve written a couple of articles about the Caching service in OS X Server 5 for El Capitan. As of OS X Server 5, the Caching service now caches local copies on the computer running the Caching service of iCloud content. This allows you to cache content once and then have it accessed by multiple devices faster. I’m torn on this option. On the one hand, I love the fact that I can cache things and on the other hand I find it frightening that a random user can cache things I might not want them to cache on behalf of another user. I know, I know, they’re encrypted with a device key. But when you have data on disk, it can always be decrypted. I almost feel like there should be a plist on machines that whitelists allowed caching servers. Maybe I should make a feature request on that.

Either way, as it stands now, I might be disabling this option in larger offices. To do so, I can write an AllowPersonalCaching key into the Config.plist file at /Library/Server/Caching/Config/. The most graceful way to do this is using the serveradmin command, followed by the settings verb and then caching:AllowPersonalCaching option, setting that equals no, as follows:

sudo serveradmin settings caching:AllowPersonalCaching = no

To turn it back on:

sudo serveradmin settings caching:AllowPersonalCaching = yes

This can also be done by dropping a Config.plist file into the correct location for new server installations. I’ll have an article out shortly on doing so, as you’d want to normalize a few options in the file before deploying en masse (e.g. if you have a large contingent of Caching servers to manage.

October 16th, 2015

Posted In: Mac OS X Server

Tags: , , , , , , ,

The caching service in Mountain Lion Server (OS X Server 10.8) by default can use any interface installed on the system. I’ve now seen a couple instances where we have a Small Tree card and when a big update comes up, we loose file services speed due to caching data. To combat this, we can tell the Caching service to use the built-in Ethernet interface exclusively instead. To do so, first use ifconfig to determine which interface is which. Then tell the caching service which to use, using the serveradmin command, followed by settings and then the name of the setting, caching:Interface, setting the value to the en of the interface you’d like to use:

serveradmin settings caching:Interface = en1

I’ve had to restart the caching service to have this change take effect:

serveradmin stop caching
serveradmin start caching

Clients will then automatically use the correct interface.

August 22nd, 2013

Posted In: Mac OS X Server

Tags: , , , , ,

These days, new services get introduced in OS X Server during point releases. OS X now has a Software Caching server built to make updates faster. This doesn’t replace Apple’s Software Update Server mind you, it supplements. And, it’s very cool technology. “What makes it so cool” you might ask, given that Software Update Server has been around for awhile. Namely, the way that clients perform software update service location and distribution with absolutely no need (or ability) for centralized administration.

Let’s say that you have 200 users with Mac Minis and an update is released. That’s 200 of the same update those devices are going to download over your Internet connection, at up to 2 to 3 gigs per download. If you’re lucky enough to have eaten at the Varsity in Atlanta, just imagine trying to drink one of those dreamy orange goodnesses through a coffee stirrer. Probably gonna’ be a little frustrating. Suck and suck and suck and it’ll probably melt enough to make it through that straw before you can pull it through. For that matter, according to how fast your Internet pipe is, there’s a chance something smaller, like an update to Expensify will blow out that same network, leaving no room for important things, like updates to Angry Birds!

Now, let’s say you have an OS X Server running the new Caching service. In this case, the first device pulls the update down and each subsequent device uses the WAN address to determine where the nearest caching service is. If there’s one on the same subnet, provided the subnet isn’t a Class B or higher, then the client will attempt to establish a connection to the caching service. If it can and the update being requested is on that server then the client will pull the update from the server once the signature of the update is verified with Apple (after all, we wouldn’t want some funky cert getting in the way of our sucking). If the download is stopped it will resume after following the same process on a different server, or directly from Apple. The client-side configuration is automatic so provides a seamless experience to end users.

Pretty cool, eh? But you’re probably thinking this new awesomeness is hard as all heck to install. Well, notsomuch. There are a few options that can be configured, but the server is smart enough to do most of the work for you. Before you get started, you should:

  • Be running Mountain Lion with Server 2.2 or better.
  • Install an APNS certificate first, described in a previous article I wrote here.
  • Have an ethernet connection on the server.
  • Have a hard drive with at least 50GB free in the server.
  • The server must be in a Class C or smaller LAN IP scheme (no WAN IPs can be used with this service, although I was able to multihome with the WAN off while configuring the service)

Once all of the requirements have been met, you will need to install the actual Caching Service. To do so, open Server.app from the /Applications directory and connect to the server with which you would like to install the Caching service.

Click on Caching from the SERVICES section of the Server sidebar. Here, you have 3 options you can configure before starting the service. The first is which volume with which to place updates. This should typically be a Pegasus or other form of mass storage that is not your boot volume. Use the Edit… button to configure which volume will be used. By default, when you select that volume you’ll be storing the updates in the Library/Server/Caching/Data of that volume.

The next button is used to clear out the cache currently used on the server. Click Reset and the entire contents of the aforementioned Data directory will be cleared.

Next, configure the Cache Size. Here, you have a slider to configure about as much space as you’d like, up to “Unlimited”. You can also use the command line to do some otherwise unavailable numbers, such as 2TB.

Once you’ve configured the correct amount of space, click on the ON button to fire up the service. Once started, grab a client from the local environment and download an update. Then do another. Time both. Check the Data folder, see that there’s stuff in there and enjoy yourself for such a job well done.

Now, let’s look at the command line management available for this service. Using the serveradmin command you can summon the settings for the caching service, as follows:

sudo serveradmin settings caching

The settings available include the following results:

caching:ReservedVolumeSpace = 25000000000
caching:SingleMachineMode = no
caching:Port = 0
caching:SavedCacheSize = 0
caching:CacheLimit = 0
caching:DataPath = "/Volumes/Base_Image/Library/Server/Caching/Data"
caching:ServerGUID = "FB78960D-F708-43C4-A1F1-3E068368655D"
caching:ServerRoot = "/Library/Server"

Don’t change the caching:ServerRoot setting on the server. This is derived from the root of the global ServerRoot. Also, the ServerGUID setting is configured automatically when connecting to Apple and so should not be set manually. When you configured that Volume setting, you set the caching:DataPath option. You can make this some place completely off, like:

sudo serveradmin settings caching:DataPath = "/Library/Server/NewCaching/NewData"

Now let’s say you wanted to set the maximum size of the cache to 800 gigs:

sudo serveradmin settings caching:CacheLimit = 812851086070

To customize the port used:

sudo serveradmin settings caching:Port = 6900

The server reserves a certain amount of filesystem space for the caching service. This is the only service I’ve seen do this. By default, it’s about 25 gigs of space. To customize that to let’s say, ‘around’ 50 gigs:

sudo serveradmin settings caching:ReservedVolumeSpace = 50000000000

To stop the service once you’ve changed some settings:

sudo serveradmin stop caching

To start it back up:

sudo serveradmin start caching

Once you’ve started the Caching service in OS X Server and familiarized yourself with the serveradmin caching options, let’s look at the status options. I always use fullstatus:

sudo serveradmin fullstatus caching

Returns the following:

caching:Active = yes
caching:state = "RUNNING"
caching:Port = 57466
caching:CacheUsed = 24083596
caching:TotalBytesRequested = 24083596
caching:CacheLimit = 0
caching:RegistrationStatus = 1
caching:CacheFree = 360581072384
caching:StartupStatus = "OK"
caching:CacheStatus = "OK"
caching:TotalBytesReturned = 24083596
caching:CacheDetails:.pkg = 24083596

The important things here:

  • An Active setting of “yes” means the server’s started.
  • The state is “STARTED” or “STOPPED” (or STARTING if it’s in the middle).
  • The TCP/IP port used 57466 by default. If the caching:Port setting earlier is set to 0 this is the port used by default.
  • The CacheUsed is how much space of the total CacheLimit has been used.
  • The RegistrationStatus indicates whether the server is registered via APNS for the service with Apple.
  • The CacheFree setting indicates how much space on the drive can be used for updates.
  • The caching:TotalBytesRequested option should indicate how much data has been requested from clients while the caching:TotalBytesReturned indicates how much data has been returned to clients.

Look into the /Library/Server/Caching/Config/Config.plist file to see even more information, such as the following:

<key>LastConfigURL</key>
<string>http://suconfig.apple.com/resource/registration/v1/config.plist</string>
<key>LastPort</key>
<integer>57466</integer>
<key>LastRegOrFlush</key>
<date>2012-12-16T04:33:13Z</date>

There are also a number of other keys that can be added to the Config.plist file including CacheLimit, DataPath, Interface, ListenRanges, LogLevel, MaxConcurrentClients, Port and ReservedVolumeSpace. These are described further at http://support.apple.com/kb/HT5590.

As you can see, this provides the host name of the server and path on that server that the Caching server requires access to, the last port connected to and the last date that the contents were flushed.

In the Data directory that we mentioned earlier is a SQLite database, called AssetInfo.db. In this database, a number of files are mentioned. These are in a file hierarchy also in that Data directory. Client systems access data directly from that folder.

Finally, the Server app contains a log that is accessed using the Logs option in the Server app sidebar. If you have problems with the service, information can be accessed here (use the Caching Service Log to access Caching logs).

The Caching Service uses the AssetCache service, located at

/Applications/Server.app/Contents/ServerRoot/usr/libexec/AssetCache/AssetCache,

then starts as the new user _assetcache user. It’s LaunchDaemon is at

/Applications/Server.app/Contents/ServerRoot/System/Library/LaunchDaemons/com.apple.AssetCache.plist.

Note: In my initial testing it appeared that after rebooting devices, that iOS updates were being cached; however, several have reported that this is not yet possible. I’ll try and replicate and report my findings later.

December 17th, 2012

Posted In: iPhone, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Network Infrastructure

Tags: , , , , , , , , , , , , , , ,