Getting started with Messages Server couldn’t really be easier. Messages Server in the macOS Server 5.2 version of the Server app uses the open source jabber project as their back-end code base. The jabber binary is located at /Applications/Server.app/Contents/ServerRoot/private/var/jabberd directory and the autobuddy binary is at /Applications/Server.app/Contents/ServerRoot/usr/bin/jabber_autobuddy. The actual jabberd binary is also stored at /Applications/Server.app/Contents/ServerRoot/usr/libexec/jabberd, where there are a couple of perl scripts used to migrate the service between various versions as well.
Setting up the Messages service is simple. Open the Server app and click on Messages in the Server app sidebar.

Click on the Edit… button for the Permissions. Here, define which users and interfaces are allowed to use the service.
From Server app, click on the checkbox for “Enable server-to-server federation” if you have multiple iChat, er, I mean, Messages servers and provide the address for servers to federate to.
Next, click on the checkbox for “Archive all chat messages” if you’d like transcripts of all Messages sessions that route through the server to be saved on the server.
You should use an SSL certificate with the Messages service. If enabling federation so you can have multiple Messages servers, you have to. Before enabling the service, click on the name of the server in the sidebar of Server app and then click on the Settings tab. From here, click on Edit for the SSL Certificate (which should be plural btw) entry to bring up a screen to select SSL Certificates.
At the SSL Certificates screen (here it’s plural!), select the certificate the Messages service should use from the available list supplied beside that entry and click on the OK button. If you need to setup federation, click back on the Messages service in the sidebar of Server app and then click on the Edit button. Then, click on the checkbox for Require server-to-server federation (making sure each server has the other’s SSL certificate installed) and then choose whether to allow any server to federate with yours or to restrict which servers are allowed. I have always restricted unless I was specifically setting up a server I wanted to be public (like public as in everyone in the world can federate to it, including the gorram reavers that want to wear your skin).
To restrict the service, then provide a list of each server address capable of communicating with your server. Once all the servers are entered, click the OK button. Obviously, if you only have one server, you can skip that. Once the settings are as you wish them to be, click on the ON/OFF switch to light up the service. To see the status of the service, once started, use the fullstatus option with serveradmin followed by the jabber indicator:
sudo serveradmin fullstatus jabber
The output includes whether the service is running, the location of jabber log files, the name of the server as well as the time the service was started, as can be seen here:
jabber:state = "RUNNING"
jabber:roomsState = "RUNNING"
jabber:logPaths:PROXY_LOG = "/private/var/jabberd/log/proxy65.log"
jabber:logPaths:MUC_STD_LOG = "/var/log/system.log"
jabber:logPaths:JABBER_LOG = "/var/log/system.log"
jabber:proxyState = "RUNNING"
jabber:currentConnections = "0"
jabber:currentConnectionsPort1 = "0"
jabber:currentConnectionsPort2 = "0"
jabber:pluginVersion = "10.8.211"
jabber:servicePortsAreRestricted = "NO"
jabber:servicePortsRestrictionInfo = _empty_array
jabber:hostsCommaDelimitedString = "osxserver.krypted.lan"
jabber:hosts:_array_index:0 = "osxserver.krypted.lan"
jabber:setStateVersion = 1
jabber:startedTime = ""
jabber:readWriteSettingsVersion = 1
There are also a few settings not available in the Server app. One of these that can be important is the port used to communicate between the Messages client and the Messages service on the server. For example, to customize this to 8080, use serveradmin followed by settings and then jabber:jabberdClientPortSSL = 8080, as follows:
sudo serveradmin settings jabber:jabberdClientPortSSL = 8080
To change the location of the saved Messages transcripts (here, we’ll set it to /Volumes/Pegasus/Book:
sudo serveradmin settings jabber:savedChatsLocation = “/Volumes/Pegasus/Book”
To see a full listing of the options, just run settings with the jabber service:
sudo serveradmin settings jabber
The output lists each setting configurable:
jabber:dataLocation = “/Library/Server/Messages”
jabber:s2sRestrictDomains = no
jabber:jabberdDatabasePath = “/Library/Server/Messages/Data/sqlite/jabberd2.db”
jabber:sslCAFile = “/etc/certificates/osxserver.krypted.com.31971C0C39DCBF4733FA671BCE3AF260769E4FB7.chain.pem”
jabber:jabberdClientPortTLS = 5222
jabber:sslKeyFile = “/etc/certificates/osxserver.krypted.com.31971C0C39DCBF4733FA671BCE3AF260769E4FB7.concat.pem”
jabber:initialized = yes
jabber:enableXMPP = yes
jabber:savedChatsArchiveInterval = 7
jabber:authLevel = “STANDARD”
jabber:hostsCommaDelimitedString = “osxserver.krypted.com”
jabber:jabberdClientPortSSL = 5223
jabber:requireSecureS2S = yes
jabber:savedChatsLocation = “/Library/Server/Messages/Data/message_archives”
jabber:enableSavedChats = yes
jabber:enableAutoBuddy = no
jabber:s2sAllowedDomains = _empty_array
jabber:logLevel = “ALL”
jabber:hosts:_array_index:0 = “osxserver.krypted.com”
jabber:eventLogArchiveInterval = 7
jabber:jabberdS2SPort = 5269
To stop the service:
sudo serveradmin stop jabber
And to start it back up:
sudo serveradmin start jabber
It’s also worth noting something that’s completely missing in this whole thing: Apple Push Notifications… Why is that important? Well, you use the Messages application to communicate not only with Mac OS X and other jabber clients, but you can also use Messages to send text messages. Given that there’s nothing in the server that has anything to do with texts, push or anything of the sort, it’s worth noting that these messages don’t route through the server and therefore still require an iCloud account. Not a huge deal, but worth mentioning that Messages server doesn’t have the same updates built into the Messages app. Because messages don’t traverse the server, there’s no transcripts.
