Installing the Mountain Lion Server VPN Server

OS X Server has long had a VPN service that can be run. The server is capable of running the two most commonly used VPN protocols: PPTP and L2TP. The L2TP protocol is always in use, but the server can run both concurrently. You should use L2TP when at all possible.

Sure, “All the great themes have been used up and turned into theme parks.” But security is a theme that it never hurts to keep in the forefront of your mind. If you were thinking of exposing the other services in Mountain Lion Server to the Internet without having users connect to a VPN service then you should think again, because the VPN service is simple to setup and even simpler to manage.

Setting Up The VPN Service In Mountain Lion

To setup the VPN service, open the Server app and click on VPN in the Server app sidebar. The VPN Settings  screen has two options available in the “Configure VPN for” field, which has two options:

  • L2TP: Enables only the L2TP protocol
  • L2TP and PPTP: Enables both the L2TP protocol and the PPTP protocol

The VPN Host Name field is used by administrators leveraging profiles. The setting used becomes the address for the VPN service in the Everyone profile. L2TP requires a shared secret or an SSL certificate. In this example, we’ll configure a shared secret by providing a password in the Shared Secret field. Additionally, there are three fields, each with an Edit button that allows for configuration:

  • Client Addresses: The dynamic pool of addresses provided when clients connect to the VPN 
  • DNS Settings: The name servers used once a VPN client has connected to the server. As well as the Search Domains configuration.
  • Routes: Select which interface (VPN or default interface of the client system) that a client connects to each IP address and subnet mask over.
  • Save Configuration Profile: Use this button to export configuration profiles to a file, which can then be distributed to client systems (OS X using the profiles command, iOS using Apple Configurator or both using Profile Manager).

Once configured, open incoming ports on the router/firewall. PPTP runs over port 1723. L2TP is a bit more complicated (with keys bigger than a baby’s arm), running over 1701, but also the IP-ESP protocol (IP Protocol 50). Both are configured automatically when using Apple AirPorts as gateway devices. Officially, the ports to forward are listed at http://support.apple.com/kb/TS1629.

Using The Command Line

I know, I’ve described ways to manage these services from the command line before. But, “tonight we have number twelve of one hundred things to do with your body when you’re all alone.” The serveradmin command can be used to manage the service as well as the Server app. The serveradmin command can start the service, using the default settings, with no further configuration being required:

sudo serveradmin start vpn

And to stop the service:

sudo serveradmin stop vpn

And to list the available options:

sudo serveradmin settings vpn

To disable L2TP, set vpn:Servers:com.apple.ppp.l2tp:enabled to no:

sudo serveradmin settings vpn:Servers:com.apple.ppp.l2tp:enabled = no

To configure how long a client can be idle prior to being disconnected:

sudo serveradmin settings vpn:Servers:com.apple.ppp.l2tp:PPP:DisconnectOnIdle = 10

By default, each protocol has a maximum of 128 sessions, configureable using vpn:Servers:com.apple.ppp.pptp:Server:MaximumSessions:

sudo serveradmin settings vpn:Servers:com.apple.ppp.pptp:Server:MaximumSessions = 200

To see the state of the service, the pid, the time the service was configured, the path to the log files, the number of clients and other information, use the fullstatus option:

sudo serveradmin fullstatus vpn

Which returns output similar to the following:

vpn:servicePortsAreRestricted = "NO"
vpn:readWriteSettingsVersion = 1
vpn:servers:com.apple.ppp.pptp:AuthenticationProtocol = "MSCHAP2"
vpn:servers:com.apple.ppp.pptp:CurrentConnections = 0
vpn:servers:com.apple.ppp.pptp:enabled = yes
vpn:servers:com.apple.ppp.pptp:MPPEKeySize = "MPPEKeySize128"
vpn:servers:com.apple.ppp.pptp:startedTime = "2012-07-31 02:05:38 +0000"
vpn:servers:com.apple.ppp.pptp:Type = "PPP"
vpn:servers:com.apple.ppp.pptp:SubType = "PPTP"
vpn:servers:com.apple.ppp.pptp:AuthenticatorPlugins = "DSAuth"
vpn:servers:com.apple.ppp.pptp:pid = 97849
vpn:servers:com.apple.ppp.l2tp:AuthenticationProtocol = "MSCHAP2"
vpn:servers:com.apple.ppp.l2tp:CurrentConnections = 0
vpn:servers:com.apple.ppp.l2tp:enabled = yes
vpn:servers:com.apple.ppp.l2tp:startedTime = "2012-07-31 02:05:39 +0000"
vpn:servers:com.apple.ppp.l2tp:Type = "PPP"
vpn:servers:com.apple.ppp.l2tp:SubType = "L2TP"
vpn:servers:com.apple.ppp.l2tp:AuthenticatorPlugins = "DSAuth"
vpn:servers:com.apple.ppp.l2tp:pid = 97852
vpn:servicePortsRestrictionInfo = _empty_array
vpn:health = _empty_dictionary
vpn:logPaths:vpnLog = "/var/log/ppp/vpnd.log"
vpn:configured = yes
vpn:state = "RUNNING"
vpn:setStateVersion = 1

Security folk will be stoked to see that the shared secret is shown in the clear using:

vpn:Servers:com.apple.ppp.l2tp:L2TP:IPSecSharedSecretValue = "a dirty thought in a nice clean mind"

Configuring Users For VPN Access

Each account that accesses the VPN server needs a valid account to do so. To configure existing users to use the service, click on Users in the Server app sidebar.

At the list of users, click on a user and then click on the cog wheel icon, selecting Edit Access to Services.

At the Service Access screen will be a list of services that could be hosted on the server; verify the checkbox for VPN is highlighted for the user.

Setting Up Client Computers

As you can see, configuring the VPN service in Mountain Lion Server is a simple and straight-forward process – much easier than eating your cereal with a fork and doing your homework in the dark.. Configuring clients is as simple as importing the profile generated by the service. However, you can also configure clients manually. To do so in OS X, open the Network System Preference pane. From here, click on the plus sign (“+”) to add a new network service.

At the prompt, select VPN in the Interface field and then either PPTP or L2TP over IPSec in the VPN Type. Then provide a name for the connection in the Service Name field and click on Create.

At the list of network interfaces in the Network System Preference pane, provide the hostname or address of the server in the Server Address field and the username that will be connecting to the VPN service in the Account Name field. If using L2TP, click on Authentication Settings.

At the prompt, provide the password entered into the Shared Secret field earlier in this article in the Machine Authentication Shared Secret field and the user’s password in the User Authentication Password field. When you’re done, click OK and then provided you’re outside the network and routeable to the server, click on Connect to test the connection.

Conclusion

Setting Up the VPN service in OS X Mountain Lion Server is as simple as clicking the ON button. But much more information about using a VPN can be required. The natd binary is still built into Mountain Lion at /usr/sbin/natd and can be managed in a number of ways. But it’s likely that the days of using an OS X Server as a gateway device are over, if they ever started. Sure “feeling screwed up at a screwed up time in a screwed up place does not necessarily make you screwed up” but using an OS X Server for NAT when it isn’t even supported any more probably does. So rather than try to use the server as both, use a 3rd party firewall like most everyone else and then use the server as a VPN appliance. Hopefully it can do much more than just that to help justify the cost. And if you’re using an Apple AirPort as a router (hopefully in a very small environment) then the whole process of setting this thing up should be super-simple.

  • http://www.macamerica.com CVR

    is there a way to implement service acl’s for vpn service as we could do with snow leopard?

  • B

    > L2TP requires a shared secret or an SSL certificate.

    Charles, I have set up Mountain Lion Server on a Mac Pro tower sitting behind a Linksys router on a private LAN (192.168.x.y) which router gets a dynamic IP from the ISP. When I set up 10.8 server on this machine, I selected the middle option on the Accessing your Server (Choose how users will access your server”) pane of the installer -> “Local Network and using VPN” … “Access your server on the local network using a host name ending in ‘.private’. Users can also access your server using a Virtual Private Network (VPN).”

    Hence, my new 10.8 server has a host name like “my.business.private” and DNS looks fine for this basic configuration (Ethernet 1 has a locally static IP i.e. 192.168.x.2). I am also using Dyn DNS (I have the Dyn DNS updater running on my Linksys router rather than on the Mac), and have signed up for Dyn’s standard domain hosting for a single domain. As such, Dyn has a SOA record for my domain akin to “my.biz.com”, and I’ve configured Dyn to create an A record akin to “zz.my.biz.com”, which A rec of course os updated when / if my ISP changes it (related note: Comcast recently has a new feature whereby you can call them and use their automated phone tree system to request your cable modem to be refreshed with a new dynamic IP if need be).

    The Mountain Lion Server VPN service, which I have not yet turned on, defaults to a VPN Host Name of “my.business.private” described in smaller text as: “Clients configured using profiles will access the VPN service from the Internet using this hostname or IP address”. What is curious is that I do not see an option in VPN service settings to enable the use of certificates (SSL) for auth. I’d rather not use pre-shared keys or passwords.

    Is it possible to use SSL with Mountain Lion Server’s VPN service in such a situation that I’ve described above with Dyn DNS? Do I need to create another SSL cert for the Dyn DNS name (akin to “zz.my.biz.com”) even though Dyn maintains its public A record (and I port forward to the Mountain Lion server)? Reading the VPN section in Chapter 9 of your Lion Server book, if I understand correctly, to achieve the above, I may need to create another DNS record in Mountain Lion Server for “zz.my.biz.com” with an A record that points to the private IP address of the Linksys router (i.e., the gateway for the Mountain Lion server) such as 192.168.x.1? Or do I need to extend my domain, for example create a new DNS record on Mountain Lion server like “123.zz.my.biz.com” and give it an A record that points to the Linksys router’s gateway address like 192.168.x.1?

    What would be totally cool is to have only SSL-based VPN auth (so no need to hand out passwords or pre-shared keys) for highly portable client machines like iPads, which could work both when connected to the local network (192.168.x.y) via Wi-Fi, and then have the ability for that same iPad with a person who could walk out of the range of the local network Wi-Fi and either hop on to another Wi-Fi or use its 3G or 4G LTE modem and easily connect right back in via VPN to the the same Mountain Lion server (kind of like VPN “roaming” if you will). I’m not sure how automated such “VPN roaming” could be on an iPad or iPhone but its one of those progressive things that I could imagine Apple pushing the envelope on in order to improve the quality of world while others fumble around (related note: the reality of the world is that its heterogeneous so I hope Mountain Lion Server’s VPN server plays nicely with non-Apple VPN clients like Windows and Android, as it should since its using IPsec, L2TP, IKE etc. but its also known that Apple’s VPN client and server implmentation is a proprietary modification of racoon — so for those inclined, strongSwan is looking like it could be an up and coming open source alternative candidate).

    Thank you for any suggestions and clarifications!

    • http://www.krypted.com Charles

      It would be awesome if we could replace the passphrase w/ a cert, but I don’t see that ever being added to the service. I’ve not seen a command line way to switch to that without third party software, and I don’t see any options for doing so in vpnd. I’ve always figured that’s one of those kinds of things that I’d just have to get a Cisco or another third party tool for. I don’t mind the VPN service in smaller installations, but I prefer to use an appliance in a larger deployment. Having said that, a different, more feature compiled rev of racoon would be great, too. Anyway, I totally see where you’re going with this and all I can say is I wish…

      Now for the other cert thing, you can absolutely use DynDNS or something like that. The A record should match the cert but it’s not required. That would be more for services. For what it’s worth, if you enable the default SSL website service site then you can install the cert by browsing to that and then once installed use it to access the services. But it seems your goal is to use it as a pre-shared key, which I don’t see happening… We can keep hoping (and filing it as a feature request) though!

  • Ron Braithwaite

    So I’ve got a question for the assembled masses:

    When I set up a new Mountain Lion Server, I chose not to manage the Airport Extreme (in retrospect, I shouldn’t have done that). Now that I am adding the VPN service and a few other things, I would really like to have 10.8 Server manage the Airport.

    So HOW do I add an Airport Extreme to a 10.8 Server?

    Thanks,
    -Ron

    • http://www.krypted.com Charles

      From the Manage menu, Close the connection. Then Open a new connection to the server and you should be prompted again. Hope that helps. :)