ChronoSync is one of those tools that’s been in the Mac community for a long time (rightfully so). It’s been a little while since I got the chance to really tinker around with ChronoSync so I thought I’d do a little article on what I got to find during my tinkerations. To get started with ChronoSync, go to their website at http://www.econtechnologies.com/chronosync/overview.html
. Next, we’re going to walk through the most basic of setups (and you can get all kinds of complicated from there if you’d like!).
Once you’ve downloaded, ChronoSync, run the installer from the disk image that was downloaded.
Then walk through the installer, basically following the defaults (unless you’d like to install to a volume other than your boot volume).
Once the installer is finished, open the app and register the product.
Once registered, you’ll see a nice screen giving you a few options. We’re going to create a single plan (synchronizer document) to backup a single source to a single target. To do so, click on the option to “Create a new synchronizer document”.
At the Setup screen, you have a right and left column. When I used to do a lot of manual migrations, I would always always always line up my source on the left and my target on the right (or invariably you risk data loss by copying in the wrong direction), so the workflow in ChronoSync has always made sense to me. Because a lot of the data I use needs root access, I’m going to select “Local Volumes (Admin access)” in the “Connect to” field and then use the Choose button to select my actual source. Repeat that process in the Right Target section of the screen.
The default action that will be performed is to backup from the left to the right targets (the term target referring to the folder, not that it’s a source or target in the backup operation). Click into the Operation field to bring up a list of the options that can be performed between your left and right targets.
The option I’m selecting is “Synchronize Bidirectional” as this is an article about syncing data. The other options are pretty well defined in the manual, but it’s worth mentioning that the Bootable Mirror options are especially useful. Once you’ve set the type of sync, you can also use the Options menu to define some pretty granular settings for your sync. For the purposes of this sync, which brings over server shares, I’m going to leave Conflict resolution set to Ask User and use the custom option under the Special File/Folder Handling section to enable the “Verify copied data” option and “Preserve Comments” option. Note that if you’re doing this on servers and would like to stop a service (such as postgres) before a sync and start it after, you can use the scripts section of this screen. You can also configure notifications, sending emails when syncs have errors, or every time there’s a sync.
Click Rules to build inclusion/exclusion rules (for example, I don’t often sync things like operating system and software installers since I can just go download them again, pretty easily). Click Archive in the sidebar if you’d like to remove files based on a trigger (e.g. if it’s been removed from the source, archive it, etc).
Next, you can simply click Synchronize to run an immediate sync of the files and folders you’ve defined in your Sync Document. Or, you can click Add to Schedule to define when you’d like to run your Synchronization Documents.
There, less than 5 minutes and we’ve got a pretty advanced sync going. Use the Log button to see how everything went. And remember, always verify that the archives and backups are running on a good schedule. For example, I like to have at least a weekly cadence to make sure that media one each side of a sync can still open. It helps me sleep better.
krypted January 31st, 2015
Posted In: Kerio, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment
Apple, archive, backup, bidirectional sync, d2d2t, MAC, Sync, Synchronize folders in OS X
Any time doing a migration of data from one IP to another where that data has a DNS record that points users towards the data, we need to keep the amount of time it takes to repoint the record to a minimum. To see the TTL of a given record, let’s run dig using +trace, +nocmd to turn off showing the version and query options, +noall to turn off display flags, +answer to still show the answer section of my reponse and most importantly for these purposes +ttlid to toggle showing the TTL on. Here, we’ll use these to lookup the TTL for the www.krypted.com A record:
dig +trace +nocmd +noall +answer +ttlid a www.krypted.com
The output follows the CNAME (as many a www record happen to be) to the A record and shows the TTL value (3600) for each:
www.krypted.com. 3600 IN CNAME krypted.com.
krypted.com. 3600 IN A 22.214.171.124
We can also lookup the MX using the same structure, just swapping out the a for an MX and the FQDN with just the domain name itself:
dig +trace +nocmd +noall +answer +ttlid mx krypted.com
The response is a similar output where
krypted.com. 3600 IN MX 0 smtp.secureserver.net.
krypted.com. 3600 IN MX 10 mailstore1.secureserver.net.
krypted January 23rd, 2014
Posted In: Active Directory, cloud, Consulting, iPhone, Kerio, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Microsoft Exchange Server, Network Infrastructure, Windows Server
change, code, dig, DNS, Linux, MAC, migration dns, named, trace, ttlid, windows
Mail is one of the hardest services to manage. Actually, mail is pretty simple in and of itself: there’s protocols people use to access their mail (such as IMAP and POP), protocols used to communicate between mail servers and send mail (SMTP, SMTPS) and then there’s a database of mail and user information. In Mavericks Server, all of these are represented by a single ON button, so it really couldn’t be easier. But then there’s the ecoysystem and the evil spammers.
As a systems administrator of a large number of mail servers, I firmly believe that there is a special kind of hell where only spam is served at every meal for spammers. Here, the evil spammers must also read every piece of spam ever sent for eternity. By the end (aka Ragnarok), they should have the chemically induced stamina of a 16 year old with the latest Sports Illustrated Swimsuit issue, enough pills of other types to not be able to use that stamina, plenty of African princes looking to donate large sums of money if only they can be helped out of their country (which should cost about 100,000 compared to a 5,000,000 payout, not a bad ROI, right?!?!?), have their conflicting stamina situation at the top of the search engines and of course, have lost all of the money made from their African princes due to getting their credit card hijacked by about 9,000 phishing scams. All in all, a special kind of hell…
But back to the point of the article, setting up mail. The things that mail administrators need to focus on to keep that mail server flowing mail to and from everyone else in the world:
- Static IP address. The WAN (and LAN probably) address should be static.
- Port Forwards. Port forwards need to be configured on the gateway for the SMTP port at a minimum and more than likely other ports used to access mail on client devices (25, 143, etc)
- DNS records. An MX record and some kind of mail.domain.com type of record should definitely be configured for the DNS servers that are authoritative for the domain. There should also be reverse records for the address of the server, usually created by the Internet Services Provider, or ISP, that match that record.
- Check the RBLs. If you have a new IP address you’ll be putting a DNS server on, check all the major Realtime BlackLists to make sure that some evil spammer hasn’t squatted on the IP before you got to it. This is true whether you’re in a colo, hosted on an IP you own or moving into space formerly occupied by a very standup company. A lot of IP addresses are blocked, as are blocks of IPs, so before moving mail to an IP, check it.
- Mail filtration (message hygiene). OS X Server has a number of mail filters built in, including clam for viruses, the ability to leverage RBLs, block specific addresses and of course RBL checking. However, this is often not enough. Third party services such as MXLogic help to keep mail from coming into your network. You also end up with an external IP to send mail that can cache mail in the event the server is down and keep mail off your network in the event that it’s spam.
- Backup. I am firmly of the belief that I’d rather not have data than not have that data backed up…
Once all of that is taken care of (I’ll add more as I think about it) then it’s time to enable the mail service in Server app 3. Actually, first let’s setup our SSL certificates. To do so, open the Server app and click on Certificates in the SERVER section of the sidebar. Here, use the “Secure services using” drop-down list and click on Custom… for each protocol to select the appropriate certificate to be used for the service.
Click OK when they’re all configure. Now let’s enable the mail service (or outsource mail). To do so, open the Server app and click on Mail in the SERVICES list in the sidebar.
At the configuration screen is a sparse number of settings:
- Provide mail for: Configures all of the domains the mail server will listen for mail for. Each account on the server has a short name and each domain name will be available for each short name. For example, an account with a shortname of charles will be available for email addresses of firstname.lastname@example.org and email@example.com per the Domain Name listing below.
- Authentication: Click Edit for a list of sources that accounts can authenticate against (e.g. Active Directory, Open Directory, Custom, Local, etc) and in some cases the specific password algorithms used for mail.
- Push Notifications: If Push is configured previously there’s no need to use this option. Otherwise, use your institutional APNS account to configure Push Notifications.
- Relay outgoing mail through ISP: Provide a server that all mail will get routed through from the server. For example, this might be an account with your Internet Services Provider (ISP), an account on an appliance that you own (such as a Barracuda) or with an external filtering service (such as MXLogic).
- Limit mail to: Configure the total amount of mail a user can have in the mail store, in Megabytes.
- Edit Filtering Settings: Configure antivirus, spam assassin and junk mail filters. The “Enable virus filtering” checkbox enables clam. The “Enable blacklist filtering” checks the RBL (or RBLs) of your choice to check whether a given server is a “known” spammer and the “Enable junk mail filtering” option enables spam assassin on the host, configuring it to block based on a score as selected using the slider.
Once you’ve configured the settings for the Mail service, click on the ON slider to enable the service. At this point, you should be able to telnet into port 25 of the host to verify that SMTP is listening, preferably from another mail server:
telnet mail.krypted.com 25
You can also check that the mail services are running using the serveradmin command along with the fullstatus option for the mail service:
sudo serveradmin fullstatus mail
Which returns with some pretty verbose information about the service, including state, connections, running protocols and the rest of the following:
mail:startedTime = ""
mail:setStateVersion = 1
mail:state = "STOPPED"
mail:protocolsArray:_array_index:0:status = "ON"
mail:protocolsArray:_array_index:0:kind = "INCOMING"
mail:protocolsArray:_array_index:0:protocol = "IMAP"
mail:protocolsArray:_array_index:0:state = "STOPPED"
mail:protocolsArray:_array_index:0:service = "MailAccess"
mail:protocolsArray:_array_index:0:error = ""
mail:protocolsArray:_array_index:1:status = "ON"
mail:protocolsArray:_array_index:1:kind = "INCOMING"
mail:protocolsArray:_array_index:1:protocol = "POP3"
mail:protocolsArray:_array_index:1:state = "STOPPED"
mail:protocolsArray:_array_index:1:service = "MailAccess"
mail:protocolsArray:_array_index:1:error = ""
mail:protocolsArray:_array_index:2:status = "ON"
mail:protocolsArray:_array_index:2:kind = "INCOMING"
mail:protocolsArray:_array_index:2:protocol = "SMTP"
mail:protocolsArray:_array_index:2:state = "STOPPED"
mail:protocolsArray:_array_index:2:service = "MailTransferAgent"
mail:protocolsArray:_array_index:2:error = ""
mail:protocolsArray:_array_index:3:status = "ON"
mail:protocolsArray:_array_index:3:kind = "OUTGOING"
mail:protocolsArray:_array_index:3:protocol = "SMTP"
mail:protocolsArray:_array_index:3:state = "STOPPED"
mail:protocolsArray:_array_index:3:service = "MailTransferAgent"
mail:protocolsArray:_array_index:3:error = ""
mail:protocolsArray:_array_index:4:status = "OFF"
mail:protocolsArray:_array_index:4:kind = "INCOMING"
mail:protocolsArray:_array_index:4:protocol = ""
mail:protocolsArray:_array_index:4:state = "STOPPED"
mail:protocolsArray:_array_index:4:service = "ListServer"
mail:protocolsArray:_array_index:4:error = ""
mail:protocolsArray:_array_index:5:status = "ON"
mail:protocolsArray:_array_index:5:kind = "INCOMING"
mail:protocolsArray:_array_index:5:protocol = ""
mail:protocolsArray:_array_index:5:state = "STOPPED"
mail:protocolsArray:_array_index:5:service = "JunkMailFilter"
mail:protocolsArray:_array_index:5:error = ""
mail:protocolsArray:_array_index:6:status = "ON"
mail:protocolsArray:_array_index:6:kind = "INCOMING"
mail:protocolsArray:_array_index:6:protocol = ""
mail:protocolsArray:_array_index:6:state = "STOPPED"
mail:protocolsArray:_array_index:6:service = "VirusScanner"
mail:protocolsArray:_array_index:6:error = ""
mail:protocolsArray:_array_index:7:status = "ON"
mail:protocolsArray:_array_index:7:kind = "INCOMING"
mail:protocolsArray:_array_index:7:protocol = ""
mail:protocolsArray:_array_index:7:state = "STOPPED"
mail:protocolsArray:_array_index:7:service = "VirusDatabaseUpdater"
mail:protocolsArray:_array_index:7:error = ""
mail:logPaths:Server Error Log = "/Library/Logs/Mail/mail-err.log"
mail:logPaths:IMAP Log = "/Library/Logs/Mail/mail-info.log"
mail:logPaths:Server Log = "/Library/Logs/Mail/mail-info.log"
mail:logPaths:POP Log = "/Library/Logs/Mail/mail-info.log"
mail:logPaths:SMTP Log = "/var/log/mail.log"
mail:logPaths:List Server Log = "/Library/Logs/Mail/listserver.log"
mail:logPaths:Migration Log = "/Library/Logs/MailMigration.log"
mail:logPaths:Virus Log = "/Library/Logs/Mail/clamav.log"
mail:logPaths:Amavisd Log = "/Library/Logs/Mail/amavis.log"
mail:logPaths:Virus DB Log = "/Library/Logs/Mail/freshclam.log"
mail:imapStartedTime = ""
mail:postfixStartedTime = ""
mail:servicePortsRestrictionInfo = _empty_array
mail:servicePortsAreRestricted = "NO"
mail:connectionCount = 0
mail:readWriteSettingsVersion = 1
mail:serviceStatus = "DISABLED"
To stop the service:
sudo serveradmin stop mail
And to start it back up:
sudo serveradmin start mail
To configure some of the settings no longer in the GUI from previous versions, let’s look at the full list of options:
sudo serveradmin settings mail
One that is commonly changed is the subject line added to messages that are marked as spam by spam assassin. This is stored in mail:postfix:spam_subject_tag, so changing would be:
sudo serveradmin settings mail:postfix:spam_subject_tag = "***DIEEVILSPAMMERSDIE*** "
A number of admins also choose to disable greylisting, done using the mail:postfix:greylist_disable option:
sudo serveradmin settings mail:postfix:greylist_disable = no
To configure an email address for quarantined mail to go, use mail:postfix:virus_quarantine:
sudo serveradmin settings mail:postfix:virus_quarantine = "firstname.lastname@example.org"
The administrator, by default, doesn’t get an email when an email containing a file infected with a virus is sent through the server. To enable this option:
sudo serveradmin settings mail:postfix:virus_notify_admin = yes
I also find a lot of Mac environments want to accept email of pretty much any size. By default, message size limits are enabled. To disable:
sudo serveradmin settings mail:postfix:message_size_limit_enabled = yes
Or even better, just set new limit:
sudo serveradmin settings mail:postfix:message_size_limit = 10485760
And to configure the percentage of someone’s quota that kicks an alert (soft quota):
sudo serveradmin settings mail:imap:quotawarn = 75
Additionally, the following arrays are pretty helpful, which used to have GUI options:
- mail:postfix:mynetworks:_array_index:0 = “127.0.0.0/8” – Add entries to this one to add “local” clients
- mail:postfix:host_whitelist = _empty_array – Add whitelisted hosts
- mail:postfix:blacklist_from = _empty_array – Add blacklisted hosts
- mail:postfix:black_hole_domains:_array_index:0 = “zen.spamhaus.org” – Add additional RBL Servers
The client side of the mail service is straight forward enough. If you are wondering where in this article we discuss using webmail, er, that’s not installed by default any longer. But the open source project previously used, roundcube, is still available for download and easily installed (the pre-reqs are all there, already). Check out the roundcube wiki installation page
here for more info on that. Also, mail groups. I hope to have a post about that soon enough. Unless, of course, I get sidetracked with having a life. Which is arguably not very likely…
krypted October 23rd, 2013
Posted In: Kerio, Mac OS X, Mac OS X Server
blacklist, Configure Mail Service, greylist, IMAP, Kerio, Mac OS X Server, mavericks, Mavericks Server, OS X Server, POP, postfix, roundcube, server.app, smtp, smtpd, use ISP, whitelist
Sometimes it seems like sqlite just isn’t equipped for some tasks. Sometimes it seems like some developers aren’t. Sometimes it ends up being a mystery as to what is really going on behind the scenes. Like watching CNN on a television right next to Fox News at the gym. Both can’t be reality. But what is real, is that journabl.db files get corrupt in Kerio all the time. And the logs often say something about SQLITE_CORRUPT &/or “database disk image is malformed”. To correct, first stop the Kerio server, then nuke the .journal.db file. Assuming the mail store is /usr/local/kerio/mailserver/store/mail on a Mac (swap /usr/local with /opt if using Linux) then the command to do so would be:
Then start up Kerio and the .journal.db file will automatically rebuild and the errors about some malformed whatnot should be out of those logs.
krypted August 16th, 2013
Posted In: Kerio, Mac OS X, Mac OS X Server
.journal.db, delete journal, error SQLITE_CORRUPT: database disk image is malformed, journal, Kerio, MAC, Mac OS X Server, reset, SQLiteDbWriteCache.h: [Mail Path]/[Domain]/[Username]/.journal.db: runVacuum - SQLite error: code 11
Awhile back I wrote an article on managing the Adaptive Firewall built into Mountain Lion Server at http://krypted.com/mac-os-x-server/managing-lion-servers-adaptive-firewall-from-the-command-line
. It’s worth mentioning that when you use this command you’re basically editing some text files. These include the blacklist, blockedHosts and whitelist folders at possibly the shortest folder at this depth in the file system that I’ve ever had the good luck to need to use /var/db/af (okay, okay, I’m sure we’ve all made /a/b/c and that’s shorter, but this is pretty close).
You should use afctl to add and remove machines from these lists. The -w option in afctl used to add a host to a whitelist will cause the host to appear in the /var/db/af/whitelist file. The -a option used to blacklist a host will add it to the /var/db/af/blacklist file. Hosts that are flagged are dropped into /var/db/af/blockedHosts and when you remove those hosts with the -r option they are removed from that file.
I think that pretty much beats that poor afctl horse to death. Simple is good sometimes!
krypted June 4th, 2013
Posted In: Kerio, Mac OS X, Mac OS X Server, Mac Security, Mass Deployment, Network Infrastructure
afctl, OS X Server, remove ip from blacklist, whitelist ip
When you setup a Kerio server, by default there’s a feature called AutoExpunge. This feature keeps mail clients from showing a message with a strikethrough through it when a message is marked for deletion. Once items are processed the message is moved to deleted and the strikethrough message is removed from the folder it was deleted from. Many users can get confused by this, so Kerio built a feature called AutoExpunge. That AutoExpunge feature instead of striking through messages just tosses them. That causes you to be unable to undo a delete.
To disable AutoExpunge, stop Kerio Mail Server and then look for AutoExpungeOnDelete option in /usr/local/Kerio/mailserver/mailserver.cfg (I like to back that file up before making any changes). Then change the value for that from 1 to 0. Then save your changes to the file and start Kerio Connect back up. Once started, test that you can undo a delete and if so, you’re good to go!
Note: If you change settings like this when the mail server is running then it can revert the settings back as the daemon is running. If that happens to you, double-check that the service is stopped before editing the file.
krypted April 23rd, 2013
Posted In: Kerio, Mac OS X Server
autoexpunge, DELETE, IMAP, Kerio, Kerio Connect 8
Kerio has a few maximums set by default. There are also a few items that are not in the Kerio Connection Administration page. When using IMAP (and some other services), you can increase the maximum number of allowed connections to allow users to be able to connect to your servers using the variety of devices they likely now have. We’ll look at doing this with IMAP (given that each account accessed by each user is likely using at minimum 2 connections) but you can do this with many other services as well.
To increase the total number of available IMAP connections:
- Open the Kerio Connect Administration page.
- Click on the Configuration disclosure box to see Services.
- Click on Services.
- From the Services page, double-click on IMAP.
- At the IMAP box, click on the Access tab.
- Increase the field for Maximum number of concurrent connections.
- Click OK.
- Click Restart to restart the IMAP service.
Now, let’s say that all of the IMAP connections are coming from what the server sees as the same IP (somewhat common with certain types of routers). Well, there’s also a setting not exposed in the web configuration tool that limits the total number of connections available for a given IP address, so let’s go ahead and increase that as well. To do so, open the mailserver.cfg file located in /usr/local/kerio/mailserver. Here, look in the service-imap table and find the MaxConnectionsIP variable. Change that to, let’s say 300 and then save the changes and restart the IMAP service again. Now you’re done. Good luck!
krypted April 4th, 2013
Posted In: Kerio, Mac OS X, Mac OS X Server
Increase SMTP IMAP connections, Kerio, Kerio Connection, maxconnectionsIP, maximum number of users per IP address in Kerio Mail Server
Large scale mail migrations can be tricky. There is a shareware app that can be used to migrate pst files from the pst format into mbox, which can then be used with Mac OS X http://www.littlemachines.com
If the migration process needs to be automated (they all seem to at scale) then a script could be written to crawl users, finds the pst files and then convert them. Or it could be done on the client side using a self-destructing launchd item. Conversion syntax for libpst would be something like the following:
readpst -o /output/folder /server/path/user.pst
Before you can use readpst, it needs to be built via libpst on the system that will run any scripts. Download libpst from http://alioth.debian.org/frs/?group_id=30390. This can be done with curl:
curl http://alioth.debian.org/frs/download.php/2492/libpst-0.5.3.tar.gz --O libpst-0.5.3.tar.gz
Next, extract the tar:
tar -zxvf libpst-0.5.3.tar.gz
Then cd into the new directory:
Then make libpst:
And now readpst should be available to convert mailboxes. This could be run from a centralized server or distributed to clients.
krypted April 27th, 2011
Posted In: Kerio, Mac OS X, Mac OS X Server, Microsoft Exchange Server, MobileMe, Ubuntu, Unix
libpst, make, readpst, script, tar
Next Page »
Almost wrote this up again and then realized I already did once (sure it was a few years ago but luckily not much changes with some of the command line stuff). Check it out here:
If you want to see more on openssl check this one out too:
krypted October 7th, 2009
Posted In: Kerio, Mac OS X, Mac OS X Server, Mac Security
Mac OS X Server, openssl, s_client, test connectivity