Who still says “like a boss?” I guess I did. Get over it. But don’t get over spam. Especially annoying are the ones we know we accidentally signed up for. Because it’s our own darn fault. But luckily, there’s a lot more tools for dealing with bulk mail (solicited or unsolicited) these days. Most modern email clients have the ability to deal with spam. Exchange/Office 365 has clutter and junk. You can build rules on sites. You can use spam assassin on your servers. But, there’s also a nice little app called unroll.me. Once you sign up you’ll have 3 ways of dealing with each message: request removal from a list, mark as rolled up into a single daily digest, or mark as good email. Download it here. The app works a lot like something like Tinder. You swipe right to like something, left to not like something. Facebook should implement this into your timeline! If you decide to mark emails as digests, you’ll get an email once a day that looks like this: This works great for organizations that actually properly remove you from lists (which is surprisingly most). Using this swiping type of workflow, you can knock through 100 or more emails in 10-15 minutes. For organizations that don’t respect unfollow or stop sending me your crap emails, there’s also always just marking them as spam. The only problem with this is that you likely have a phone, a computer, a home computer, and maybe a tablet. No one wants to mark the same email as spam four times and then potentially have emails disappearing and not being able to figure out which computer they were marked as junk on. There are lots and lots of options for this type of thing. But given the ease of use an quick evisceration I can do on my mailbox, I rather like unfollow.me. Give it a shot. You might hate it. I don’t.
Mail is one of the hardest services to manage. Actually, mail is pretty simple in and of itself: there’s protocols people use to access their mail (such as IMAP and POP), protocols used to communicate between mail servers and send mail (SMTP, SMTPS) and then there’s a database of mail and user information. In OS X Server 5 for El Capitan and Yosemite, all of these are represented by a single ON button, so it really couldn’t be easier. But then there’s the ecoysystem and the evil spammers. As a systems administrator of a large number of mail servers, I firmly believe that there is a special kind of hell where only spam is served at every meal for spammers. Here, the evil spammers must also read every piece of spam ever sent for eternity. By the end (aka Ragnarok), they should have the chemically induced stamina of a 16 year old with the latest Sports Illustrated Swimsuit issue, enough pills of other types to not be able to use that stamina, plenty of African princes looking to donate large sums of money if only they can be helped out of their country (which should cost about 100,000 compared to a 5,000,000 payout, not a bad ROI, right?!?!?), have their conflicting stamina situation at the top of the search engines and of course, have lost all of the money made from their African princes due to getting their credit card hijacked by about 9,000 phishing scams. All in all, a special kind of hell… ￼ But back to the point of the article, setting up mail. The things that mail administrators need to focus on to keep that mail server flowing mail to and from everyone else in the world:
- Static IP address. The WAN (and LAN probably) address should be static.
- Port Forwards. Port forwards need to be configured on the gateway for the SMTP port at a minimum and more than likely other ports used to access mail on client devices (25, 143, etc)
- DNS records. An MX record and some kind of mail.domain.com type of record should definitely be configured for the DNS servers that are authoritative for the domain. There should also be reverse records for the address of the server, usually created by the Internet Services Provider, or ISP, that match that record.
- Check the RBLs. If you have a new IP address you’ll be putting a DNS server on, check all the major Realtime BlackLists to make sure that some evil spammer hasn’t squatted on the IP before you got to it. This is true whether you’re in a colo, hosted on an IP you own or moving into space formerly occupied by a very standup company. A lot of IP addresses are blocked, as are blocks of IPs, so before moving mail to an IP, check it.
- Mail filtration (message hygiene). OS X Server has a number of mail filters built in, including clam for viruses, the ability to leverage RBLs, block specific addresses and of course RBL checking. However, this is often not enough. Third party services such as MXLogic help to keep mail from coming into your network. You also end up with an external IP to send mail that can cache mail in the event the server is down and keep mail off your network in the event that it’s spam.
- Backup. I am firmly of the belief that I’d rather not have data than not have that data backed up…
- Domains: Configures all of the domains the mail server will listen for mail for. Each account on the server has a short name and each domain name will be available for each short name. For example, an account with a shortname of charles will be available for email addresses of email@example.com and firstname.lastname@example.org per the Domain Name listing below.
- Authentication: Click Edit for a list of sources that accounts can authenticate against (e.g. Active Directory, Open Directory, Custom, Local, etc) and in some cases the specific password algorithms used for mail.
- Push Notifications: If Push is configured previously there’s no need to use this option. Otherwise, use your institutional APNS account to configure Push Notifications.
- Relay outgoing mail through ISP: Provide a server that all mail will get routed through from the server. For example, this might be an account with your Internet Services Provider (ISP), an account on an appliance that you own (such as a Barracuda) or with an external filtering service (such as MXLogic).
- Limit mail to: Configure the total amount of mail a user can have in the mail store, in Megabytes.
- Edit Filtering Settings: Configure antivirus, spam assassin and junk mail filters. The “Enable virus filtering” checkbox enables clam. The “Enable blacklist filtering” checks the RBL (or RBLs) of your choice to check whether a given server is a “known” spammer and the “Enable junk mail filtering” option enables spam assassin on the host, configuring it to block based on a score as selected using the slider.
telnet mail.krypted.com 25You can also check that the mail services are running using the serveradmin command along with the fullstatus option for the mail service:
sudo serveradmin fullstatus mailWhich returns with some pretty verbose information about the service, including state, connections, running protocols and the rest of the following:
mail:startedTime = "" mail:setStateVersion = 1 mail:state = "STOPPED" mail:protocolsArray:_array_index:0:status = "ON" mail:protocolsArray:_array_index:0:kind = "INCOMING" mail:protocolsArray:_array_index:0:protocol = "IMAP" mail:protocolsArray:_array_index:0:state = "STOPPED" mail:protocolsArray:_array_index:0:service = "MailAccess" mail:protocolsArray:_array_index:0:error = "" mail:protocolsArray:_array_index:1:status = "ON" mail:protocolsArray:_array_index:1:kind = "INCOMING" mail:protocolsArray:_array_index:1:protocol = "POP3" mail:protocolsArray:_array_index:1:state = "STOPPED" mail:protocolsArray:_array_index:1:service = "MailAccess" mail:protocolsArray:_array_index:1:error = "" mail:protocolsArray:_array_index:2:status = "ON" mail:protocolsArray:_array_index:2:kind = "INCOMING" mail:protocolsArray:_array_index:2:protocol = "SMTP" mail:protocolsArray:_array_index:2:state = "STOPPED" mail:protocolsArray:_array_index:2:service = "MailTransferAgent" mail:protocolsArray:_array_index:2:error = "" mail:protocolsArray:_array_index:3:status = "ON" mail:protocolsArray:_array_index:3:kind = "OUTGOING" mail:protocolsArray:_array_index:3:protocol = "SMTP" mail:protocolsArray:_array_index:3:state = "STOPPED" mail:protocolsArray:_array_index:3:service = "MailTransferAgent" mail:protocolsArray:_array_index:3:error = "" mail:protocolsArray:_array_index:4:status = "OFF" mail:protocolsArray:_array_index:4:kind = "INCOMING" mail:protocolsArray:_array_index:4:protocol = "" mail:protocolsArray:_array_index:4:state = "STOPPED" mail:protocolsArray:_array_index:4:service = "ListServer" mail:protocolsArray:_array_index:4:error = "" mail:protocolsArray:_array_index:5:status = "ON" mail:protocolsArray:_array_index:5:kind = "INCOMING" mail:protocolsArray:_array_index:5:protocol = "" mail:protocolsArray:_array_index:5:state = "STOPPED" mail:protocolsArray:_array_index:5:service = "JunkMailFilter" mail:protocolsArray:_array_index:5:error = "" mail:protocolsArray:_array_index:6:status = "ON" mail:protocolsArray:_array_index:6:kind = "INCOMING" mail:protocolsArray:_array_index:6:protocol = "" mail:protocolsArray:_array_index:6:state = "STOPPED" mail:protocolsArray:_array_index:6:service = "VirusScanner" mail:protocolsArray:_array_index:6:error = "" mail:protocolsArray:_array_index:7:status = "ON" mail:protocolsArray:_array_index:7:kind = "INCOMING" mail:protocolsArray:_array_index:7:protocol = "" mail:protocolsArray:_array_index:7:state = "STOPPED" mail:protocolsArray:_array_index:7:service = "VirusDatabaseUpdater" mail:protocolsArray:_array_index:7:error = "" mail:logPaths:Server Error Log = "/Library/Logs/Mail/mail-err.log" mail:logPaths:IMAP Log = "/Library/Logs/Mail/mail-info.log" mail:logPaths:Server Log = "/Library/Logs/Mail/mail-info.log" mail:logPaths:POP Log = "/Library/Logs/Mail/mail-info.log" mail:logPaths:SMTP Log = "/var/log/mail.log" mail:logPaths:List Server Log = "/Library/Logs/Mail/listserver.log" mail:logPaths:Migration Log = "/Library/Logs/MailMigration.log" mail:logPaths:Virus Log = "/Library/Logs/Mail/clamav.log" mail:logPaths:Amavisd Log = "/Library/Logs/Mail/amavis.log" mail:logPaths:Virus DB Log = "/Library/Logs/Mail/freshclam.log" mail:imapStartedTime = "" mail:postfixStartedTime = "" mail:servicePortsRestrictionInfo = _empty_array mail:servicePortsAreRestricted = "NO" mail:connectionCount = 0 mail:readWriteSettingsVersion = 1 mail:serviceStatus = "DISABLED"To stop the service:
sudo serveradmin stop mailAnd to start it back up:
sudo serveradmin start mailTo configure some of the settings no longer in the GUI from previous versions, let’s look at the full list of options:
sudo serveradmin settings mailOne that is commonly changed is the subject line added to messages that are marked as spam by spam assassin. This is stored in mail:postfix:spam_subject_tag, so changing would be:
sudo serveradmin settings mail:postfix:spam_subject_tag = "***DIEEVILSPAMMERSDIE*** "A number of admins also choose to disable greylisting, done using the mail:postfix:greylist_disable option:
sudo serveradmin settings mail:postfix:greylist_disable = noTo configure an email address for quarantined mail to go, use mail:postfix:virus_quarantine:
sudo serveradmin settings mail:postfix:virus_quarantine = "email@example.com"The administrator, by default, doesn’t get an email when an email containing a file infected with a virus is sent through the server. To enable this option:
sudo serveradmin settings mail:postfix:virus_notify_admin = yesI also find a lot of Mac environments want to accept email of pretty much any size. By default, message size limits are enabled. To disable:
sudo serveradmin settings mail:postfix:message_size_limit_enabled = yesOr even better, just set new limit:
sudo serveradmin settings mail:postfix:message_size_limit = 10485760And to configure the percentage of someone’s quota that kicks an alert (soft quota):
sudo serveradmin settings mail:imap:quotawarn = 75Additionally, the following arrays are pretty helpful, which used to have GUI options:
- mail:postfix:mynetworks:_array_index:0 = “127.0.0.0/8″ – Add entries to this one to add “local” clients
- mail:postfix:host_whitelist = _empty_array – Add whitelisted hosts
- mail:postfix:blacklist_from = _empty_array – Add blacklisted hosts
- mail:postfix:black_hole_domains:_array_index:0 = “zen.spamhaus.org” – Add additional RBL Servers
Comments on this site have been a pain since I enabled them about 2 1/2 years ago. I believe I enabled them due to something some judgmental person said when they couldn’t comment on an article I had written. During the first year, there was a lot of fine tuning the spam blocking to try and keep out the spammy crap. That continues to be a work in progress, but it seems to be in pretty good shape. During those couple of years I ended up racking up a queue of about 7,000 in the spam category and another 2,000+ in the pending category (which meant I need to deal with them). I was dealing with comments every day, but I’d miss a few and it built up over the course of a couple of years. Tonight, I either addressed or cleared out all but 17. My database is much happier. The 17 remaining are thoughtful questions and require thoughtful answers, so I’ll get to them when I have time to provide such an answer. In the meantime, note that now that it’s all cleaned up, if there are any comments, feel free to post and I should actually respond at this point… Sorry for being latent on those up ’till now.
There are a number of ways that you can protect your WordPress site from spam bots. The first is to only allow authenticated users to post comments. Doing so can still be a bit unwieldy, but this feature is built into WordPress and so pretty straight forward to use. Some, who deal with large amounts of spam bots then choose to completely disable the commenting feature outright (Settings -> Discussion -> Uncheck Allow people to post comments on new articles), but comments can still be made on existing articles and commentary is one of the best features of WordPress for many. To stop comments on older articles, also disable commenting on older articles (same page but also choose the Automatically close comments on articles older than option as well). No site should have to disable comments or bend to the will of a spam bot. You can also then choose (same page again) to email the administrator when a comment is made and then choose to not publish comments until the administrator approves them. But spam bots will still attack, and now you’ll just get a ton of junk email. So many will turn to plug-ins for WordPress. There are a few of those that I like a lot. One is called Invisible Defender. Invisible Defender adds a couple of fields that are suppressed using the style sheets. These invisible comment fields, because they’re not displayed to a browser should then never be filled out. Therefore, if a field is filled out, it had to have been done by a bot. Those comments are then automatically blocked. Then there’s the ability to force captcha (shows you funny garbled letters and you type them into a verify field). Captcha for account creation means that all but the most sophisticated bots will fail. This form of forcing an additional form of verification that a visitor is a real human can then be circumvented by users of OpenID, FaceBook and other services, using plug-ins that allow those users to be authenticated through the third party (typically requires a little theme customization). Then there are the antispambee and akismet plug-ins, which look at the actual comments and attempt to determine which ones are spam. These make a good layer of defense but should not be the only layer used. Regrettably, any time you have user generated content on a web site you are going to have automated bots attempting to do a number of things, most likely sell black market pharmaceuticals and other items of questionable origin. There are also bots that attempt to exploit the login page of the WordPress admin (<DOMAIN>/wp-admin.php or /wp-login.php. These are defeated an entirely different way. One of the best strategies is to lock out those who have attempted a number of invalid attempts that exceeds a threshold that you define. Amongst those is Login Lockdown WordPress Security. Another layer for protecting the administrative side of the site is to add an .htaccess file to provide an additional layer of security on top of WordPress. You can also change the URLs of your login page, which I usually use a plug-in called Stealth Login for. Finally, I like to back up WordPress in an automated fashion. There are a lot of plug-ins to do this, but I’ve always used WordPress Database Backup. Why? Because it works every time I tested it. I haven’t even bothered to test a good backup and restore for another software package because WordPress Database Backup always works, backs up data to another server I have, and it hasn’t failed me yet. I always test the restores of data that I’m backing up and I recommend that you test this (mileage may vary) if you choose to put it into production as well (false senses of security are in many cases worse than no security).
I’m just not that social. Therefore, I have removed Mingle, the social networking aspect of Krypted.com. I may reintroduce something like this in the future, but for now it seems that it’s just a source of frustration…
WordPress uses MySQL as a back-end. I’ve seen a number of scenarios where someone was comment spammed. The comments weren’t approved and so never appeared on the site, but they were starting to fill up the MySQL database given that there were about 40,000 in one case and about 55,000 in another. In order to trash them you can use the following query from mysqladmin (once connected to the database of course):
DELETE FROM wp_comments WHERE comment_approved = ‘0’
I once denied someone’s request to add me as a friend on Facebook and got an earful about how they bought one of my books and couldn’t believe I would be so rude, etc. Since then I’ve been an open networker on most of the social networks. It’s kinda’ weird sometimes to listen to people talk about how they keep track of their friends through feeds when I have too many to keep track of, but the tools continue to become more sophisticated and I’m getting closer to be able to do so. Having said that, there is a new thing I’ve been noticing recently. Someone adds you as a friend and then tags you in a photo that you’re not in. Perhaps its a photo where they’ve added your profile picture or your name for a character from a comic book or video game. There’s really no reason for them to do this. You click on their profile and they have thousands of friends and have posted hundreds of these things, perhaps too many to do by hand. Wondering why they’re doing it I browse around their profile, looking for links or something to explain why – but nothing… Most spam that we get is for someone to make a buck off a product. Maybe they’re selling Viagra, maybe it’s someone trying to get you to wire them money so they can send you that $10,000,000 check that only you can cash or maybe it’s for some seedy website. Either way they want you to do something that results in payment being made to them through some fashion. This is different. It’s people just posting weird collages of other people’s profile pictures and then tagging 16 to 20 people. But there’s no apparent financial gain. It’s confusing to me… Why do it? The only fix I can think of is to de-friend them and/or just untag the photos. But now I feel the need to track them and try and figure out what the point is, or whether there is a point…
There are so many types of spam I’m starting to loose track… I check the logs for my web site occasionally. I don’t typically have time to look more than once a week and I don’t have any time to correlate the logs against the articles or do any kind of statistical analysis. I just post what I am thinking about or working on and that’s basically what I have always felt a site like this should be. I guess looking at the logs is just looking for a little external validation… When I check the logs the number one thing I’m looking for is what sites are referrals coming from. I’m noticing a rising trend: web log spam. Basically, the referrer is a site and that site does not actually refer traffic to my site, but it does oddly enough show up in the logs. There is no trace of my site on the referrer’s site and no malicious attempt on my site is made. Additionally, the referrer doesn’t link to an image specifically but the root of the site, so I know it isn’t funny erroneous use of images from my site (which I also see from time to time). Instead it’s completely illegitimate traffic that seems to have roots in a weird advertising campaign: namely to get me to go to their site. At first I was rather annoyed. Then I felt like King Midas and thought maybe I should just stay away from my logs. But at the end of the day, I am actually amused and maybe just a little impressed with the originality. It seems there is no shortage of advertising vehicles for those who want me to look at their site badly enough. My own strategy for advertising, in case you hadn’t noticed, is to write bad articles and hope people come back anyways – so they’ve clearly got one up on me!
Captcha is a nice anti-spam technique for websites. By forcing a user to enter a word that is a bit scrambled on the screen you can eliminate a large amount of spam that you would otherwise have to manually fix. reCaptcha is a free service that provides captcha functionality through an API. That’s what I’m using on this site and to be honest what I’m growing quite fond of. At this point I’ve leveraged it for about 5 sites in the past month and all have seen a dramatic drop in spam over previous techniques I’ve tried. This has been across Joomla!, WordPress and the latest: MediaWiki. The ConfirmEdit Extension for MediaWiki makes it pretty simple to install and get up and running. Check it out here.
24,000 comments. Pretty much all spam. I think I have to disable comments now… 🙁 I hate spam…