GFI/SORBS – should I use them?

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.
Today I’m looking at how this information should affect your choice of spam filtering technology.

Should my spam filters use GFI/SORBS data?

Simply, NO. The quality of reputation data GFI provide is too false-positive prone to rely on in production, even as part of a scoring system.

After all, a false positive is far worse than a false negative, as far as RBL (or general filtering system) usability is concerned.@delivery_kitty

And the problem isn’t just false positives.

Because it takes a long time before a spamming IP address reliably appears on the blacklist, not much spam is stopped. SORBS appears completely unsuitable to the most common way of spamming, via botnets.abuse department at xs4all, a major EU ISP

Your ISP
If you receive mail via your ISP then you’re unlikely to have problems with SORBS blocking your mail, as very few successful ISPs will use it for blocking outright. If you’re at a smaller ISP then they may well be using spam filters such as SpamAssassin, with a dependency on GFI / SORBS data sources, though.
But it’s not worth contacting your ISP unless you find out mail is being bounced or put into a junk folder due to a SORBS listing, or if you can tell by looking at the headers of email you receive that it’s being scored against SORBS.
(If you’re concerned about use of third-party reputation sources by your ISP, you could ask them to provide – or, better, publish – a list of the data sources they use, so their customers can make well-informed decisions about their filtering.)
Your Mailserver
If you run your own inbound mailserver, make sure it is not configured to use any of the SORBS blacklists for blocking email. How to do that varies depending on the server, but for commonly used linux mailservers grepping the configuration files for the string “sorbs” is probably a good place to check.
(There are some great blacklists, with very low false positive rates, to consider using instead – for IP based reputation: spamhaus zen, spamcop, cbl – and for URL reputation: spamhaus dbl, uribl, surbl)
SpamAssassin
SpamAssassin is a widely used server-side score based spam filter. Unfortunately it seems to ship with SORBS blacklists turned on “out of the box”.
I believe that adding the following to /etc/spamassassin/local.cf will disable it – I could be wrong, and would appreciate feedback from any SpamAssassin experts out there.

score RCVD_IN_SORBS_BLOCK 0
score RCVD_IN_SORBS_DUL 0
score RCVD_IN_SORBS_HTTP 0
score RCVD_IN_SORBS_MISC 0
score RCVD_IN_SORBS_SMTP 0
score RCVD_IN_SORBS_SOCKS 0
score RCVD_IN_SORBS_WEB 0
score RCVD_IN_SORBS_ZOMBIE 0

The default SpamAssassin scores are pretty low, so it doesn’t pay that much attention to SORBS – but that a spam filter as influential as SpamAssassin uses such a poor source of data at all is a bit of a problem. Hopefully the SpamAssassin developers will look at the issue for a future release.
Commercial Products
If you’re using a commercial spam filter, check where they’re getting their reputation data from. If you have an existing commercial filter that can use external blacklists, make sure it’s use of SORBS is disabled.
If you’re considering purchasing a new commercial spam filter, there are two things you need to consider. First, if the filter supports using SORBS or other GFI-derived reputation data make sure that can be disabled. Second, if you’re considering a commercial product that uses SORBS or GFI data out of the box, despite the multi-year history of false positives and other problems, think about how solid their other product engineering decisions might be.

I for one will not be considering any products from GFI. I had budgeted and received approval for $20K worth of GFI NSM next year. I will not be making that purchase after this latest episode with SORBS.Skyhawk

Outsourced Services
Outsourced spam filtering services can be very opaque about what approaches they use to decide whether or not email is spam, and will often hide their use of external reputation services.
Some of them are more open than others. Proofpoint posted SORBS DUHL DNS Block List Causing Widespread Email Deliverability Issues Once Again (note that GFI told Proofpoint the problem was fixed on Nov 30th, which we know isn’t true), but it’s rare that a SaaS provider will be that open about how a problem is caused by their reliance on a third-party service. Kudos to Proofpoint for their openness (though they should look elsewhere for reputation data).
Edit: Proofpoint have clarified that they were discussing the problems some of their customers had sending email due to false SORBS listings – not that they were using any data from GFI themselves. Sorry, guys. So if you’re looking for a filtering appliance or outsourced service that’s GFI/SORBS-free (and also quite a nice product), Proofpoint is worth a look.
If your SaaS or outsourced spam filter provider has a clear statement in their product description or contract which third-party data sources they use, then you have the information you need. If not, you should probably contact your support representative and find out whether they use SORBS or not. If they decline to make any statement on it, assume the worst.

In the case of SORBS, this is (at least) the second major misclassification issue we’ve observed in the last 90 days. Email administrators who currently rely on SORBS should be aware of these issues and take action as necessary.Proofpoint

There’s nothing wrong with an outsourced provider using reputable third-party services but if they’re relying on poor quality data sources you may find mail to you being bounced for no good reason, at any time. If that’s the situation you’d be well advised to consider looking at alternative filtering providers.

And Finally

There’s a lot more that could be said, but I’m sure you’re interested in seeing some non-GFI/SORBS content on this blog (and there’s a limit to the amount of technical and business analysis I really want to do for someone other than a paying customer).
Laura will probably revisit the subject soon, going into some more detail about the policy problems that I just touched lightly on and looking more generally about what other companies can learn. And I know several other industry bloggers are planning on discussing GFI and SORBS in the next week or two.
I’ll be gathering links and some other information, including a PDF version of this series of articles suitable for mailing out, at https://wordtothewise.com/sorbs/ over the next day or two.