Blocking

Do spamtraps exist?

One of the folks on the Email Geeks slack asked me a question last week that I thought was really insightful and has a somewhat nuanced answer.

The Economics of Cold Outreach

It’s time we talk about cold outreach mail. In the last 2 years the volume and aggressiveness of cold outreach mail seems to have exploded. There are dozens of companies out there who are selling services to companies to facilitate cold outreach. My own sales mailbox is full of requests from companies to help them solve their delivery problems.

Deliverability matters because we are the conscience of our companies. We are the ones who tell our companies, and particularly the marketing team, no. We’re the ones looking out for the health of our company reputation, the recipient’s inbox and the email ecosystem as a whole.

They Must Have Changed Something…

One of the most common refrains I hear from folks with delivery problems is that the filters must have changed because their mail suddenly started to go to the bulk folder. A few years ago, I posted about how even when there is no change in the sender’s behavior, reputation can slowly erode until mail suddenly goes to the Gmail bulk folder. Much of that still applies – although the comments on pixel loads (what other folks call ‘open rates’) are a bit outdated due to changes in Gmail behavior.

Is email dead?

These last few years have been something, huh? Something had to give and, in my case, that something was blogging. There were a number of reasons I stopped writing here, many of them personal, some of them more global. I will admit, I was (and still am a little) burned out as it seemed I was saying and writing the same things I’d been saying and writing for more than a decade. Taking time off has helped a little bit, as much to focus on what I really want to talk about.

Mentally modelling filters

When we talk about filters, we often think there is one filter. But, in many cases there are multiple stages of filters, each examining mail in a different way.

Increase in Barracuda IP blocks

A number of folks are talking about a significant uptick in Barracuda IP blocks over the last few days. These blocks appear to be affecting wide ranges of IPs across multiple networks.

All filters are not equal

Many questions about delivery problems often assume that there is one standard email filter and the rules are the same across all of them. Unfortunately, this isn’t really the case.

5 steps for addressing deliverability issues

Following on from my reading between the lines post I want to talk a little bit about using the channels. From my perspective the right way to deal with 99% of issues is through the front door.
Last week I found myself talking to multiple folks in multiple fora (emailgeeks slack channel, mailop, IRC) about how to resolve blocking issues or questions. All too often, folks come into these spaces and start by asking “does anyone know someone at…” Fundamentally, that’s the wrong first question. Even if the answer is yes. It’s even the wrong question if a representative of the company is on the list where you’re asking for help.
If that’s the wrong question, what is the right question? Where can we start to get help with issues when we’re stuck trying to fix a delivery problem we don’t understand?

Delisting at ATT

ATT used to have a webform to use to request delisting. I’ve heard reports over the last few months that the form isn’t working. This week, the website hosting the form disappeared. I don’t know for sure, but this looks like this is either deliberate or there’s just no one in charge of the site and it got lost.
ATT provides an email address for delisting, too. Unfortunately, I’m also hearing they’re not responding to that address. There are two possible reasons. One, they’ve never answered and they just delist or not depending on stats. Two, they’re not monitoring that address, either.
In any case, the delisting isn’t working and I don’t know when it will be. I know some people have contacted ATT reps, so they are aware of the current issues. More as I find out.

Filters do what we tell them

In the email space we talk about filters as if they were sentient beings. “The filters decided…” “The filters said…” This is convenient shorthand, but tends to mask that filters aren’t actually deciding or saying anything. Filters are software processes that follow rules dictated by the people who create and maintain them. The rules flow from the goals set by the mailbox provider. The mailbox provider sets goals based on what their users tell them. Users communicate what they want by how they interact with email.

What we end up with is a model where a set of people make decisions about what mail should be let in. They pass that decision on to the people who write the filters. The people who write the filters create software that evaluates email based on those goals using information collected from many places, including the endusers.
What mail should be let in is an interesting question, with answers that differ depending on the environment the filter is deployed in.
Consumer ISPs typically want to keep their users happy and safe. Their goals are to stop harmful mail like phishing, or mail containing viruses or malware. They also want to deliver mail that makes their users happy. As one ISP employee put it, “We want our users to be delighted with your mail.”
Businesses have a few other goals when it comes to filters. They, too, need filters to protect their network from malicious actors. As businesses are often directly targeted by bad actors, this is even more important. They also want to get business related email, whether that be from customers or vendors. They may want to ensure that certain records are kept and laws are followed.
Governments have another set of goals. Universities and schools have yet another set of goals. And, of course, there are folks who run their own systems for their own use.
Complicating the whole thing is that some groups have different tolerances for mistakes. For instance, many of our customers are folks dealing with being blocked by commercial filters. Therefore, we don’t run commercial filters. That does mean we see a lot of viruses and malware and rely on other strategies to stop a compromise, strategies that wouldn’t be as viable in a different environment.
Filters are built to meet specific user needs. What they do isn’t random, it’s not unknowable. They are designed to accomplished certain goals and generally they’re pretty good at what they do. Understanding the underlying goals of filters can help drive solutions to poor delivery.
Use the shorthand, talk about what filters are doing. But remember that there are people behind the filters. Those filters are constantly maintained in order to keep up with ever changing mail streams. They aren’t static and they aren’t forgotten. They are updated regularly. They are fluid, just like the mail they act on.

Use the form…

A lot of senders get frustrated with the time it can take to get a response from some ISPs. It’s totally understandable, for a lot of companies delivery problems are all hands on deck level problems. They want them fixed and they want them fixed IMMEDIATELY. They want feedback that their issue is being addressed. They want to know someone at the ISP knows there is a problem.
I’ve talked before about visiting my friend Anna and watching her laptop screen explode with IMs from senders who wanted help with an AOL issue. She’s awesome and conscientious and tried to address all of those issues as fast as she could. She did want senders to feel like their issues were important and that someone inside AOL cared about the mail blocks.

I was always a strong advocate for following the official pathways for addressing problems. That was the whole point of the 2009 blog post. These days it’s easier to do than it ever was. Many ISPs have forms and process around handling delivery issues. This is good! In the past getting an answer to “why is my mail blocked” required knowing the right people. Now, it’s not about who you know. The ISPs and filtering companies who are open to senders have postmaster pages, unblock forms and official request channels. Those that don’t have those channels have made certain business decisions to not provide support for senders.
Despite the availability of webforms and knowledge bases and detailed information, a lot of people still think that the only way to get attention or get an issue addressed is to get someone on the phone. It’s not, though.
ISPs have their processes. If you want things handled quickly use those processes. Even in the places where very helpful reps are, they can’t (on order of lawyers and executives) help people unless there is a ticket already open.
Always, always use the recommended processes before trying to find “a real person.” Most of the time your issue can be solved faster if you fill out the form than if you hunt around for a person. In the worst case, all that time will be wasted as the person in question will tell you to fill out the form.

Prepping for EEC

Tomorrow I head off to New Orleans to the EEC conference. It’s my first one and I’m really looking forward to meeting some of the people I only know online.
I’ll be speaking on two panels on Friday:
All You Ever Wanted to Know about Deliverability (But Were Afraid to Ask) at 10:50. This is your chance to ask those questions of myself and other experts in the field. I always enjoy Q&A panels and actually hearing from folks what their big deliverability questions are. (and remember, if you have a question, you can always send one to me for Ask Laura)
and the closing Keynote panel
ISP Postmasters & Blacklist Operators: Defending Consumer Inboxes at 1:10. I’m on a panel with various ISP postmasters, blacklist operators and we’ll be talking about what it’s like dealing with the deluge of mail. For instance, there is a huge outbreak of bot-spam at the moment, and a lot of the filters are struggling to keep up. In fact, I’m a last minute replacement for one filter company as they are in all-hands-on-deck firefighting mode to keep their customers safe.
Hope to see you there!

Ethics in Internet Operations

In early September, I posted about a survey being done by Jan Schaumann regarding how sysadmins viewed their ethical obligations with regard to users. The results of this have now been published by Jan. He’s also shared his talk and slides on the data.
Well worth a look through the data. I took a quick run through of his talk and it looked interesting and is definitely going on my to-read list.

Do you run spam filters?

Jan Schaumann is putting together a talk on ethics in as related to folks managing internet operations. He has a survey and is looking for folks who wrangle the machines that run the internet. I’m copying his post, with permission, due to a slightly NSFW image on his announcement.

Do system administrators have too much power?

Yesterday, Laura brought a thread from last week to my attention, and the old-school ISP admin and mail geek in me felt the need to jump up and say something in response to Paul’s comment. My text here is all my own, and is based upon personal experience as well as those of my friends. That said, I’m not speaking on their behalf, either. 🙂
I found Paul’s use of the word ‘SysAdmin’ to be a mighty wide (and — in my experience — probably incorrect) brush to be painting with, particularly when referring to operations at ISPs with any significant number of mailboxes. My fundamental opposition to use of the term comes down to this: It’s no longer 1998.
The sort of rogue (or perhaps ‘maverick’) behavior to which you refer absolutely used to be a thing, back when a clean 56k dial-up connection was the stuff of dreams and any ISP that had gone through the trouble to figure out how to get past the 64k user limit in the UNIX password file was considered both large and technically competent. Outside of a few edge cases, I don’t know many system administrators these days who are able to (whether by policy or by access controls) — much less want to — make such unilateral deliverability decisions.
While specialization may be for insects, it’s also inevitable whenever a system grows past a certain point. When I started in the field, there were entire ISPs that were one-man shows (at least on the technical side). This simply doesn’t scale. Eventually, you start breaking things up into departments, then into services, then teams assigned to services, then parts of services assigned to teams, and back up the other side of the mountain, until you end up with a whole department whose job it is to run one component of one service.
For instance, let’s take inbound (just inbound) email. It’s not uncommon for a large ISP to have several technical teams responsible for the processing of mail being sent to their users:

Politics and Delivery

Last week I posted some deliverability advice for the DNC based on their acquisition of President Obama’s 2012 campaign database. Paul asked a question on that post that I think is worth some attention.

Data is the key to deliverability

Last week I had the pleasure of speaking to the Sendgrid Customer Advisory Board about email and deliverability. As usually happens when I give talks, I learned a bunch of new things that I’m now integrating into my mental model of email.
One thing that bubbled up to take over a lot of my thought processes is how important data collection and data maintenance is to deliverability. In fact, I’m reaching the conclusion that the vast majority of deliverability problems stem from data issues. How data is collected, how data is managed, how data is maintained all impact how well email is delivered.
Collecting Data
There are many pathways used to collect data for email: online purchases, in-store purchases, signups on websites, registration cards, trade shows, fishbowl drops, purchases, co-reg… the list goes on and on. In today’s world there is a big push to make data collection as frictionless as possible. Making collection processes frictionless (or low friction) often means limiting data checking and correction. In email this can result in mail going to people who never signed up. Filters are actually really good at identifying mail streams going to the wrong people.
The end result of poor data collection processes is poor delivery.
There are lots of way to collect data that incorporates some level of data checking and verifying the customer’s identity. There are ways to do this without adding any friction, even. About 8 years ago I was working with a major retailer that was dealing with a SBL listing due to bad addresses in their store signup program. What they ended up implementing was tagged coupons emailed to the user. When the user went to the store to redeem the coupons, the email address was confirmed as associated with the account. We took what the customers were doing anyway, and turned it into a way to do closed loop confirmation of their email address.
Managing Data
Data management is a major challenge for lots of senders. Data gets pulled out of the database of record and then put into silos for different marketing efforts. If the data flow isn’t managed well, the different streams can have different bounce or activity data. In a worst case scenario, bad addressees like spamtraps, can be reactivated and lead to blocking.
This isn’t theoretical. Last year I worked with a major political group that was dealing with a SBL issue directly related to poor data management. Multiple databases were used to store data and there was no central database. Because of this, unsubscribed and inactivated addresses were reactivated. This included a set of data that was inactivated to deal with a previous SBL listing. Eventually, spamtraps were mailed again and they were blocked. Working with the client data team, we clarified and improved the data flow so that inactive addresses could not get accidentally or unknowingly reactivated.
Maintaining Data
A dozen years ago few companies needed to think about any data maintenance processes other than “it bounces and we remove it.” Most mailbox accounts were tied into dialup or broadband accounts. Accounts lasted until the user stopped paying and then mail started bouncing. Additionally, mailbox accounts often had small limits on how much data they could hold. My first ISP account was limited to 10MB, and that included anything I published on my website. I would archive mail monthly to keep mail from bouncing due to a full mailbox.
But that’s not how email works today. Many people have migrated to free webmail providers for email. This means they can create (and abandon) addresses at any time. Free webmail providers have their own rules for bouncing mail, but generally accounts last for months or even years after the user has stopped logging into them. With the advent of multi gigabyte storage limits, accounts almost never fill up.
These days, companies need to address what they’re going to do with data if there’s no interaction with the recipient in a certain time period. Otherwise, bad data just keeps accumulating and lowering deliverability.
Deliverability is all about the data. Good data collection and good data management and good data maintenance results in good email delivery. Doing the wrong thing with data leads to delivery problems.

Email filtering: not going away.

I don’t do a whole lot of filtering of comments here. There are a couple people who are moderated, but generally if the comments contribute to a discussion they get to be posted. I do get the occasional angry or incoherent comment. And sometimes I get a comment that is triggers me to write an entire blog post pointing out the problems with the comment.
Today a comment from Joe King showed up for The Myth of the Low Complaint Rate.

URL reputation and shorteners

A bit of a throwback post from Steve a few years ago. The problem has gotten a little better as some shortening companies are actually disabling spammed URLs, and blocking URLs with problematic content. I still don’t recommend using a public URL shortener in email messages, though.
Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.

Does volume cause blocking?

There seems to be a never ending debate about volume and how it affects delivery and revenue. I regularly get questions asking if ISPs block senders just for volume.
The answer is no. Unless you’re actually sending enough mail to overwhelm the incoming infrastructure, something that’s difficult on today’s internet, you’re unlikely to be blocked due to simply sending a high volume of mail.
Sending mail recipients don’t want, or mail that looks like spam, that will get the mail blocked or filtered.

Don't wait to address delivery problems

One of the worst ways to deal with blocking issues is to ignore them and hope your mail magically moves from the bulk folder back into the inbox. While this does happen as ISPs and filter companies update their filters, it’s not that common and it’s usually the result of a sender actually cleaning up their sending processes and improving the quality of the mail they send.
Do not ignore blocks. What I generally tell people is that it takes at least as long to repair a bad reputation as it took to get that bad reputation in the first place. If you wait months before actually addressing delivery problems, you’re not going to make a change and have the filters react in hours.
This doesn’t mean that every block is a business crisis. Blocks happen and they do go up and down based on thresholds and automatic monitoring scripts and content. But if a block happens consistently for 4 or 5 days in a row it is time to look at what you’re doing. Don’t just focus on the sidelines and little stuff, either. Look at your marketing program and the mail you’re sending.

Spam is not a moral judgement

Mention an email is spam to some senders and watch them dance around trying to explain all the ways they aren’t spammers. At some point, calling an email spam seems to have gone from a statement of fact into some sort of moral judgement on the sender. But calling an email spam is not a moral judgement. It’s just a statement of what a particular recipient thinks of an email.
There are lots of reasons mail can be blocked and not all those reasons are spam related. Sometimes it’s a policy based rejection. Mailbox providers publishing a DMARC record with a reject policy caused a lot of mail to bounce, but none of that was because that user (or that mailing list) was sending spam. Most cable companies prohibit customers from running mail servers on their cable connection and mail from those companies is widely rejected, but that doesn’t mean the mail is spam.
Sometimes a block is because some of the mail is being sent to people who didn’t ask for it or are complaining about it. This doesn’t make the sender a bad person. It doesn’t make the sending company bad. It just means that there is some issue with a part of the marketing program that need to be addressed.
The biggest problem I see is some senders get so invested in convincing receivers, delivery experts and filtering companies that they’re not spammers, that they miss actually fixing the problem. They are so worried that someone might think they’re spammers, they don’t actually listen to what’s being said by the blocking organization, or by their ISP or by their ESP.
Calling email spam isn’t a moral judgement. But, if too many people call a particular email spam, it’s going to be challenging to get that mail to the inbox. Instead of arguing with those people, and the filters that listen to them, a better use of time and energy is fixing the reasons people aren’t liking your email.

Contacting an ISP that doesn't have a postmaster page

How do you contact an ISP about a block that doesn’t have a postmaster page? While there’s no one answer, I do have some suggestions.
Start by contacting the postmaster@ or abuse@ addresses. For smaller ISPs, the same people handling outbound abuse are the people handling inbound filtering.
When you contact them have the following:

What is a dot-zero listing?

Some email blacklists focus solely on allowing their users to block mail from problematic sources. Others aim to reduce the amount of bad mail sent and prefer senders clean up their practices, rather than just blocking them wholesale. The Spamhaus SBL is one of the second type, using listings both to block mail permanently from irredeemable spammers and as short term encouragement for a sender to fix their practices.
All a blacklists infrastructure – and the infrastructure of related companies, such as reputation monitoring services – is based on identifying senders by their IP addresses and recording their misbehaviour as records associated with those IP addresses. For example, one test entry for the SBL is the IP address 192.203.178.107, and the associated record is SBL230. Because of that they tend not to have a good way to deal with entities that aren’t associated with an IP address range.
Sometimes a blacklist operator would like put a sender on notice that the mail they’re emitting is a problem, and that they should take steps to fix that, but they don’t want to actually block that senders mail immediately. How to do that, within the constraints of the IP address based blacklist infrastructure?
IP addresses are assigned to users in contiguous blocks and there’s always a few wasted, as you can’t use the first or last addresses in that range (for technical / historical reasons). Our main network consists of 128 IP addresses, 184.105.179.128 to 184.105.179.255, but we can’t put servers on 184.105.179.128 (as it’s our router) or 184.105.179.255 (as it’s the “broadcast address” for our subnet).
So if Spamhaus wanted to warn us that we were in danger of having our mail blocked, they could fire a shot across our bow without risk of blocking any mail right now by listing the first address in our subnet – 184.105.179.128 – knowing that we don’t have a server running on that address.
For any organization with more than 128 IP addresses – which includes pretty much all ISPs and ESPs – IP addresses are assigned such that the first IP address in the range ends in a zero, so that warning listing will be for an address “x.y.z.0” – it’s a dot-zero listing.

The 5 stages of a Spamhaus listing

Courtesy of Spencer over at Experian.
The 5 Stages of Recovery

Questions about Spamhaus

I have gotten a lot of questions about Spamhaus since I’ve been talking about them on the blog and on various mailing lists. Those questions can be condensed and summed up into a single thought.

Censoring email

It seems some mail to Apple’s iCloud has been caught in filters. Apparently, a few months ago someone sent a script to a iCloud user that contained the phrase “barely legal teen” and Apple’s filters ate it.
The amount of hysteria that I’ve seen in some places about this, though, seems excessive. One of my favorite quotes was from MacWorld and just tells me that many of the people reporting on filtering have no idea how filters really work.

Spamhaus changes

A number of ESPs are reporting an increase in SBL listings of big, well known brands. InterestingSBLs seems to confirm this.
Just on the month of June I see tweets reporting SBL listings for: Disney (again, and again) AAA Michigan, NRCC, the Mitt Romney campaign, Macy’s (again) Facebook, Walmart Brazil, Safeway, Bacardi.
What happened? I think there are a number of reasons for an increase in SBL listings of well known brands.
The first is that botnets are rapidly becoming a solved problem. That’s not to say that they’ve gone away, or that we should stop being vigilant about the spam and malicious mail coming out of them, but that there are more and better tools to deal with botnets than there have been in the past. That means that the folks at Spamhaus can look at different classes of unsolicited email.
I believe Spamhaus has some new mail feeds that let them see mail they were previously not seeing. Anyone who has multiple email addresses can tell you that the type of spam that one address gets is often vastly different than the type of mail another email address gets. When dealing with spamtrap feeds, that means that there is unsolicited mail that isn’t seen by the feed. I know there are companies who claim to have lists of hundreds of thousands of spamtraps, and I don’t doubt that some enterprising spammers have discovered Spamhaus spamtraps in the past. Adding new feeds means that Spamhaus will see spam that they were previously missing due to their traps being compromised.
As well as bringing up new feeds, I suspect Spamhaus has better tools to mine the data. This means they can see patterns and problem senders in a clearer way and list those that meet the Spamhaus listing criteria.
I’m not saying the Spamhaus standards have changed. Spamhaus has always said they will list anyone sending unsolicited bulk email. But, as with many organizations what they could do was limited by the available resources. That resource allocation has changed and they can deal with more senders.
What does all this mean for senders? In a perfect world it wouldn’t mean anything. Senders would actually be sending mail only to people who had asked to receive it. Senders would have good list hygiene and pull off abandoned addresses long before they could be turned into spamtraps.
But we all know this isn’t a perfect world. There are a lot of senders that have lists with years of cruft on them. And not all of those addresses on the list actually opted-in to receive that mail. Many of those senders have good stats, decent opens, low unknown user rates, and low complaint rates. But that doesn’t mean there aren’t problems with the lists. And those hidden problems may mean that just because you haven’t had a Spamhaus listing in the past doesn’t mean there isn’t going to be one in your future. It means senders who want to avoid SBL listings need to pay attention to list hygiene and dead addresses. It means the source of addresses and their audit trail is even more important than ever.
Meanwhile, ESPs are struggling to cope with the ongoing and increasing SBL listings.
EDIT: Mickey attributes some of the increase in listings to Spamhaus being better able to detect appended lists.

Why so many domains

There’s a company that advertises a lot on TV. The ads are well done, they tell a clear story in the 30 seconds. They feature a pretty and happy young woman dancing around. There is a great catchy tune. From all appearances it’s a successful ad campaign.
The point of the ad campaign is to drive traffic to a website where the domain owner can collect a lot of information and sell it on to advertisers. Every month or so, the landing URL changes. In watching this campaign over the last year or two, I’ve seen at least half a dozen different URLs used in the television ads. Now, it’s perfectly possible that this is part of an overall strategy, but I am not sure. The initial website is highlighted so clearly in the catchy tune, I can’t believe it is part of their marketing strategy.
Which leads me to wonder if there is a bigger problem with their advertising. Do they change domains so frequently because they’re seeing domain based blocking?

Everybody wins!

There was a recent question on a mailing list during a discussion of spam and delivery problems. A number of folks who work in delivery were discussing how a bad address got on a list. Someone who works on the spam blocking end of things asked why do you care how a bad address got onto a mailing list?
For recipients, they usually don’t care. They just want the unsolicited mail to stop. It’s a position I have no problem with; I want the unsolicited mail to stop, too. But understanding why a particular sender is sending mail to addresses that never asked for it can be an important step in making it stop. Not by the receivers and the spam filters, they’ll just block the bad sender and move on. Or if they’re an ISP or ESP they’ll just throw the sender off for AUP violations and let the sender be somebody else’s problem.
In the broader context, though, this only changes the source of the spam. It doesn’t help the victim; the bad sender can always find another host and they will continue to mail people who never asked for that mail. And, in fairness to these senders, often they are mailing lists of mixed sources. Some of the addresses didn’t opt-in, and don’t want the mail, but a lot of addresses on their list did opt-in and do want their mail. Fixing their problem means they can mail people who want their mail. The sender is happy, the recipients are happy and the receivers are happy; everybody wins!
Everybody winning is something I can get fully behind.

Hunting the Human Representative

Yesterday’s post was inspired by a number of questions I’ve fielded recently from people in the email industry. Some were clients, some were colleagues on mailing lists, but in most cases they’d found a delivery issue that they couldn’t solve and were looking for the elusive Human Representative of an ISP.
There was a time when having a contact inside an ISP was almost required to have good delivery. ISPs didn’t have very transparent systems and SMTP rejection messages weren’t very helpful to a sender. Only a very few ISPs even had postmaster pages, and the information there wasn’t always helpful.
More recently that’s changed. It’s no longer required to have a good relationship at the ISPs to get inbox delivery. I can point to a number of reasons this is the case.
ISPs have figured out that providing postmaster pages and more information in rejection messages lowers the cost of dealing with senders. As the economy has struggled ISPs have had to cut back on staff, much like every other business out there. Supporting senders turned into a money and personnel sink that they just couldn’t afford any longer.
Another big issue is the improvement in filters and processing power. Filters that relied on IP addresses and IP reputation did so for mostly technical reasons. IP addresses are the one thing that spammers couldn’t forge (mostly) and checking them could be done quickly so as not to bottleneck mail delivery. But modern fast processors allow more complex information analysis in short periods of time. Not only does this mean more granular filters, but filters can also be more dynamic. Filters block mail, but also self resolve in some set period of time. People don’t need to babysit the filters because if sender behaviour improves, then the filters automatically notice and fall off.
Then we have authentication and the protocols now being layered on top of that. This is a technology that is benefiting everyone, but has been strongly influenced by the ISPs and employees of the ISPs. This permits ISPs to filter on more than just IP reputation, but to include specific domain reputations as well.
Another factor in the removal of the human is that there are a lot of dishonest people out there. Some of those dishonest people send mail. Some of them even found contacts inside the ISPs. Yes, there are some bad people who lied and cheated their way into filtering exceptions. These people were bad enough and caused enough problems for the ISPs and the ISP employees who were lied to that systems started to have fewer and fewer places a human could override the automatic decisions.
All of this contributes to the fact that the Human Representative is becoming a more and more elusive target. In a way that’s good, though; it levels the playing field and doesn’t give con artists and scammers better access to the inbox than honest people. It means that smaller senders have a chance to get mail to the inbox, and it means that fewer people have to make judgement calls about the filters and what mail is worthy or not. All mail is subject to the same conditions.
The Human Representative is endangered. And I think this is a good thing for email.

First step in delivery

Ever trawl through your logs and notice that there is a delivery problem somewhere? I’m sure everyone sending email in any volume has.
What’s the first thing you do when you discover a block?

Spamhaus rising?

Ken has a good article talking about how many ESPs have tightened their standards recently and are really hounding their customers to stop sending mail recipients don’t want and don’t like. Ken credits much of this change to Spamhaus and their new tools.

URL Shortening and Email

Any time you put a URL in mail you send out, you’re sharing the reputation of everyone who uses URLs with that hostname. So if other people send unwanted email that has the same URL in it that can cause your mail to be blocked or sent to the bulk folder.
That has a bunch of implications. If you run an affiliate programme where your affiliates use your URLs then spam sent by your affiliates can cause your (clean, opt-in, transactional) email to be treated as spam. If you send a newsletter with advertisers URLs in it then bad behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as spam.
Even if the hostname you use for your URLs is unique to you, if it resolves to the same IP address as a URL that’s being used in spam, that can cause delivery problems for you.
What does this mean when it comes to using URL shorteners (such as bit.ly, tinyurl.com, etc.) in email you send out? That depends on why you’re using those URL shorteners.
The URLs in the text/html parts of my message are big and ugly
Unless the URL you’re using is, itself, part of your brand identity then you really don’t need to make the URL in the HTML part of the message visible at all. Instead of using ‘<a href=”long_ugly_url”> long_ugly_url </a>’ or ‘<a href=”shortened_url”> shortened_url </a>’ use ‘<a href=”long_ugly_url”> friendly phrase </a>’.
(Whatever you do, don’t use ‘<a href=”long_ugly_url”> different_url </a>’, though – that leads to you falling foul of phishing filters).
The URLs in the text/plain parts of my message are big and ugly
The best solution is to fix your web application so that the URLs are smaller and prettier. That will make you seem less dated and clunky both when you send email, and when your users copy and paste links to your site via email or IM or twitter or whatever. “Cool” or “friendly” URLs are great for a lot of reasons, and this is just one. Tim Berners-Lee has some good thoughts on this, and AListApart has two good articles on how to implement them.
If you can’t do that, then using your own, branded URL shortener is the next best thing. Your domain is part of your brand – you don’t want to hide it.
I want to use a catchy URL shortener to enhance my brand
That’s quite a good reason. But if you’re doing that, you’re probably planning to use your own domain for your URL shortener (Google uses goo.gl, Word to the Wise use wttw.me, etc). That will avoid many of the problems with using a generic URL shortener, whether you implement it yourself or use a third party service to run it.
I want to hide the destination URL from recipients and spam filters
Then you’re probably spamming. Stop doing that.
I want to be able to track clicks on the link, using bit.ly’s neat click track reporting
Bit.ly does have pretty slick reporting. But it’s very weak compared to even the most basic clickthrough reporting an ESP offers. An ESP can tell you not just how many clicks you got on a link, but also which recipients clicked and how many clicks there were for all the links in a particular email or email campaign, and how that correlates with “opens” (however you define that).
So bit.ly’s tracking is great if you’re doing ad-hoc posts to twitter, but if you’re sending bulk email you (or your ESP) can do so much better.
I want people to have a short URL to share on twitter
Almost all twitter clients will abbreviate a URL using some URL shortener automatically if it’s long. Unless you’re planning on using your own branded URL shortener, using someone else’s will just hide your brand. It’s all probably going to get rewritten as t.co/UgLy in the tweet itself anyway.
If your ESP offers their own URL shortener, integrating into their reporting system for URLs in email or on twitter that’s great – they’ll be policing users of that just the same as users of their email service, so you’re unlikely to be sharing it with bad spammers for long enough to matter.
All the cool kids are using bit.ly, so I need to to look cool
This one I can’t help with. You’ll need to decide whether bit.ly links really look cool to your recipient demographic (Spoiler: probably not) and, if so, whether it’s worth the delivery problems they risk causing.
And, remember, your domain is part of your brand. If you’re hiding your domain, you’re hiding your branding.
So… I really do need a URL shortener. Now what?
It’s cheap and easy to register a domain for just your own use as a URL shortener. Simply by having your own domain, you avoid most of the problems. You can run a URL shortener yourself – there are a bunch of freely available packages to do it, or it’s only a few hours work for a developer to create from scratch.
Or you can use a third-party provider to run it for you. (Using a third-party provider does mean that you’re sharing the same IP address as other URL shorteners – but everyone you’re sharing with are probably people like you, running a private URL shortener, so the risk is much, much smaller than using a freely available public URL shortener service.)
These are fairly simple fixes for a problem that’s here today, and is going to get worse in the future.
(9/18/17: Closing comments because this post attracts spam comments)

Bit.ly gets you Blocked

URL shorteners, like bit.ly, moby.to and tinyurl.com, do three things:

Amendment is futile, part 2

When Yahoo filed for dismissal of the Holomaxx complaint, they ended the motion with “Amendment would be futile in this case.” The judge granted Yahoo’s motion but did grant Holomaxx leave to amend. Holomaxx filed an amended complaint earlier this month.
The judge referenced a couple specific deficiencies of Holomaxx’s claims in his dismissal.

Blocklist BCP

As many of you may be aware there is a draft document working its way through the Internet Research Task Force (IRTF) discussing best common practices for blocklists. The IRTF is a parallel organization to the IETF and is charged with long term research related to the Internet. The Anti-Spam Working Group was chartered to investigate tools and techniques for dealing with spam.
Recently the ASRG posted a draft of a best practices document aimed at those running blocklists (draft-irtf-asrg-bcp-blacklists-07). This document has been under development for many years. The authors have used this document to share their experiences with running blocklists and their knowledge of what works and what doesn’t.
Best practices documents are never easy to write and consensus can be difficult. But I think that the authors did a good job capturing what the best practices are for blocklists. I do support the document in principle and, in fact, support many of the specific statements and practices outlined there. As with any best practices documents it’s not perfect but overall it reflects the current best practices for blocklists.
Ken Magill’s article about the BCP
Anti-Abuse buzz article about the BCP

Why is shared hosting like phishing?

A client of a friend was getting rejection messages when they tried to send mail

GFI/SORBS – should I use them?

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.
Today I’m looking at how this information should affect your choice of spam filtering technology.

GFI/SORBS – I'm blacklisted, now what?

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
In the past week we’ve demonstrated that the SORBS reputation data is riddled with mistakes, poor practices, security holes and operational problems, and that the quality of the end result is really too poor to be useful.
What does this mean to you though? There are really two aspects: 1. what to do if you’re blacklisted or blocked by GFI or based on GFI/SORBS data and 2. how this information should affect your choice of spam filtering technology. We’ll be looking at the first point today, and the second tomorrow.

GFI/SORBS considered harmful, part 3

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
In the last few days we’ve talked about GFI’s lack of responsiveness, the poor quality of their reputation and blacklist data, and the interesting details of their DDoS claims. Today we’re going to look at (some of) the fundamental problems with GFI’s procedures and infrastructure that cause those issues. Some of the subset of issues I’ve chosen highlight are minor, some are major, but they show a pattern of poor decisions.
SSL Certificates
When you use SSL on a web connection it brings you two benefits. The first is that it encrypts the connection between your browser and the webserver, so that it’s very difficult for anyone to watch or tamper with your interaction with that webserver. The second, more important, reason is to make sure that you’re talking to the webserver you think you’re talking to, to avoid man-in-the-middle attacks.
This security relies on you trusting the certification authority that issues the SSL certificate that the website uses. A website providing services to the public should always use an SSL certificate created by one of a small number of reputable certification authorities that are pre-loaded into all webservers as “trusted”. These SSL certificates are something that need to be be purchased, but they’re very inexpensive – less than ten dollars a year.

GFI/SORBS – a DDoS Intermezzo

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
I’ve been stage-managing for a production of The Nutcracker this week, so musical terminology is on my mind. In opera, the intermezzo is a comedic interlude between acts of an opera series.
This comedic interlude is about the “DDoS” – a distributed denial of service attack. What is a denial of service attack?

GFI/SORBS considered harmful, part 2

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
Yesterday I talked about GFI responsiveness to queries and delisting requests about SORBS listings. Today I’m going to look at data accuracy.
The two issues are tightly intertwined – a blacklist that isn’t responsive to reports of false positive listings will end up with a lot of stale or inaccurate data, and a blacklist that has many false positives will likely be overwhelmed with complaints and delisting requests, and won’t be able to respond to them – leading to a spiral of dissatisfaction and inaccurate data feeding off each other.

GFI/SORBS considered harmful

Act 1 • Act 2 • Intermezzo • Act 3 • Act 4 • Act 5
Management Summary, Redistributable Documents and Links
A little over a year ago the SORBS blacklist was purchased by GFI Software. I had fairly high hopes that it would improve significantly, start behaving with some level of professionalism and competence and become a useful data source, in much the same way that the SpamCop blacklist turned into an accurate, professionally run source of data after they transitioned from being a volunteer run blacklist to a service of IronPort.
GFI’s statement a year ago was:

It's not illegal to block mail

My post “We’re going to party like it’s 1996” is still getting a lot of comments from people. Based on the comments, either people aren’t reading or my premise wasn’t clear.
Back in 1996 the first lawsuits were brought against ISPs to stop ISPs from blocking email. These suits were failures. Since that time, other senders have attempted to sue ISPs and lost. Laws have been written protecting the rights of the ISPs to block content they deem to be harmful.
Dela says that he was just attempting to open up a conversation, but I don’t see what he thinks the conversation is. That ISPs shouldn’t block mail their customers want? Sure, OK. We’re agreed on that. Now, define what mail recipients want. I want what mail I want, not what someone else decides I might want.
Marketers need to get over the belief that they own end users mailboxes and that they have some right to send mail to people. You don’t.
When marketers actually start sending wanted mail, to people who actually subscribe – not just make a purchase, or register online or happen to have an easily discoverable email address – then perhaps marketers will have some standing to claim they are being treated illegally. Until and unless that happens, the ISPs are well within their rights to block mail that their users don’t want.

Guide to resolving ISP issues

I often get a chuckle out of watching some people, who are normally on the blocking end of the delivery equation, struggle through their own blocking issues. A recent situation came up on a mailing list where someone who has very vehement opinions about how to approach her particular blocklist for delisting and that the lists policies are immutable. The company she works for is having some delivery issues and she’s looking for a contact to resolve the issues.
While digging through my blog posts to see if there was any help I could provide, I realized I don’t have a guide to resolving blocking issues at ISPs. Much of the troubleshooting can be done without ever contacting the ISPs or the blocklists.
Identify the issue.
There are a number of techniques that ISPs use to protect their users from malicious or problematic mail, from rate-liming incoming mail, putting mail in the bulk folder, or blocking specific IP addresses. Step one to resolving any delivery problem is to identify what is happening to the mail. In order to resolve the issue, you have to know what the issue is.
All too often, the description of a delivery problem is: My mail isn’t getting delivered. But that isn’t very clear as to what the actual problem is. Are you being temp failed? Is mail being blocked? Is mail going to the bulk folder? Is this something affecting just you or is it a widespread problem?
Troubleshoot your side.
Collect as much data about the problem as you can. Dig through logs and get copies of any rejection messages. Follow any URLs that are present in the bounce messages. Try sending a bare bones email to yourself at that ISP with just URLs, is it still blocked? What if you send from a different IP, does the same thing happen?
There is a lot of troubleshooting a sender can do without having to contact an ISP, and the information can lead to resolution that doesn’t involve having to contact the ISP. Also, many current ISP blocks are dynamic, they come up and go down without any human intervention. Those blocks that require contact to get them resolved have clear instructions in the bounce message.
Fix your stuff.
Whether it’s a reputation issue or a minor technical issue, fix the problem on your end. Just moving IP addresses or changing a URL isn’t a sustainable fix. There is a reason mail is being blocked or filtered and if you don’t fix that issue, the blocks are just going to come back. After you do fix your stuff, expect to see changes in a few days or a week. The ISP filters are generally quite responsive to sender improvements so if you’ve fixed the stuff you should see changes pretty quickly. Expect unblocking or filtering to take a little longer than the block was in place.
If you can’t figure out what the problem is, hire a consultant. Here at Word to the Wise we can often quickly identify a problem and provide a path to resolution. Sometimes the problem isn’t even the ISPs, we’ve had multiple cases where our clients were using custom software and their software wasn’t SMTP compliant and we were able to identify the problem and get their mail working again. There are a host of other independent consultants out there that can also help you identify and resolve blocking problems.
Contact the ISPs.
If there is a hard block or after fixing what you think the underlying problem is, you’ll have to contact the ISP. Many ISPs provide self service websites and contact forms to facilitate this process. Generally, though, most issues aren’t going to require contact.

Content based filtering

A spam filter looks at many things when it’s deciding whether or not to deliver a message to the recipients inbox, usually divided into two broad categories – the behaviour of the sender and the content of the message.
When we talk about sender behaviour we’ll often dive headfirst into the technical details of how that’s monitored and tracked – history of mail from the same IP address, SPF records, good reverse DNS, send rates and ramping, polite SMTP level behaviour, DKIM and domain-based reputation and so on. If all of those are OK and the mail still doesn’t get delivered then you might throw up your hands, fall back on “it’s content-based filtering” and not leave it at that.
There’s just as much detail and scope for diagnosis in content-based filtering, though, it’s just a bit more complex, so some delivery folks tend to gloss over it. If you’re sending mail that people want to receive, you’re sure you’re sending the mail technically correctly and you have a decent reputation as a sender then it’s time to look at the content.
You want your mail to look just like wanted mail from reputable, competent senders and to look different to unwanted mail, viruses, phishing emails, botnet spoor and so on. And not just to mechanical spam filters – if a postmaster looks at your email, you want it to look clean, honest and competently put together to them too.
Some of the distinctive content differences between wanted and unwanted email are due to the content as written by the sender, some of them are due to senders of unwanted email trying to hide their identity or their content, but many of them are due to the different quality software used to send each sort of mail. Mail clients used by individuals, and content composition software used by high quality ESPs tends to be well written and complies with both the email and MIME RFCs, and the unwritten best common practices for email composition. The software used by spammers, botnets, viruses and low quality ESPs tends not to do so well.
Here’s a (partial) list of some of the things to consider:

We're gonna party like it's 1996!

Over on deliverability.com Dela Quist has a long blog post up talking about how changes to Hotmail and Gmail’s priority inbox are a class action suit waiting to happen.
All I can say is that it’s all been tried before. Cyberpromotions v. AOL started the ball rolling when they tried to use the First Amendment to force AOL to accept their unsolicited email. The courts said No.
Time goes on and things change. No one argues Sanford wasn’t spamming, he even admitted as much in his court documents. He was attempting to force AOL to accept his unsolicited commercial email for their users. Dela’s arguments center around solicited mail, though.
Do I really think that minor difference in terminology going to change things?
No.
First off “solicited” has a very squishy meaning when looking at any company, particularly large national brands. “We bought a list” and “This person made a purchase from us” are more common than any email marketer wants to admit to. Buying, selling and assuming permission are par for the course in the “legitimate” email marketing world. Just because the marketer tells me that I solicited their email does not actually mean I solicited their email.
Secondly, email marketers don’t get to dictate what recipients do and do not want. Do ISPs occasionally make boneheaded filtering decisions? I’d be a fool to say no. But more often than not when an ISP blocks your mail or filters it into the bulk folder they are doing it because the recipients don’t want that mail and don’t care that it’s in the bulk folder. Sorry, much of the incredibly important marketing mail isn’t actually that important to the recipient.
Dela mentions things like bank statements and bills. Does he really think that recipients are too stupid to add the from address to their address books? Or create specific filters so they can get the mail they want? People do this regularly and if they really want mail they have the tools, provided by the ISP, to make the mail they want get to where they want it.
Finally, there is this little law that protects ISPs. 47 USC 230 states:

Getting removed from an ISP block

A question came up on a mailing list about how long it typically took to resolve a spam block at an ISP. I don’t think that question actually has a single answer, as each ISP has their own, special, process.
ISPA takes 5 minutes. You fill out a form, it runs through their automated system and you’re usually delisted.
ISPB asks a lot of questions in their form, so it takes about 15 minutes to collect all the data they want and 10 minutes to fill out their form. Then, using very, very short words you keep repeating what you need to the tier 1 person who initially responded. That person eventually figures out they can’t blow you off and throws your request to tier 2, who handles it immediately.
ISPC has a different, somewhat long form. Again, you spend time collecting all the data and then fill out the somewhat obscure form. You get a response, but it’s a boilerplate totally unrelated to the initial request, so you keep answering until you find a tier 1 rep who can read and do what you initially asked.
ISPD has a form that takes about 2 minutes to fill out. Unfortunately, it goes to an outsourced postmaster team in the Far East and response times are ranging from days to months right now.
ISPE has an email address and if you catch them on a good day, they’re very helpful. Sometimes there’s no response, though.
ISPF has a troubleshooting page and accept requests to fix things, but never respond in any visible manner.
ISPG they tells you to talk to Spamfiltering Company H.
Spamfiltering company H answers their email in a prompt and friendly manner. OK, sometimes the answers are just “wow, your client/customer/IP range is sending lots of spam,” but hey, it’s an answer.
Spamfiltering company I is a useless bag of protoplasm and don’t even answer the email address they give you on their webpages. In a fit of fairness, I have heard they will occasionally respond, but usually that response is to tell you to go pay some apparently unrelated company a bribe to get delisted.
Spamfiltering company J doesn’t have a lot of ways to contact them, but have a lot of folks that participate in various semi-public arenas so if you’re even slightly part of the community, you can email them and they’re very helpful.
Spamfiltering company K is totally useless, but will tell you to have recipients whitelist you.

I'm on a blocklist! HELP!

Recently, an abuse desk rep asked what to do when customers were complaining about being assigned an IP address located on a blocklist. Because not every blocklist actually affects mail delivery it’s helpful to identify if the listing is causing a problem before diving in and trying to resolve the issue.

Legitimate mail in spamfilters

It can be difficult and frustrating for a sender to understand they whys and wherefores of spam filtering. Clearly the sender is not spamming, so why is their mail getting caught in spam filters?
I have a client that goes through this frustration on rare occasions. They send well crafted, fun, engaging content that their users really want. They have a solid reputation at the ISPs and their inbox stats are always above 98%. Very, very occasionally, though, they will see some filtering difficulties at Postini. It’s sad for all of us because Postini doesn’t tell us enough about what they’re doing to understand what my client is doing to trigger the filters. They get frustrated because they don’t know what’s going wrong; I get frustrated because I can’t really help them, and I’m sure their recipients are frustrated because they don’t get their wanted mail.
Why do a lot of filter vendors not communicate back to listees? Because not all senders are like my clients. Some senders send mail that recipients can take or leave. If the newsletter shows up in their inbox they may read it. If the ad gets in front of their face, they may click through. But, if the mail doesn’t show up, they don’t care. They certainly aren’t going to look for the mail in their bulk folder. Other senders send mail that users really don’t want. It is, flat out, spam.
The thing is, all these senders describe themselves as legitimate email marketers. They harvest addresses, they purchase lists, they send mail to spamtraps, and they still don’t describe themselves as spammers. Some of them have even ended up in court for violating various anti-spam laws and they still claim they’re not spammers.
Senders are competing with spammers for bandwidth and resources at the ISPs, they’re competing for postmaster attention at the ISPs and they’re competing for eyeballs in crowded inboxes.
It’s the sheer volume of spam and the crafty evilness of spammers that drives the constant change and improvement in spamfilters. It’s tough to keep up with the spamfilters because they’re trying to keep up with the spammers. And the spammers are continually looking for new ways to exploit recipients.
It can be a challenge to send relevant, engaging email while dealing with spamfilters and ISPs. But that’s what makes this job so much fun.

The view from a blacklist operator

We run top-level DNS servers for several blacklists including the CBL, the blacklist of infected machines that the SpamHaus XBL is based on. We don’t run the CBL blacklist itself (so we aren’t the right people to contact about a CBL listing) we just run some of the DNS servers – but that means that we do get to see how many different ways people mess up their spam filter configurations.
This is what a valid CBL query looks like:

Spamtraps

There is a lot of mythology surrounding spamtraps, what they are, what they mean, how they’re used and how they get on lists.
Spamtraps are very simply unused addresses that receive spam. They come from a number of places, but the most common spamtraps can be classified in a few ways.

It doesn't matter what you say

“What should we tell the ISP?” is a frequent question from my customers. The answer is pretty simple. It doesn’t usually matter what you tell the ISP. What matters are your actions.
If a sender is having delivery problems then the solution is not to call the ISP and talk to them about why the sender’s mail should not be delivered to the bulk folder. Instead, the solution is to evaluate the email and the address acquisition process and the list hygiene process. Identify where potential problems are and then resolve those problems.
Typically, the ISPs won’t need to be contacted. The changes to the email will register and delivery will improve. In some cases, particularly when there’s been some major mistake, contacting the ISP and explaining the mistake and what steps have been taken to stop the mistake from happening in the future may help resolve the issue faster. But if nothing has changed, then there’s no reason for the ISP to expect anything to change.
It doesn’t matter what you say. It matters what you do.

Bad year coming for sloppy marketers

MediaPost had an article written by George Bilbrey talking about how 2010 could be a difficult year for marketers with marginal practices. George starts off the article by noticing that his contact at ISPs are talking up how legitimate companies with bad practices are causing them problems and are showing up on the radar.
This is something I talked about a few weeks ago, in a series of blog posts looking at the changes in 2010. The signs are out there, and companies with marginal practices are going to see delivery get a lot more difficult. George lists some practices that he sees as problems.

The coming changes

Yesterday I talked about how I’m hearing warnings of a coming paradigm shift in the email industry. While these changes will affect all sender, ESPs in particular are going to need to change how they interact with both ISPs and their customers.
Currently, ESPs are able to act as “routine conveyers.” The traffic going across their network is generated by their customers and the ESP only handles technical issues. Responsible ESPs do enforce standards on their customers and expect mailings to meet certain targets. They monitor complaints and unknown users, they monitor blocks and reputation. If customers get out of line, then the ESP steps in and forces their customer to improve their practices. If the customer refuses, then the ESP disconnects them.
Currently standards for email are mostly dictated by the ISPs. Many ESPs take the stance that if any mail that is not blocked by the ISPs then it is acceptable. But just because a certain customer isn’t blocked doesn’t mean they’re sending mail that is wanted by the recipients.
It seems this reactive approach to customer policing may no longer be enough. In fact, one of the large spam filter providers has recently offered their customers the ability to block mail from all ESPs with a single click. This may become a more common response if the ESPs don’t start proactively policing their networks.
Why is this happening? ISPs and filtering companies are seeing increasing percentages of spam coming out of ESP netspace. Current processes for policing customers are extremely reactive and there are many ESPs that are allowing their customers to send measurable percentages of spam. This situation is untenable for the filtering companies or the ISPs and they’re sending out warnings that the ESPs need to stop letting so much spam leave their networks.
Unsurprisingly, there are many members of the ESP community that don’t like this and think the ISPs are overreacting and being overly mean. They do not think the ISPs or filtering companies should be blocking all an ESPs customers just because some of the customers are sending unwanted mail. Paraphrased, some of the things I’ve heard include:

Legitimate email marketers need to take a stand

I was reading an article on Virus Rants and the opening paragraph really stood out.

Controlling delivery

How much control over delivery do senders have? I have repeatedly said that senders control their delivery. This is mostly true. Senders control their side of the delivery chain, but there is a point where the recipient takes over and controls things.
As a recipient I can

Problems at Cox: Resolved

People mailing to Cox in the wee hours of this morning may have received a rejection message citing the Invaluement DNSBL.
554 IMP a.b.c.d blocked. IPBL100 – Refer to Error Codes section at http://postmaster.cox.net for more information.
I spoke with one of the folks at Cox and they said there was an error in the implementation causing non-listed IPs to be rejected erroneously between about 4am to 8am (Eastern) this morning. The problem has been resolved as of 8am, and all traffic is flowing normally. The also stated that attempts to resend any blocked messages will succeed. They do apologize for any problems this may have caused.
For those of you with aggressive bounce handling, removing addresses after a single 550 bounce, you will also want to re-enable any cox.net subscribers that bounced off during this configuration problem.

Dealing with blacklists

Al has a good post listing the top 5 things senders should remember when dealing with blacklists.
One of the critical things to remember about blocklists is that they are an early warning sign. Sure, some of them are one crank and his cat and will not hurt your overall delivery. A sender may be listed for totally spurious reasons . On the other hand, many of the widely used public lists and the private lists at the big ISPs, list IPs that they see as doing something wrong.
The challenge for anyone listed on any IP based blocklist is to look inside and determine what it is that they’re doing that caused the listing. The first step is to look at the technical issues, does your mail look like something coming out of infected bots? Is there a configuration problem? If the answer is no, then senders have to look at their practices. Are they sending mail to people who don’t expect it? Are they sending mail to people who didn’t ask for it? Most listings that will affect large numbers of recipients fall into the above 2 categories: technical or practices.
Technical problems can be fixed easily, once they’re identified. Permission or practice problems can also be fixed, but may require a sender reassess how they are using email and what value email brings to the business.

The secret to dealing with ISPs

What is the secret to dealing with ISPs?
The short answer is: Don’t do it if at all possible. Talking to ISP reps generally isn’t going to magically improve your reptuation. There is no place in the reputation systems where delivery can be modified because the delivery specialist knows or is liked by the postmaster at an ISP.
With my clients, I work through delivery issues and can solve 80 – 90% of the issues without ever having to contact anyone at the ISPs. 90% of the remaining issues can be handled using the publicly available contacts and websites provided by the ISPs.
In the remaining cases, the “secret” to getting useful and prompt replies is to:

Apparent changes at mail.com

I was poking around at some DNS this weekend and happened to do a MX lookup for mail.com and noticed something changed. Previously mail.com mail was handled by Outblaze (now owned by IBM). It seems, though, that mail.com is now outsourcing their mail delivery to AOL.

Marketing to businesses

“If you do stupid things, you’re going to get blocked,” says Jigsaw CEO Jim Fowler in an interview with Ken Magill earlier this week.
Jigsaw is a company that rewards members to input their valuable business contacts. Once the addresses are input into Jigsaw, they are sold to anyone who wants them. Jigsaw gets the money, the people providing information get… something, the people who provided business cards to Jigsaw members get spammed and the people who downloaded the lists get to deal with a delivery mess. Sounds like a lose for everyone but Jigsaw.
Except that now Jigsaw is listed on the SBL for spam support services. Well, that’s going to cause some business challenges, particularly given how many companies use the SBL as part of their filtering scheme.
It’s hard to think of a situation where I would appreciate someone I gave a business card to providing my information to a site that then turns around and lets anyone download it to send email to. I know, I know, there are a million companies out there I’ve never heard of that have The Product that will Solve All my Problems. But, really, I don’t want them in my work mailbox. The address I give out on my business cards is, for, y’know, people to contact me about what I’m selling or to contact me about things they’ve already purchased from me. That address is not for people to market to. I have other addresses for vendors, and even potential vendors, to contact me.
Jigsaw clearly facilitates spam to businesses by collecting email addresses and then selling them on. This is a drain on small businesses who now have inboxes full of valuable offers to wade through. Perhaps their stint on the SBL will make them reconsider their spam support services.
HT: Al

AOL changes bounce behaviour

A couple other bloggers have commented on the recent AOL blog post talking about changes to the MAILER-DAEMON string on bounce messages.

Blocked for phishing

A couple clients recently have had bounces from different places indicating that their mails were caught by the recipients’ anti-virus filter. These are some of my better clients sending out daily newsletters. They’ve been mailing for years and I know that they are not phishing. They asked me to investigate the bounce messages.
The information I had to work with was minimal. One bounce said:

Troubleshooting Yahoo delivery

Last week Jon left a comment on my post Following the Script. He gives a familiar story about how he’s having problems contacting Yahoo.

Yahoo fixed erroneous rejection problem

Yahoo announced over the weekend that they fixed their rejection problem. It may take some time to filter out to all their MTAs, but they do believe the issue is resolved.

Winning friends and removing blocks

I do a lot of negotiating with blocklists and ISPs on behalf of my clients and recently was dealing with two incidents. What made this so interesting to me was how differently the clients approached the negotiations.
In one case, a client had a spammer slip onto their system. As a result the client was added to the SBL. The client disconnected the customer, got their IP delisted from the SBL and all was good until the spammer managed to sweet talk the new abuse rep into turning his account back on. Predictably, he started spamming again and the SBL relisted the IP.
My client contacted me and asked me to intercede with Spamhaus. I received a detailed analysis of what happened, how it happened and how they were addressing the issue to prevent it happening in the future. I relayed the info to Spamhaus, the block was lifted and things are all back to normal.
Contrast that with another client dealing with widespread blocking due to a reputation problem. Their approach was to ask the blocking entity which clients they needed to disconnect in order to fix the problem. When the blocking entity responded, the customer disconnected the clients and considered the issue closed. They didn’t look at the underlying issues that caused the reputation problems, nor did they look at how they could prevent this in the future. They didn’t evaluate the customers they disconnected to identify where their processes failed.
The first client took responsibility for their problems, looked at the issues and resolved things without relying on Spamhaus to tell them how to fix things. Even though they had a problem, and is statistically going to have the occasional problem in the future, this interaction was very positive for them. Their reputation with the Spamhaus volunteers is improved because of their actions.
The second client didn’t do any of that. And the people they were dealing with at the blocking entity know it. Their reputation with the people behind the blocking entity was not improved by their actions.
These two clients are quite representative of what I’ve seen over the years. Some senders see blocking as a sign that somehow, somewhere there is a flaw in their process and a sign they need to figure out how to fix it. Others see blocking as an inconvenience. Their only involvement is finding out the minimum they need to do to get unblocked, doing it and then returning to business as usual. Unsurprisingly, the first type of client has a much better delivery rate than the second.

Contradictions

In the span of 48 hours the following two things happened.
Josh Baer posts over on deliverability.com about GoDaddy’s policies and recommends no email marketer use GoDaddy as a registrar because they are so hostile to email marketing that they charge customers for complaints. To quote Josh:

$234M default judgment against spammers

MySpace has won a 234 million dollar judgment against Walt Rines and Sanford Wallace.
“MySpace has zero tolerance for those who attempt to act illegally on our site,” [MySpace Chief Privacy officer] Nigam said in a statement. “We remain committed to punishing those who violate the law and try to harm our members.”
These are two of the spammers responsible for me learning to read headers and report spam. Both of them have previous judgments against them. Wallace sued AOL to force AOL to accept his mail. Eventually the judge ruled against Cyber Promotions and Wallace.

Followup to EEC spamming

Ken has a followup to his article last week about the EEC spamming.

Email non-viable for acquisition

Chris Marriott over at iMediaConnection talks about all the reasons email is a non-starter as a replacement for direct mail. This is something I have been telling clients for a while now. Chris mentions a number of reasons for why email is not an acquisition tool.

Postini bug

Ben over at MailChimp has an article talking about a recent experience with Postini and an actual bug that causes Postini to interact badly with another spamfilter and block non-spam.

More on spamfiltering feedback

Al wrote a post commenting on my post from last Thursday on spamfilters talking to senders who are being filtered. I think his take on it is close to mine. I would point out that Google has a pretty opaque system and no feedback to senders, but a lot of people seem to think their filters are accurate and do a good job.
Overall, I think there is room for discussion and feedback between senders and recipients, but on both sides the goal needs to be improving the enduser experience.

Sender complaints about spamfiltering

JD posed a question in my post about Postini and trying to sort out a customer getting marked as spam by their filtering mechanism and I think it bears more discussion than can be done in comments.

Troubleshooting a Postini block

Mail from one of my clients is being filtered at Postini and they asked me to look into this. Not that there is anything that can be done, of course. Even before they were bought out by Google, they were the poster child for a spam filtering company that believed they could do no wrong. It was difficult, if not impossible to get a straight answer from Postini about filtering, and the only statement they would ever make in regards to blocking problems was ‘have the recipient whitelist your mail.’
It is not just that Postini will not talk with people who are blocked, they will not talk to their own customers, either. Many years ago, I was dealing with another Postini issue for a customer. This customer was a Postini customer and was sending mail to themselves to test their new ESP. Postini was blocking the mail and the customer wanted me to find out why. After a couple days of digging I did actually find a really-o truly-o human at Postini. [1] He explained to me that a single line of text, followed by an unsubscribe link was spam, always spam and nothing but spam. He also explained that the only way for that mail to be let through, was for my customer to turn off his Postini filters.
Fast forward 4 years and I once again have a customer blocked by Postini. Usually, I tell customers there is nothing to be done for Postini blocks and that no one can find any information about them, but this customer is insistent. This particular customer has extremely clean mailing practices, sends highly relevant and wanted mail and consistently gets 95+% inbox delivery. They are not spammers, not even a little bit. Because I know this customer is so clean, I poked around a little to find some information about them. They do use the ReturnPath Mailbox Monitor so I have a copy of the headers Postini is adding. I also discovered that Postini is now providing a decoder service for their headers at https://www.postini.com/support/header_analyzer.php
The response you get back from pasting in a header is not that useful if you have found any of the numerous explanations of Postini headers, but it does show some willing. Note, there is no way to ask a question or provide feedback to Postini on the listing.
There is not much that can be done to deal with Postini filtering your email. The best you can do is have your recipients whitelist you.
[1] I believe I am the only person on the delivery end that has ever been able to actually talk to a live human at Postini, and I think that is only because I called them from the same area code they are in and some engineer decided to return the message I left on their corporate voicemail.

Forgery and spamware

Recently there has been a massive uptick in forgeries. I have been seeing hundreds of bounce back messages, peaking at more than 1000 in an hour. I have been talking about this with people who monitor large spamtrap feeds, large MTAs and spamfilters and it seems this is not an isolated experience. The consensus seems to be that there is new spamware out there which is using email addresses on the spam list as a From: address
The volume itself is annoying. Thousands of messages a day from “mailer-daemon” telling me that the mail I sent with the subject line “Get a longer tool” cannot be delivered to some random address some where. These are coming to at least 3 separate email addresses. One of them was given to Intuit back in 2001/2002 when I registered a copy of Quicken, and ended up leaked to loan spammers and is all over spam lists. The other two are addresses scraped from websites. Same spammer has them, same spammer is using them as part of his spam run.
Even more annoying than the volume, though, is the challenge/response emails. “Your email to jobobjimbo@example.com cannot be delivered until you click this link.” I have been adding every domain I can find that is using c/r to my filters, and just discarding the c/r emails so I do not have to deal with them. That is not my ideal solution, it does mean that if someone using c/r ever tries to contact me I will not see the challenge and our communications cannot happen.
Some people have recommended that the right way to deal with challenges from forged spam are actually to answer the challenges. As the reasoning goes, if someone using c/r is going to outsource their spam filtering to a victim of spam forgery, then they should expect that the “spam filter” may have a different opinion than they do. While I always sympathized with this viewpoint, I was not sure I would ever confirm spam forgeries. The sheer volume of c/r stuff I have received in the last few weeks has almost convinced me that people who use c/r deserve every bit of spam they get. If a c/r filter lets in spam, then perhaps they will reconsider their choice to spew challenges out to forged email addresses.
The amount of c/r spam I am getting as part of the forgery runs is decreasing, I think I have finally managed to block the primary sources. It does mean I will not be able to communicate with people who use c/r in the future, but I find this a small price to pay for not having to be an outsourced spam filter. I get enough of my own spam, I really do not want to have to deal with yours.

Dealing with ISPs when you are blocked

Here is some advice on dealing with ISPs over a blocking issue.

Affiliates: what is a company's responsibility

Many of my clients come to me when they end up with delivery problems due to the actions of affiliates. These can either be listings in some of the URL blocklists (either public or private) or escalations of IP based listings. In many of the cases I have dealt with affiliates, the affiliates have sloppy mailing practices or are out and out spammers.
Recently the FTC settled with Cyberheat over their liability for the behaviour of their affiliates. In this settlement Cyberheat is required to monitor their affiliates as follows:

Yahoo delays, part 3: Yahoo speaks

Yahoo is aware of the recent problems and have been working feverishly to fix them. A Yahoo employee posted to a mailing list earlier today, explaining some of the recent issues. The summary is:
1) The Yahoo delays are a result of a tighter spam filtering policy. The delays are the result of the system erroneously recognizing email as spam and deferring delivery. They do believe that retrying long enough will result in all mail being delivered to Yahoo recipients.
2) They have been continually making fixes to the system over the last few days and senders should see queues start to empty over the next few hours.
3) They believe the adjustments made will resolve the deferral problems. If you continue to see problems, you can contact them through the form at http://postmaster.yahoo.com/.
4) They are working to provide more self-serve information at http://postmaster.yahoo.com/ as well as timely service updates.
Loose ends from my previous Yahoo posts:

What really is "spam" anyway?

A few days ago I was reading the attempt by e360 and Dave Linhardt to force Comcast to accept his mail and to stop people posting in the newsgroup news.admin.net-abuse.email from claiming he is a spammer. The bit that pops out at me in this complaint of his, is the fact that he believes that by complying with the minimal standards of the CAN-SPAM act, he is not spamming.
The problem with this claim is that CAN SPAM lists the minimal standards an email must meet in order to avoid prosecution. CAN SPAM does not define what is spam, it only defines the things senders must do in order to not be violating the act. There is no legal definition of spam or of what is not spam.
To add to the confusion there are a number of confusing and contradictory definitions of spam. Definitions people have used over the years include:

Why do ISPs limit emails per connection?

A few years ago it was “common knowledge” that if you were sending large amounts of email to an ISP the most polite way to do that, the way that would put the least load on the receiving mailserver, was to open a single SMTP session to the mailserver and then to send all the mail for that ISP down that single connection.
That’s because the receiving mailserver is concerned about two main resources when handling inbound email – the pool of “slots” assigned one per inbound SMTP session, and the bandwidth (network and disk, and related resouces such as memory and CPU) consumed by the inbound mail – and this approach means the sender only uses one slot, and it allows the receiving mailserver to control the bandwidth used simply by accepting data on that one connection at a given rate. It also amortizes all the connection setup costs over multiple emails. It’s a beautiful thing – it just doesn’t get any more efficient than that.
That seems perfect for the receiving ISP – but ISPs don’t encourage bulk senders to do this. Instead many of them have been moving from “one connection, lots of mail through it” to “multiple connections, a few messages through each”. They’re even limiting the number of deliveries permitted over a single connection. Why would that be?
The reason for this is driven by three things. One is that the number of simultaneous inbound SMTP sessions that a mailserver can handle is quite tightly limited by the architecture of most mailservers. Another is that the amount of mail that’s being sent to large ISP mailservers keeps going up and up – so there are sometimes more inbound SMTP sessions asking for access than the mailserver can handle. The third is that ISPs know that there are different categories of email being sent to their users – 1:1 mail from their friends that they want to see as soon as possible, wanted bulk mail that their users want to see when it arrives and spam; lots and lots of spam.
So ISPs want to be able to do things like accept 1:1 mail all the time, while deferring bulk mail and spam to allow them to shed traffic at times of peak load. But they can only make decisions about whether to accept or defer delivery in an efficient way at SMTP connection time – they pick and choose amongst the horde of inbound connection attempts to prioritize some and defer others, letting them keep within the number of inbound sessions that they can handle simultaneously.
But once the ISP lets a bulk mailer connect to deliver their mail, they lose most of the ability to further control that delivery as the sender might send thousands of emails down that connection. (Even if the ISP has the ability to throttle bandwidth – as some do to control obvious spam – that just means that the sender would tie up an expensive inbound delivery slot for longer).
So, in order to allow them to prioritize inbound connections effectively the ISP needs to terminate the session after a few deliveries, and then make that sender start competing with other senders for a connection again.
So ISPs aren’t limiting the number of deliveries per SMTP connection to make things difficult for senders, or because they don’t understand how mail works. They’re doing it because that lets them prioritize wanted email to their users. The same is true when they defer your mail with a 4xx response.
It might be annoying to have to deal with these limits on delivery, but for legitimate bulk mail senders all this throttling and prioritization is a good thing. Your mail may be given less priority than 1:1 mail – but, if you maintain a good reputation, you’re given higher priority than all the spam, higher priority than all the email borne viruses, higher priority than all the junk email, higher priority than the 419 spams. And higher priority than mail from those of your competitors who have a worse reputation than yours.

Yahoo and Spamhaus

Yahoo has updated and modified their postmaster pages. They have also put a lot of work into clarifying their response codes. The changes should help senders identify and troubleshoot problems without relying on individual help from Yahoo.
There is one major change that deserves its own discussion. Yahoo is now using the SBL, XBL and PBL to block connections from listed IP addresses. These are public blocklists run by Spamhaus. Each of them targets a different type of spam source.
The SBL is the blocklist that addresses fixed spam sources. To get listed on the SBL, a sender is sending email to people who have never requested it. Typically, this involves email sent to an address that has not opted in to the email. These addresses, known as spamtraps, are used as sentinel addresses. Any mail sent to them is, by definition, not opt-in. These addresses are never signed up to any email address lists by the person who owns the email address. Spamtraps can get onto a mailing list in a number of different ways, but none of them involve the owner of the address giving the sender permission to email them.
Additionally, the SBL will list spam gangs and spam supporters. Spam supporters include networks that provide services to spammers and do not take prompt action to remove the spammers from their services.
The XBL is a list of IP addresses which appear to be infected with trojans or spamware or can be used by hackers to send spam (open proxies or open relays). This list includes both the CBL and the NJABL open proxy list. The CBL list machines which appear to be infected with spamware or trojans. The CBL works passively, looking only at those machines which actively make connections to CBL detectors. NJABL lists machines that are open proxies and open relays.
The Policy Block List (PBL) is Spamhaus’ newest list. Spamhaus describes this list as

Why does everyone tell you to avoid .biz in your emails?

… or Why do spam filters sometimes have some very strange ideas?
It’s been dogma for a long time that if you’re doing email marketing you should avoid using a .biz domain in your mails. Even if your main website was in .biz, you should use something different in your messages, perhaps a website you buy solely for use in email that redirects to your real .biz website. Last year I looked at why that was, and what could be done about it.
One main reason for avoiding it has been resolved (so if you’ve been avoiding using .biz URLs in your mail now might be a good time to re-test that decision). And enough time has gone by that I can share the ugly reasons as to why .biz was considered a sure sign of spam without good reason for so long without upsetting everyone.
The simple reason was SpamAssassin. SpamAssassin is very widely used to filter mail, both in it’s open source version and buried anonymously deep inside countless commercial spam filters and filtering appliances. Not only that, but SpamAssassin is readily available, so most people looking to do pre-mailing content checks or looking at why content-based filters are objecting to a particular email will use SpamAssassin as their model. It’s very widely deployed, and influential far beyond the size of it’s deployed base.
SpamAssassin is a score-based spam filter – it checks an email against hundreds of rules, adds up the scores of each rule that matches and, in typical setups, decides the mail is spam if the total score is five or more. Pretty reasonable, but here are a few of the rules and scores (from the 2006 version of SpamAssassin)

Greylisting: that which Yahoo does not do

Over the last couple days multiple people have asserted to me that Yahoo is greylisting mail. The fact that Yahoo itself asserts it is not using greylisting as a technique to control mail seems to have no effect on the number of people who believe that Yahoo is greylisting.
Deeply held beliefs by many senders aside, Yahoo is not greylisting. Yahoo is using temporary failures (4xx) as a way to defer and control mail coming into their servers and their users.
I think much of the problem is that the definition of greylisting is not well understood by the people using the term. Greylisting generally refers to a process of refusing email with a 4xx response the first time delivery is attempted and accepting the email at the second delivery attempt. There are a number of ways to greylist, per message, per IP or per from address. The defining feature of greylisting is that the receiving MTA keeps track of the messages (IP or addresss) that it has rejected and allows the mail through the second time the mail is sent.
This technique for handling email is a direct response to some spamming software, particularly software that uses infected Windows machines to send email. The spam software will drop any email in response to a 4xx or 5xx response. Well designed software will retry any email receiving a 4xx response. By rejecting anything on the first attempt with a 4xx, the receiving ISPs can trivially block mail from spambots.
Where does this fit in with what Yahoo is doing? Yahoo is not keeping track of the mail it rejects and is not reliably allowing email through on the second attempt. There are a couple reasons why Yahoo is deferring mail.

Blacklisted on FiveTen: no big deal

Al posted an analysis on DNSBL Resource about the effectiveness of the FiveTen blacklist.
He says:

Blocklists and standards

I received a comment this morning on my post about e360 v. Spamhaus, which I think brings up a point that deserves a post of it’s own. Skinny says:

More on Truthout

Ken Magill comments on the reaction of truthout.org to being blocked by AOL and Hotmail.
I do agree with Al, if both AOL and Hotmail are blocking your email, then you’re doing something wrong.

They’re not blocking you because they hate you.

Really. They’re blocking you because you’re doing something that is triggering their blocking mechanisms.
This has happened over and over and over again. Some political or activist website sends out an email that gets blocked by some large ISP and the political site turns it into a giant crisis that means the ISP hates them or is trying to shut them up or is trying to silence their message.
Except that’s not what is going on. The folks at the large ISPs who handle blocking and incoming mail are incredibly smart and conscientious . They take their jobs seriously. They, both personally and corporately, want their customers (the end recipients) to receive the email they want. Additionally, they do not want to deliver mail that the recipients did not ask to receive.
In almost no cases is the block a particular activist site encounters a result of the ISP not liking the content of the email. If an activist site is being blocked it’s due to complaints or reputation or something that ISPs measure and block on. Some person at the ISP didn’t read your email, decide they didn’t like what you had to say and then block that email. That email was blocked because something related to that email triggered the thresholds for blocking.
Of course, as with everything online, there are caveats. In this case it’s that the above statements really only hold true for large ISPs in free countries. There are some countries in the world that do block email based on content, and that is dictated by the government. Likewise, some small ISPs will block based on the guy in charge not liking the email.
Generally, though, if an activist site is being blocked by a large ISP in the US or other free countries it is because their mailings are somehow not complying with that ISPs standards. Instead of starting an email campaign or blog campaign to shame the ISP for suppressing speech, it is much more productive to actually contact the ISP in question and find out what went wrong.