Reputation as measured by the ISPs

Part 3 in an ongoing series on campaign stats and measurements. In this installment, I will look a little closer at what other people are measuring about your email and how that affects your reputation at the ISPs.
Part 1: Campaign Stats and Measurements
Part 2: Measuring Open Rate
Reputation at the ISPs is an overall measure of how responsive recipients are to your email. ISPs also look at how much valid email you are sending. Anything the ISP can measure and use to distinguish good mail from bad is used in calculating reputation.
Some of the major metrics ISPs use include the following.
Invalid Address Rates
The ISPs count how much mail from any particular IP address is hitting non-existent addresses. If you are mailing a large number of email addresses that do not exist (550 user unknown), this is a suggestion that your address collection techniques are not very good. Responsible mailers do have the occasional bad address, including typos, expired/abandoned addresses, but the percentage in comparison to the number of real email addresses is low. How low is low? Public numbers suggest problems start at 10% user unknowns, but conversations with ISP employees show they consider lower levels a hint there may be a problem.
To calculate bounce rate ISPs take the total number of addresses that were for invalid accounts and divide that by the total number of addresses that the sender attempted to send mail to. Rates above 10% may cause significant delivery issues on their own, rates lower that 10% may still contribute to poor delivery through poor reputation scores.
Spamtraps
ISPs pay a lot of attention to how much mail is hitting their “trap” or “bait” accounts. There are a number of different sources of these trap accounts: old abandoned email addresses, addresses that never existed or even role accounts. Hits to a trap account tells the ISP there are addresses on your list that did not opt-in to receive mail. And if there are some addresses they know about that did not opt-in, it is likely that there are other addresses that did not opt in.
Spamtraps tend to be treated as an absolute number, not as a percentage of emails. Even a single spamtrap on a list can significantly harm delivery. According to the ReturnPath Benchmark report lists with a single spamtrap had nearly 20% worse delivery than lists without spamtraps.
This is spam clicks (FBL complaints)
Complaints from users are heavily used by ISPs. This tells them directly how many people are objecting to your email. In this case, permission is removed from the equation. Even if a sender has permission to send email, the recipient can say “no, I don’t want this, it is spam.” The ISPs put more weight on what their users tell them than on what the senders tell them.

The customer is always right. In my opinion, there is no such thing as ‘overuse’ of the report spam button. The more feedback we get, the better. Our job is to keep the user’s inbox in the state they want it. The more they tell us what they do and don’t want, the clearer picture we get about who is sending unwanted mail. So I would say, yes, it does affect my ability to do my job in that it enables me to actually do my job.
It might cause my job to involve more detailed research into people’s preferences and what to do with mail that people disagree about, but I don’t see that as a problem.
Just because a marketer doesn’t like that we consider our users’ opinions to be more important than theirs is not really a problem either as far as I’m concerned. I’m here to serve my users, not them. They can either send mail that people don’t respond negatively to, or I can put their mail in the spamfolder. It’s not like they are going to make any money by repeatedly mailing people who think their mail is spam anyway.

In many ways relying on the users to provide feedback is a good thing. The ISP gets a direct measure of what the recipients think without having to filter through a lot of obfuscation from bad senders.
Complaint percentages are measured by taking the number of “this is spam” clicks divided by the number of emails delivered to the inbox. Percentages under 0.3% usually result in reasonably good delivery, depending on other metrics. Percentages higher than 1% usually result in poor delivery, even if other metrics are good.
“This is not spam“ clicks
This measures how many people tracked down wanted mail delivered to the spam folder and tell the ISP the mail is not spam. These clicks are vital in reputation scores. Senders who are having intermittent bulk foldering are most affected by these types of stats. If your recipients don’t care enough about your mail to go into the bulk folder and find it, then the ISP believes that the mail is not necessarily wanted.
ISPs calculate the ratio’s differently and there is not a standard formula for how valuable a this is not spam click is.
The secret sauce
This is what distinguishes one ISPs filtering process from another. Each ISP and each spam filtering company has their own secret sauce. The things above are things that the ISPs have confirmed to me that they measure. I believe they also measure other things, including recipient profiles, recipient clicks, probably some stuff they won’t ever admit outside a development meeting. The secret sauce is also how they weight the different factors. ie, a this is spam click is not weighted the same as a this is not spam click. How important are complaints versus mail sent to dead addresses? How vital are spam traps? Some ISPs probably even have trusted reporter setups where people with good histories of accurately reporting spam have their reports weighted more heavily than unknown people.
The good news is, with the exception of the secret sauce, all of these factors are under control of the sender and senders can make changes to their mailing programs that will improve reputations and delivery.

Related Posts

Measuring open rate

In this part of my series on Campaign Stats and Measurements I will be examining open rates, how they are used, where they fail and how the can be effectively used.
There has been an lot written about open rates recently, but there are two posts that stand out to me. One was the EEC’s post on renaming open rate to render rate and Mark Brownlow’s excellent post on what open rate does and does not measure. I’ve also weighed in on the subject.
Overall, I find open rates to be a very frustrating metric. Some senders, particularly those relatively new to email marketing, are so sure they know what open rate is and what it means, that they don’t take any time to actually understand the number. While the name “open rate” seems self explanatory, it’s actually not. Open rate is actually not a measure of how many recipients open an email. However, there are times where open rate is a useful metric for measuring a marketing program over time.
What is an open?
If asked, most people will tell you that open rate is the number of emails that were opened by the recipients. The problem is that this isn’t actually true. An open is counted when a tagged image in an email is rendered by the recipient’s email client. Not all mail clients render images by default, but the emails are still available for the recipient to read. If a user clicks on a link in an email that has not had an image rendered, some ESPs count that as an open as well as a click. In other cases, visiting a link in an email with no image rendered is just a click, no open is recorded.
What is the open rate?
Open rate is generally the percentage of email opens divided by some number representing the number of emails sent. Many senders use the number of emails sent minus the number of bounced emails, others use just the number of emails sent without factoring in the number of emails bounced.
Open rate is a secondary metric. While it does not measure the success, or failure, of a campaign directly, it can be used as a indicator for campaigns. Many people use open rate as a metric because it’s easy to measure. Direct metrics, such as clicks or average purchase or total purchase, may take days or even weeks to collect and analyze. Open rates can be calculated quickly and easily.
What the open rate isn’t
Open rate is not a measure of how many people opened a mail. It is not a measure of how many people read a mail. It really only records that an image in a particular email is loaded and, sometimes, that a link was clicked on. Open rates can be wildly different depending on how the sender measures opens and how the sender measures sends.
What senders use open rates for
To compare their open rates with industry averages
As I talked about above, this use of open rates is problematic at best. You cannot compare numbers, even when they have the same name, if the numbers were arrived at using different calculations. Open rate is not open rate and unless you know the underlying algorithm used you cannot compare two open rates. This is a poor use of open rate.
As a metric for advertising rates
Since a sender can manipulate the open rate by using different calculation methods, this is a good metric for the advertiser to use. It is not so great for the purchaser though, who is at the mercy of the sender’s metrics. There are contractual ways a purchaser can protect herself from an unscrupulous marketer, but only if she understands how open rate can be manipulated and takes steps to define what open rate is in use.
To judge the success of campaigns over time
A single open data point doesn’t mean very much, however, using consistently measured open rates a sender can measure trends. Open trends over time are one area that open rates can help senders judge the success, or failure, of a marketing campaign.
As one metric in A/B testing
Comparing open rates in A/B testing gives some indication of which campaigns recipients may be more interested in. As with trends over time, the lone measurement isn’t useful, but as a comparative metric, it may provide senders with insight into a particular mailing.
To judge the engagement of recipients
Over the long term, recipients who do not interact with a mailing become dead weight on the list. Too many non responders can hurt a sender’s reputation at an ISP. List hygiene, in the form of removing people who never open or click on an email, is an important part of reputation management.
As metrics for email campaigns go, open rate is limited in what it measures about an email campaign. However, as a quick way to measure trending or do head to head comparisons it is a useful metric.

Read More

TWSD: Lying and Hiding

Another installment in my ongoing series: That’s What Spammers Do. In today’s installment we take a look at a company deceiving recipients and hiding their real identity.
One of my disposable addresses has been getting heavily spammed from mylife.com. The subject lines are not just deceptive, they are provably lies. The mail is coming from random domains like urlprotect.com or choosefrequency.com or winnernotice.com advertising links at safetyurl.com or childsafeblogging.com or usakidprotect.com.
The spam all claims someone is “searching for…” at their website. The only thing is, the email address is associated with a fake name I gave while testing a website on behalf of a client. I know what website received the data and I know what other data was provided during the signup process. I also know that the privacy policy at the time said that my data would not be shared and that only the company I gave the information to would be sending me email.
Just more proof that privacy policies aren’t worth the paper they’re written on. But that’s not my real issue here.
The real issue is that I am receiving mail that is clearly deceptive. The subject lines of the emails up until yesterday were “(1) New Message – Someone Searching for You, Find Out…” Yesterday, I actually clicked through one of the messages to confirm that the emails were ending up at mylife.com. After that, the subject lines of the emails changed to “(1) New Person is Searching for You.”  I don’t know for sure that my click has caused the change in subject lines, but the timing seems a bit coincidental.
It’s not that someone, somewhere gave mylife.com bad data, or that someone typed a name into the mylife.com search engine and the mylife.com database showed that name and my email address were the same. Neither this name or this email address show up in a google search and I can say with certainty that this is a unique address and name combination given to a specific website. Therefore, the subject lines are clearly and demonstrably lies.
The spams are also coming from different domains and advertising links in different domains. The content is identical, the CAN SPAM addresses are identical. While the court may not rule this is deceptive under the rules of CAN SPAM, it certainly is an attempt to avoid domain level spam filters.
Who are mylife.com? Well, their website and the CAN SPAM address on their spam claims they are the company formerly known as reunion.com. I’ve talked about reunion.com here before. They have a history of harvesting addresses from users address books. They were sued for deceptive email practices under California law, but won the case just recently. They seem to think that the court case was permission to send deceptive email and have thus ramped up their deceptive practices.
If you are a legitimate email marketer, there are a couple take home messages here.
1) Spammers send mail with different domains, from different IP addresses, that contain identical content, landing pages and CAN SPAM addresses. Legitimate marketers should not rotate content and sends through different domains or different IP addresses. Pick your domain, pick your IP and stick with it.
1a) Spammers use randomly chosen domain names and cycle through domains frequently. Legitimate marketers must not use unrelated domains in marketing. Use a domain name that relates to your product, your industry or you.
2) Spammers send mail with deceptive subject lines. Legitimate marketers should make sure their subject lines are clear and truthful.
3) Spammers send mail in violation of the privacy policy under which information was collected. Legitimate marketers should be very careful to handle data in accordance with their privacy policies.
That’s what spammers do. Is that what you do?

Read More

Building a list for the long term

Mark Brownlow asks 2 key questions senders should be thinking about for their list building strategy for 2009.

Read More