URL shorteners, like bit.ly, moby.to and tinyurl.com, do three things:
- Make a URL shorter
- Track clicks on the URL
- Hide the destination URL
Making URLs shorter was their original role, and it’s why they’re so common in media where the raw URL is visible to the recipient – instant messaging, twitter and other microblogs, and in plain text email where the “real” URL won’t fit on a single line.
From the moment they were invented they’ve been used to trick people to click on links to pages they’d rather not visit, from musical classics to less tasteful content. And, in just the same way, spammers quickly found that they were a good way to avoid content-based filters or to hide a suspicious looking target URL.
Inevitably, URL shorteners that are persistently abused by spammers (especially those where that’s done with the support of the URL shortener operator) start to be seen as a sign of spam, and email that uses them will be treated with suspicion by content-based spam filters and often sent to the spam folder.
bit.ly is probably the highest profile URL shortener, so it’s the one you’ll most likely see people trying to use in email. What effects does that have?
Now being “totally owned” by the Canadian Pharmacy gang, thousands of URLs being spammed with very slow takedowns. Not good.SpamHaus on bit.ly
bit.ly have been on SpamHaus’s radar for quite a while. They’re listed on the SBL multiple times. They’re listed in the DBL – SpamHaus’s newish domain based blacklist, intended for content-based filtering of email. All this means that emails that contain bit.ly URLs are increasingly likely to have serious delivery problems.
This isn’t unique to bit.ly: many other URL shorteners have similar problems – j.mp, su.pr, and others. Nor is it unique to SpamHaus: many other spam filters, public and private, are starting to treat common URL shorteners with suspicion.
Naive use of URL shorteners in your email will send it to the spam folder.
More about why you shouldn’t do that – and what you can do instead – tomorrow.
RSS - Posts
If I send a legitimate email that the recipient wants to receive and it is blocked by a spam filter I think the spam filter is in the wrong. Now I understand you are giving advise to senders and suggesting they avoid shorteners in their emails is probably a good idea but people do — they copy links from Twitter/Facebook etc. or they just want to make long URLs look pretty — and the anti-spam community should deal with that.
I don’t have enough evidence to judge whether bit.ly is “totally owned by spammers”. I guess in email it’s mostly used for bad purposes; on Twitter the situation may be totally different though. I also bet they could (and thus should) be doing more to prevent their service from being abused though (although the few times I complained to them on Twitter they responded promptly – on a Saturday). I also think it would help if they made it easier for spam-filters to check the ‘real content’ of their URLs — e.g. by offering it via DNS. But even now it is not very difficult for a spam filter to obtain that hidden content and use that in its decision on whether the email is spam.
Btw, you are right bit.ly is listed on the DBL but it currently gives a 127.0.1.3 response which means it is a “spammed redirector domain”.
(Also, j.mp is merely an alias for bit.ly)
Gah! I was going to publish a post about this later this week, and you beat me to the punch. (… he says as he presses “publish” and pastes the link into the URL box below.)
Weirdly, Terry Zink just posted a “counterpoint” post in response to your post, Steve. IMHO you are right and he is not.
It may be that Steve is talking from a sender’s point of view (better not do this as it might get you blocked) and Terry from a recipient’s/filter’s (we’d be stupid if we’d block this). So they may both be kind of right.
But I have yet to see evidence that confirms the bold statement of this post’s title.
I do have examples of legitimate email sent this week containing a bit.ly link that was not blocked by all of two dozens of spam-filters I checked. Because bit.ly-links don’t occur in most legitimate emails I don’t have anything like a representative sample to show whether a bit.ly-link makes it more likely for the email to be blocked.
I should be able to draw some conclusions on whether a bit.ly link increases the likelyhood of a spam message to be blocked – though that will take a little more time to generate. Let me know if there is an interest.
@Martijn Terry is responding to the post title, rather than the contents of the post, I think. “Increasingly likely to have serious delivery problems” is a very different, and more nuanced, observation than “decides to block messages with links to bit.ly”.
The post title isn’t speculation, though – it’s based on the empirical experiments of several people who’ve found that some ISPs are sending quite a lot of email using bit.ly links to the spam folder. Not all of it, not mail from regular correspondents and so on. But quite a lot of it. Sending roughly the same email to roughly the same recipients, mail using bit.ly was tagged as spam, mail that wasn’t, wasn’t.
Big, competent consumer ISPs tend to respond a lot faster to changes in mail patterns than commercial spam filters – they have more immediate access to data, and a more homogeneous recipient base. Commercial filters, in turn, tend to lead open source projects somewhat (for a bunch of reasons, not least of which is deployment update frequency). So I’m not surprised you’re not seeing anything obvious in the spam filters you have handy. Yet.
More tomorrow.
Thanks, Steve. That makes sense. The few (legitimate) emails I checked were (more or less) personal emails, not newsletters. The situation may be different there – and I have no reason to believe it isn’t.
Having looked at my logs, I am actually surprised how little spammers make use of bit.ly. With a few exceptions, all I’ve seen using bit.ly in the past ten days is a German language jewelry spam campaign. So perhaps bit.ly _is_ making life difficult for spammers.
I thought bitly’s hmason had recently patched things up with Spamhaus? I guess not.
In an even more obscure connection to a link shortener, a client recently experienced deliverability problems. Their website site is hosted in shared network space at Heroku, that also hosts a link shortener that is listed on Spamhaus. When their emails linked to their website, and the same IP as the link shortener, Gmail blocked the email and referenced the Spamhaus listing. The client is now having to move their website off Heroku.
http://www.spamhaus.org/sbl/sbl.lasso?query=SBL107994
http://www.spamhaus.org/sbl/sbl.lasso?query=SBL107992
http://www.spamhaus.org/sbl/sbl.lasso?query=SBL107993
[…] to the wise: Bit.ly gets you blocked Risky.biz: Groupon leaks entire Indian user […]
Dumb question: do plain links no longer work? Or is it just as suspicious?
@Brian We saw the same thing in January – http://blog.wordtothewise.com/2011/01/why-is-shared-hosting-like-phishing/
[…] behaviour by other senders with the same advertisers can cause your email to be spam foldered. And, as we discussed yesterday, if spammers use the same URL shortener you do, that can cause your mail to be marked as […]
I don’t see a legitimate reason to use bit.lys in email anyway, bit.ly were solely created to shorten urls if a url in an email is going to be clicked on and not manually typed why use.
Now I’ve seen it used for tracking purposes, if that is the case purchase a good analytics package and you won’t have to worry about this.
@Michael: I don’t think it’s up to ‘us’ to say what people should or should not do. People use shorteners in emails so filters will have to deal with them.
Telling senders what is best to do – as Steve does ver well in this post’s follow-up – is a different matter of course.
HEY STEVE WHEN I TYPE IN MY BIT.LY LINK IT LEAVES A MESSAGE SAYING THAT THE LINK WILL BE SHORTEN. I DID NOT ASK FOR THIS. IS THIS DOING ME MORE HARM THAN GOOD?
I use bitly URL Shorten regularly on Facebook 420 all character Wall Share Posts. I was using twitter but it is a sure thing to get a shorten by posting the long URL….sometimes over 140 characters. I am unaware of any blocking on Facebook. They do not like activated URL’s in general.
Have I missed something here, I use a url shortrner not for aesthetic purposes but for protection, hiding my affiliate links and thus stopping people hijacking them. Is this not legitimate. Also is there a better answer?
No, trying to use bit.ly to hide your affiliate links isn’t legitimate if you’re trying to hide the fact that the links you’re pitching are paying you to do so. That’s not a sign of someone acting in good faith.
It also doesn’t work for a bunch of reasons – one being that anyone can add a “+” to the end of the bit.ly link and get a bunch of information about it, including the target URL. The better answer is to be honest about the fact that you’re part of an affiliate programme.
Yes, sometime genuine shortened URL also gets blocked. But it seems that issue has been resolved. take a look here http://www.spamhaus.org/sbl/query/SBL108937
Hi! Can anyone offer me some advice? I am getting inundated with emails tht I do not want that start with “tinyurl”. They are always about selling me something. When I hit the unsubscribe button, it takes me to another ad. In the fine print it says that I can mail them ( regular mail) at some address. Don’t they have to have a working unsubscribe button? I can’t afford the time or the number of stamps to undo these turds. Please help!