Automated link checking getting more sophisticated

As the volume and severity of malicious email increases, filters are increasingly following links in emails. This is really nothing new. Barracuda and other filters have been inspecting links automatically for years. From what I’ve seen there does seem to be some level of risk analysis based on domain reputation. That makes sense, not only is following links computationally expensive, it can also delay mail receipt.

Earlier this week I was on a call with a client who mentioned they were seeing opens and clicks on messages before those messages were reported as delivered. There were multiple situations where they were seeing this behaviour. In two of the cases the clicks and opens were around the same time as or slightly before the receiving MTA sent a 250OK. The other case was much more interesting. In this case, the opens and clicks happened a few hours before the mail was accepted by the receiving server.

Wait, what? My first thought was the receiving server was holding the connection open for hours while it waited. But that doesn’t make sense, they’re not going to hold open the connections for that long. The challenge here is that client is using a SMTP provider and we don’t have a good way to get the full logs through the API. If the client had log access we could do a bunch of data work and see what exactly is going on.

In the absence of logs what can we do to understand the process here?

Facts:

  1. Links in the mail are being followed before the mail has been accepted for delivery.
  2. Sometimes there are hours between link activity and delivery.
  3. Recipient mail servers have access to the entire email before they decide whether or not to accept it.
  4. Mail servers can be configured to answer anything, including giving a 4xy or 5xy response when accepting the mail.

What if a filtering appliance is examining the message during the SMTP transaction and deciding it doesn’t have enough information to make a yes/no decision. The receiving server defers the message, saying come back later. The sending server comes back at the normal retry interval of 2 – 3 hours. Now, there is enough data, so the message is accepted and delivered to the recipient. Thus the recipient finally sees the message, a few hours after the links have been followed.

There are other possibilities, but this one fits with the established facts and the behaviour we’re seeing. Yes, it is possible for opens and clicks to happen before mail is delivered. But these opens and clicks are not from the user and should not be counted as user engagement.

 

Related Posts

Reputation is in the eye of the beholder

A few years ago reputation was generally recognised as one thing. If a sending reputation or IP reputation was good in one place it was likely good in other places. Different entities mostly reputation using the same set of signals albeit slightly tweaked to meet their own needs. More recently there is a divergence in how reputation is measured, meaning delivery can be vastly different across entities.

Read More

Send Actual SMTP

It’s rare I find mail that violates the SMTP spec (rfc5321 and rfc5322). I’ve even considered removing “send mail from a correctly configured mail server” from my standard Best Practices litany.

Read More

Spamhaus rising?

Ken has a good article talking about how many ESPs have tightened their standards recently and are really hounding their customers to stop sending mail recipients don’t want and don’t like. Ken credits much of this change to Spamhaus and their new tools.

Read More