Friday, February 1, 2008

Reject spam from zombies

Image courtesy of www.gdata.de

(image courtesy of www.gdata.de)
The best thing that could happen to reduce email (spam) backscatter (a.k.a. collateral spammage) on the Internet today would be to have all mail servers configured so that they would reject emails from zombie PCs.

So, I have to wonder. Why don't mail server admins use these block lists and reject spam from zombies? All the information about configuring various mail servers to reject mail from IPs on block lists can be found on the SpamCop FAQ web site.

Let's break it down in to two parts: 1) Rejecting email and 2) Identifying Zombies

Rejecting email

Rejecting email is not a new thing. It has always been part of the original RFC821 that allowed for email transfers to "fail" during SMTP. The history of why it became fashionable to accept emails first, then validate and possibly bounce them afterward, is probably tied to Microsoft's domination of the Internet with its software. But I'm not enough of an expert to venture an explanation. I can only say that die-hard sendmail users (one of the original mail servers on the Internet) know what rejecting is, and how good it can be with respect to spam fighting.

But even Microsoft wised up and allows rejecting. Their latest mail server will allow emails to be rejected (see the bit about Spam Confidence Level) rather than bounced. The administrators only have to make sure they understand why rejecting is better and that they've enabled it.

Some companies do a disservice to their customers by not informing them of all of the risks of bouncing. For example, Barracuda Networks provide a whitepaper on Email Non Delivery Receipts (bounces). Yes, there is useful text explaining the difference between rejecting and bouncing, and the "best practices" for how to draft NDRs. However, Barracuda Networks fails to mention that most spams have forged "From:" addresses and that most of those well written NDRs will go to the users whose email addresses have been forged in the spams that get bounced! All in the spirit of making sure a "false positive" doesn't get mishandled!

Those backscatter victims (like me and possibly you if you found this blog) will not be happy to get a well written NDR that says a message we sent (that we didn't send!) was blocked by the Barracuda Spam Firewall. If anything, it implies that Barracuda are foolish enough to be tricked by the forged sender address. On the one hand, they are experts who deal with spam filtering. On the other hand, they "ignore" that spammers forge the "From:" address. Smells like marketing... Anyway, that's what a whitepaper is.

Identifying Zombies

Here's a fact: 100% of the backscatter I get today results from bounced spam sent originally from zombie PCs. How do I know this? The backscatter messages often contain the RFC822 "Received:" headers of the spam that had my email address forged in the "From:" field. From these headers, I can see that the injection point of the spam was a zombie.

How do I know it's a zombie? That's easy. The IP address of the "from" host on the last "Received:" line is on a DNSBL, such as bl.spamcop.net, cbl.abuseat.org or zen.spamhaus.org. The first two DNSBL are for "known" spam-sending IPs (they are dynamically updated), and the last one is for IP address that are located on ISPs that have a policy that no email should be sent by such IPs. That's right - a "home PC" should not be acting like a mail server (Mail Transfer Agent) outside of the local network.

You'll notice that I don't need any high-powered Bayesian filtering or keywords or TCP/IP probes to decide if it's a zombie is sending me spam. It's just basic common sense. Has the IP shown up as a known spam-sender (open relay) or does the IP have no business sending SMTP traffic? If the answer is yes (i.e., it's on one of those block lists), I conclude it's a zombie and reject the email (spam) it wants to send me.

Rejecting spam from zombies

Here's an example of a missed opportunity that makes backscatter victims suffer. Barracuda, who definitely can use block lists to reject spam from zombies, could have stressed in their whitepaper about NDRs that rejecting email is a good compromise with respect to false positives and backscatter. That is, when a message gets rejected (rather than bounced), there are two possibilities:
  1. It's a spam message: the rejection causes the spammer's mail server (usually a zombie) to have to deal with the bounce. This means that the zombie does nothing and moves onto the next spam to send. No collateral spamage (damage?).
  2. It's a legitimate message (false positive): the rejection causes the legitimate sender's mail server to generate an NDR with the reason for the rejection in the message. This NDR will likely be written in the language of the legitimate sender.
To conclude, if you're an administrator of a mail server (or Barracuda Spam Firewall), make the Internet a better place and reject mail from zombies! Please don't generate NDRs to forged sender email addresses like mine!

Thursday, December 27, 2007

Barracuda Networks and their customers could to do more to stop the backscatter



Here's my latest tally of backscatter received from Barracuda firewalls since 2007-09-16:
1940 messages
According to the press releases, Barracuda says 95% of email being sent is spam. According to most people's experiences, 100% of spams have forged "from" addresses. The success of their firewall product, and the continual increase in spam are probably the reasons for an increase in email backscatter. Sadly, too many Barracuda Spam Firewall customers still enable auto-replies for spams that get blocked.

When I get such backscatter, it's easy to fight back with an auto-reply of my own (thanks to Thunderbird's filters). Several Barracuda Spam Firewall customers have replied to me when I (automatically) contacted them about their firewalls creating backscatter.

Here are a few of those relatively rare, yet encouraging replies. I have left out the names of the individuals involved for privacy reasons:
date: Dec 21, 2007 6:02 AM
subject: RE: Please configure your spam firewall to stop bouncing spams to me

First of all, sorry for the inconvenience and we thank you your advice.

We have changed the wrong configuration parameter.

Please, let us know if you receive still bouncing spams in the next days.

Date: Dec 3, 2007 8:17 AM
Subject: RE: Please configure your spam firewall to stop bouncing spams to me (was: **Message you sent blocked by our bulk email filter**)

Sir –

Thank you for the information. I would have never known about this problem without your email. I have made the recommend changes on my Barracuda filter.

date: Nov 28, 2007 12:11 AM
subject: RE: Please configure your spam firewall

The suggested changes have been made. Thanks for the heads up.

date: Oct 31, 2007 1:48 PM
subject: RE: Please configure your spam firewall to stop bouncing spams to me (was: **Message you sent blocked by our bulk email filter**)

Sorry for the inconvenience, I disabled the feature

Thanks, there is enough crap going around, no use having it bounce around on top of that

date: Oct 29, 2007 7:22 PM
subject: RE: Please configure your spam firewall to stop bouncing spams to me (was: **Message you sent blocked by our bulk email filter**)

Please accept our apologies for any trouble caused by backscatter originating from our Barracuda. We have disabled the notifications that were causing the messages to be sent.

Thanks!
Next is an example of a depressing response, which shows an administrator who is clueless about the damage she is causing the rest of the users on the internet. Her suggestion is that I just block her bounces...
date Dec 7, 2007 7:31 AM
subject RE: Please configure your spam firewall to stop bouncing spams to me (was: **Message you sent blocked by our bulk email filter**)

Thanks for your email. We understand your frustration with receiving notifications of spoofed emails forged with your address. However, the notifications serve a purpose to alert you that: 1) someone is using your address to send spam; 2) alert you that you might be infected and are sending potentially infected emails.

If you feel that you are receiving too many false positives from our Barracuda, please feel free to add our domain to your blocking list.
Don't you like how she turned it around as doing us all a service! My response to this nonsense got escalated to the VP of IT in her company, who wrote me a message that was very defensive, to which I replied below:
Hello,

On Dec 7, 2007 9:20 AM, (Anonymized) wrote:
The next time you want to criticize someone, slap some credentials behind your name.
Who says credentials are necessary to say that your Barracuda is spamming me?!

I'm complaining about the spam your Barracuda is sending to me because you've enabled the feature that most people recognize as abusive. Just Google it! I won't be the only person who is upset about this. If you want credentials to back up what I'm saying, you're just being ignorant and not listening. Again, if you don't trust me, try Google:

http://www.google.ca/search?hl=en&q=barracuda+backscatter&btnG=Google+Search&meta=

I have sent many, many, many complaints to Barracuda owners about this problem. When I get a response, it's one of the following:

1) Thank you for pointing this out to us, we are correcting the problem.
2) Backscatter isn't my fault. Too bad for you.
3) postmaster does not exist.

Since you're a VP, I trust you know which one is the more professional and customer-oriented. Yes, I'm holding you to your credentials!
I assure you we have better things to do with our time than pick on someone like yourself, and send them bogus emails telling them that they might be infected. Gmail supports an outlook interface which is a commonly targeted service for spammers, have you considered the fact that someone, yes someone malicious may have in fact cracked your password and might be using your account.
This is a possible explanation, but there are no facts to support my Gmail has been hacked.

I have already more than 30,000 backscatter emails, and I found out how it works. I am not the only one who's a victim of this kind of spamming problem. If you check your Barracuda logs, I'm willing to bet you'll find it's bouncing spams to other people.
Funny, this "VP" never wrote back. Perhaps he's still trying to figure out how to turn off the auto-reply feature of their Barracuda Spam Firewall - maybe he's asking his underlings what a log file is?...

Finally, here's the most common response I get when I reply to Barracuda backscatter (the domain example.com is used below, but it will be something else depending on the Barracuda box that sends it out):
This is an automatically generated Delivery Status Notification

Delivery to the following recipient failed permanently:

postmaster@example.com

Technical details of permanent failure:
PERM_FAILURE: SMTP Error (state 13): 550 <postmaster@example.com>: Recipient address rejected: No such user (postmaster@example.com)
This shows that Barracuda Networks are not doing a great job at having customers set up their boxes properly. Pert near all of these domains end up getting a listing for being RFC-ignorant with respect to postmaster. I don't have a count yet, but it's got to be over 100 domains that I've reported there, "thanks" to the Barracuda backscatter from their poorly configured appliances.

p.s. A more rare event is that sometimes a Barracuda Spam Firewall actually blocks my automated response to its backscatter, claiming my request for them to stop bouncing spams to me is itself a spam! Although telling me that my request was blocked isn't backscatter, it will get them listed as being RFC-ignorant. They're blocking requests to the postmaster address.

Tuesday, December 11, 2007

Why I stopped reporting (Googlepages, Blogspot) spams to Google


I've been a Spam Fighter for 7+ years, and Google has got me frustrated. I'm quite happy about the little spam I get on my Gmail account. But they are doing a bad job of controlling spam on their systems. Worst of all, they don't accept automated reports, but instead want users to spend time reporting spams and abuse on Google systems manually.

Every other ISP or web service accepts automated spam reports via systems like SpamCop.net. Google requires us to report spams with on-line web forms. A good example is this web form to report an adult-content blogspot.com page. I can't tell you how many spams I have got with blogspot.com links that redirect to porn sites in China somewhere. I have reported them all to Google using that stupid form. Guess what? The links are all still up, sometimes weeks after I report them. There is never any closure or follow up, despite when I include my email address.

So, I'm resorting to ranting about this poor quality of service in a blog. You can see some of the other rants about this subject in this discussion in the Google Group for Blogspot. Try some of the links (if you're not afraid of seeing porn) to see if they're still up. Google knows about these but is not taking quick action.

Google has some of the sharpest, most creative people working for them. Yet they require us to submit spam reports about Gmail abuse using archaic web forms that cannot parse emails or require us to submit Googlepages abuse emails one link at a time.

Google refuses SpamCop.net automated reports about spams that contain links to Googlepages.com or blogspot.com pages. Here's an example of what happens when you put a spam into SpamCop that contains a Googlepages.com link:

Re: http://burtsmithwx.googlepages.com/index.html (Administrator of network hosting website referenced in spam)
To: abuse@google.com (refuses to accept this type of report)
To: abuse#google.com@devnull.spamcop.net (Notes)

Re: http://yodatrinidadt.googlepages.com/index.html (Administrator of network hosting website referenced in spam)
To: abuse@google.com (refuses to accept this type of report)
To: abuse#google.com@devnull.spamcop.net (Notes)
Accepting automated SpamCop reports would be the most intelligent way for Google to fight the spam on their systems.

Finally, let's not forget that spammers use software to create the spams, the Googlepages.com and Blogspot.com pages. So why shouldn't Google use software to detect and delete them!? Perhaps it doesn't affect the Google bottom line, and so they haven't put any resources on it.

As of today, I'm no longer reporting any more spam to Google with their archaic methods. They need to modernize on this aspect and start accepting automated reports from SpamCop.net.

Finally, stop exploiting the volunteer spam reporters! We have better things to do!

Interesting links: Blog Spam: A review

Saturday, October 27, 2007

Victims of email backscatter unite!

Currently I receive close to 500 backscattered messages per day at one of my email addresses. Some spammers must be really pissed off at me, or maybe my alias is a good one to use because it's on a common email alias provider. Anyway, I decided it's time to do something about it, if possible.

First, you may be asking what's a backscattered message? It's a "bounce" email that arrives in your in-box, that usually has a subject of "Delivery notification: delivery has failed", "Deliver status notification", "failure notice", etc. The messages typically originate from "Mailer daemon", "postmaster", or "Mail delivery subsystem". Backscatter are messages that tell you about problems with messages you did not send.

Huh? This is weird... What's up with that? Well, you get backscattered messages because of two problems:

  1. a spammer has forged your email address in the "From:" field of the spam she sent;
  2. some email server has received that spam, and for some reason decided to reply to the forged email address (yours), probably to tell you:
    • it couldn't be delivered because the destination address was not valid, the user's mailbox was full, etc.;
    • the user is on vacation (vacation auto-responder);
    • the message contained a virus and was blocked;
    • the message was detected as spam and blocked.

The last two reasons are particularly incredible. If the mail server is smart enough to determine a message was a virus or spam, then it should know darn well that replying to its "From:" address makes no sense because -- guess what -- spammers/viruses nearly always forge the "From:" address!

The following image shows the anatomy of email backscatter. Click on it to see a bigger version.

Anatomy of email backscatter

The bad news is, you won't be able to stop spammers from forging your email address (problem #1 above). But there is some good news -- you may be able to do something about the servers that bounce messages (they create the backscatter messages, problem #2). Did you know that AOL, Hotmail, Yahoo, Gmail, etc. are examples of email services that do NOT create backscatter? This is because they are well administered and their email servers are correctly configured.

Sadly, there are too many email servers on the Internet today that create backscatter. See the links about backscatter on the side menu of this blog for information about the technical details. One of the worst offenders is the Barracuda Spam Firewall. I get sometimes 15 of their bounces per day, telling me the "Message you sent was blocked by our bulk email filter". Their software accuse me of sending spams, because the designer of the software made a mistake in thinking it was a good thing to reply to spams that have forged emails. That is irresponsible, and Barracuda should fix their product.

All ranting aside, if you're willing to work a little, I suspect there is a huge potential to fight spam if one is a victim of backscatter. Here are a couple of thoughts about useful information that could be exploited, provide there was some way to process it:

  • Email bounces arrive practically in "real time" from spam runs. As soon as the spammer starts hitting mail servers with her spam, the bounces of failed messages come in to the victim of the forged "From:" address.
  • Some of the bounces themselves contain full headers of the spam email. Most often, one can trace back the IP address to a spam-sending zombie.
  • A spam run will typically have the same subject of spam, or even the same "spamvertised" URL to a web page.
  • (if you have concrete ideas or resources about how to use this information, send me an email or post your ideas here in response)

There is a block-list for backscattering hosts at www.backscatterer.org. I have already contacted them, but they are only interested in my backscatter if I host an MX on UCEProtect.net's system. Apparently their backscatter-reporting is hard-coded from that system. Sadly, I would lose the backscatter if I were to move my email there, as it's coming into an email alias, which forwards to my Gmail system. By the way, Gmail does a great job of classifying spam, including backscattered messages.

You can report backscatter to Spamcop.net, and, in principle, the administrators of offending sites will get a SpamCop report. However, the link that SpamCop sends to them does a poor job of explaining the problem and the solution, and I don't think it has much affect. Sadly, a backscattering host will only get put on the SpamCop block-list if enough different people report it. So, if you're a victim of backscatter, you can make a difference by reporting it to SpamCop!

I report all backscatter (and other spams) to SpamCop fairly easily using a combination of Gmail, FreePOPs, Thunderbird and SpamCop's quick-report feature (which is free). See how to setup FreePOPs to download spam from your Gmail spam folder. In the interest of getting more people to report their backscatter, I would be happy to help you set up your system to do it easily. Please post your requests here, but don't identify your emails on this blog. If you want me to know specific info, you can email me at my Gmail address if you choose.