Jacob Haller (jwgh) wrote,
Jacob Haller

  • Mood:

Why I hate Sender Policy Framework

Sender Policy Framework is a reasonably recent antispam scheme that aims to reduce spam by breaking current email standards in ways that will be trivial for spammers to circumvent.

The SPF website contains the following summary of why it's supposed to work:

The Rationale: SPF Explained

Have you ever gotten spam from yourself? I have, and I've been thinking hard about how to stop it! I didn't send it. It came from a spammer. If we could stop spammers from forging mail, we could easily tell spam from ham and block the bad stuff.

SPF makes it easy for a domain, whether it's an ISP, a business, a school or a vanity domain, to say, "I only send mail from these machines. If any other machine claims that I'm sending mail from there, they're lying."

When an AOL user sends mail to you, an email server that belongs to AOL connects to an email server that belongs to you. AOL uses SPF to publish the addresses of its email servers. When the message comes in, your email servers can tell if the server on the other end of the connection belongs to AOL or not.

And that's it! SPF aims to prevent spammers from ruining other people's reputations. If they want to send spam, they should at least do it under their own name.

And as a user, SPF can help you sort the good from the bad. Reject mail that fails an SPF check. Use it to help your spam filters make a decision. Have confidence that mail that SAYS it's coming from your bank, your credit card company, or the government really is!

If you do get spam that passed an SPF check, then you know you should hold the sending domain responsible for the message.

So what's wrong with this?

First there is a question regarding what address a message can be considered to have 'come from'. The obvious choice is the address that appears in the 'From:' header, but that's not the address SPF uses, and for good reason.

To see why using that address would be a mistake, consider discussion mailing lists. There are a bunch of mailing lists out there that allow subscribers to post messages which then are redistributed to the other subscribers of the mailing list. (I run several of these.) If I post a message to one of these lists, the message will go out with my email address in the 'From:' header, but it will be distributed by the mail server that the mailing list is set up to use, not from any mailing list I have control over. If SPF looked at the 'From:' address (which, again, it does not) and looked to see if the mail server the message was coming from was listed as one of the 'allowed servers' for the domain, it almost certainly wouldn't find it, and the result would be that the message would be rejected. So this sort of validation would break mailing lists.

There is another choice in considering where a message has 'come from', however, that doesn't have this problem. It's the message's return path.

When an email is sent, an address is specified to which any nondelivery notifications that are generated by the email should be sent. That address is called a return path. (It also has a bunch of other names.) So if the email is sent to a bad address, the return message saying that the address was bad goes to this return path address.

Most of the time the return path and the 'From:' address are identical, but it isn't required to be. For instance, in the mailing list example above, mailing lists will usually have a special 'bounce address' that nondelivery notifications are supposed to go to so the mailing list software can automatically process them and remove bad addresses that are subscribed to the mailing list. When the mailing list software distributes a message to the mailing list's recipients it replaces the existing return path with this bounce address (typically something like owner-listname@lists.example.com).

Additionally, nondelivery notifications themselves typically have a nonexistent or null return path (the SMTP command to set a return path is 'MAIL FROM:<email@address>'; for nondelivery notifications the command used is 'MAIL FROM:<>'). This is basically to prevent pointless mail from being sent (if I send a message to a bad address, and that bounces, and the nondelivery notification gets sent back to my address which now is bad for some reason so that the nondelivery notification itself bounces, does anyone really want to see it?) and also to reduce the possibility of mail loops.

Anyway, the return path is another good candidate for determining what email address a message 'came from', it won't break mailing lists the way using the 'From:' header would, and that is in fact what SPF uses.

But that's dumb too. The SPF site says, 'Have you ever gotten spam from yourself? I have, and I've been thinking hard about how to stop it!' But this won't stop it! Since the return path and the 'From:' address don't have to have any relationship to each other, and since the return path is usually not displayed by most email programs, SPF won't actually prevent you from receiving mail that appears to be from you. In fact I suspect that it would be relatively trivial to modify at least some existing spamware so that it uses different addresses in the 'From:' and return path. In fact, it could use a null return path and SPF would then have nothing to validate against!

(If you want to see what the SPF folks have to say about 'From:' vs. return path, here it is.

What does SPF have to say about the null return path issue? They give two solutions. They acknowledge that the first solution won't actually work, and their second solution is vague, overly complicated, and (I suspect) won't work either (you'd either need to spend a lot of time figuring out routines to decode nondelivery notifications in all the several trillion changing formats that they occur, or you'd have to live with rejecting a lot of legitimate nondelivery notifications). They also say this, which I think is interesting:

In either case an MTA should reject messages from null senders that have more than one recipient.

Since I can think of several scenarios where this results in legitimate mail getting lost I think that a little more information on their reasoning here would be nice.

So I don't think SPF will put much of a dent in spam at all. In fact, I think it will be largely ineffective.

But the real reason that I hate it is because it violates standards in a way that reduces the portability of email addresses.

For example, I have a vanity domain, jwgh.org, which I use when sending personal email. Now, in an SPF environment, what do I have to do in order to be able to send mail using that address?

One solution would be to specify my return path to be an address that, according to the SPF records for the address's domain, would be allowed to send mail using the mail servers that I happen to be using. (For instance at home I use cox.net's mail servers so I could set the return path to my cox.net address.) The big problem with this is that I don't think that any of the common email programs people use to compose mail these days allow you to do this. There's also a lesser issue I will go into later.

The other solution, and the one that I think the SPF folks want me to use, is to create SPF records for jwgh.org that list all of my ISP's mail servers so that when someone receives mail from me they can use SPF to check to see that the mail server it came from is in fact one of the ones that my SPF records say jwgh.org mail can come from.

One of the problems with this is that this requires me to know a lot more about my ISP's infrastructure than I've needed to before now. For instance I'm not actually sure what the names and IPs of all of cox.net's mail servers are. I can find out, I suppose, but it's a bit of a pain, and what happens if they change their setup without me realizing it?

Another problem is that if I take my laptop and go visit my sister in New Hampshire, who uses Verizon DSL, then I have to add Verizon's mail servers to my list of SPF servers (if I can find out what they are) if I want to send email. (This scenario is a problem even if I could specify my return path to be different from my 'From:' address, because I don't have a verizon.net address. I could use my sister's, but then if I sent mail to a bad address and it bounced my sister would end up with the bounce, which would also be no good.)

There are various workarounds to this, but they are a pain in the butt, and I am annoyed that I may have to figure out how to implement them just because of some stupid antispam scheme that isn't going to work anyway.

And of course the really annoying thing is that a lot of big sites are going to start using SPF anyway despite its unreliability, because they are so overwhelmed with spam that they will try almost anything. (I believe that AOL has signed up, for instance.)

(There are various other stupid things about SPF, but I leave finding them as an exercise for the interested reader.)

Tags: email, rant, spf

  • unconscious or deniable = OK

    I happened across this little quotation from the head of NASA a little while ago: In an interview with NPR's Steve Inskeep airing May 31, 2007 on…

  • letter from Asimov's

    I'm just getting around to reading the March 2007 issue of Asimov's, which contains the following letter from a reader of an essay by Kristin…

  • T-Mobile Hotspot grumble

    Every so often it becomes necessary for me to work someplace that isn't my house. These days I like to go to a Borders to do this. Border's is a…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.