Ponders Corner

By Rick Moen


I'm a pedant. It gets me chicks.
 -- Xiphias Gladius

Preface

In the years that I've been privileged to know Rick Moen, I've seen him wax pedantic [1] on many occasions; in fact, my image of The Canonical Rick involves him comfortably ensconced in a large armchair, perhaps with a mug or a snifter of some notable potable, and holding forth on a technical topic. Now, not everyone sees the term "pedantic" in the positive light that I do; however, there is a salient factor to Rick's pontifications that raises them sky-scraper-high above the average. Just as Isaac Asimov noted with regard to curses pronounced by those who control technology, Rick's well-articulated, well-informed, and finely-detailed descriptions of these technical topics are effective. Whether he's speaking about security, network engineering, or mail, I've found that I always learn something new from reading what he writes - and usually save his posts for later reference.

Odd as this may sound for the guy who publishes the LG every month, the idea of exposing these mini-treatises to a broader audience had not occurred to me until this past month; my weak excuse is "fish are not aware of water". Rick has very kindly granted us permission to publish them, and I've taken on the job of reformatting them for publication (they all started out as posts to a list, or one-to-one conversations between us.) Any slips in style are probably my omissions; the rest is classic Rick Moen. I hope you enjoy it and find it as useful as I have.

-- Ben Okopnik, Editor-in-Chief, Linux Gazette

Mail Relaying/Authentication

[ This conversation came about when I asked Rick to set me up with a somewhat unusual email configuration at his linuxmafia.com server. -- Ben ]

The term "smarthost" harks back to the days before SMTP was universal and DNS reasonably reliable: The idea was that you assumed that your own host probably wasn't as well-connected and capable as, say, the UCBVAX machine at Berkeley. So, you would configure your local outbound mail processes to lob all outgoing mail over to UCBVAX via batched, dialed-up UUCP, or SMTP, or whatever, trusting to UCBVAX to perform necessary DNS lookups and redeliver the mail. UCBVAX was, in that sense, dubbed a "smarthost" for your site, in the sense that it's smarter about DNS and mail connections than yours is.

Not only UCBVAX but also pretty much every other mail site used to perform this service routinely for anyone and everyone, as a convenience for the Internet community. Then, along came spammers. Pretty soon, having a mail server available for redelivery of mail to and from everyone came to be seen as a public menace, like having an unfenced swimming pool in a neighbourhood full of children. The term for such a thing became "open relay", and self-appointed spamcops started maintaining databases (blocklists) of IPs that had been tested as being willing to relay for random members of the public.

When we relay mail for you at my server, what we want is not an open relay, but relaying specifically for outbound mail coming from you. The question, then, is: By which of the several possible means is my MTA going to know that the mail being received is from you?

As an example of a problem that can occur, my friend Seth David Schoen owns "loyalty.org", and I agreed to be an MX host for that domain. I just now noticed that he stopped using me as an MX, and didn't bother to tell me: 

  ~ $ dig loyalty.org mx +short
  50 loyalty.org.
  80 mx.movealong.org.
  ~ $

As a convenience for him, I've had: 

  ~ $ cat /etc/exim4/eximconfig/domains/relay 
  # List of domain names accepted for relaying
  #
  loyalty.org
  ~ $

That would be relevant if you were asking me to accept mail for your domain, but it's a different problem. There's something called "SMTP AUTH", a modification of standard SMTP specifically for situations like yours, where you provide some sort of client authentication, as part of an extended variant on SMTP negotiation, on mail you want to relay outbound through my mail host. This file (root-readable only) seems relevant: 

~ $ linuxmafia:/etc/exim4/eximconfig/accept# cat auth_logins 
# Login names and passwords for client authentication.  This allows users from
# remote hosts to send messages through this Exim server with similar
# privileges to local users and without being rejected as forgeries.
#
# USAGE:  login[@domain]        password
#
# For this to work, the remote user must setup SMTP authentication in their
# E-mail client software.  Most clients (Such as MS Outlook) are capable
# of this.
# 
# IMPORTANT:  If a login exists on this server you should ideally use a
#             different password for SMTP authentication to their normal
#             login password for increased security.
#
#             Because this file contains passwords, please ensure that it
#             is not readable by other users on your system (I.e:
#             chmod o-rwx accept/auth_logins)  Also, ensure that it has
#             the correct ownership and group for your Exim MTA to be
#             able to read it.
#
# By default, remote authenticated users must use a local or relay domain
# name (See domains/local and domains/relay), however this can be disabled
# by setting config/check_sender_auth to No (Not recommended - You don't
# want users sending forged messages through your server!)
#
# Also, by default remote authenticated users must send using their login
# name (And domain if specified) as their sender address (This can be
# disabled by setting config/check_sender_auth_exact to No (Not recommended.))
#
# If a domain is not specified in their login below, any local/relay domain
# can be used, but the localpart (Before the @) must match their login. If
# a domain is also specified, they must also use that domain for sending
# (NOTE:  They will also need to specify the full name@domain in the
# authentication settings of their E-mail client software.)

The referenced "domains/relay" file is the one quoted previously, that has loyalty.org in it.

[1] It's the 1937 model, and it's quite shiny by now. Boy, they really built'em well back then...

Mail Redirection vs. DSNs

[ This was from a discussion among the LG staff with regard to our recent wholesale changeover of list addresses to "*@lists.linuxgazette.net". The question of how the old list addresses ("*@linuxgazette.net") should respond to mail sent to them is an important one; we wanted to notify anyone who sent mail there by mistake, but did not want to leave the sysadmin who handles the mail for LG processing the tons of spam that would surely continue to hit them. The solution that Heather suggested and Rick details below is an excellent one, and is now in process of being implemented. -- Ben ]

I recently wrote, quoting Heather: 

> > 	4) let the MTA punch up a redirect
> > 
> > Generally in terms of server load that last is good because the MTA
> > really tells the other MTA "dude, they moved". 
> 
> I just looked up the RFCs on this.  
> 
> Looks like the main Delivery Status Notification error code to return
> would be "551 User has moved; please try <address>".  RFCs 1893 and 3463
> also specify an optional "extended status code" of 5.5.1 for this situation, 
> accompanied by descriptive text such as "Bad destination mailbox
> address."
> 
> Here's a page that seems to include tips on how to do it in sendmail:
> http://www.technoids.org/cf_README.txt  (Relevant to Kayos.)
> 
> (Look for the phrase "551 User has moved".) 
> 
> I dropped on the floor any sendmail clue I had years ago, though.
On reflection, it's a brilliant idea; I'm glad you suggested it, Heather. Much better than /etc/alias / procmail / Perl / formail-based autoresponders. This post will try to explain why I think so. It has to do with the advantages of DSNs (delivery status notifications) over bounces.

Consider the path of a piece of mail you send to, say, the Smaug (LUG in Santa Cruz, CA) list hosted at SVLUG (smaug@lists.svlug.org).

Let's say that Smaug decide to move their mailing list to a new home at lists.scruz.org. The smaug@lists.svlug.org address is very well known, and there's no way the officers can hunt down and update all references to it elsewhere. So, they ask SVLUG to create some automated mechanism to let people know, when/if they write in.

Approach A:

SVLUG's admin chooses to create a /etc/alias entry for address "smaug" that pipes any mail arriving at that address to a Perl script that runs formail and sends back the claimed (note!) sender address a response e-mail advising of where the mailing list has moved to.

Spammers of course will have long had smaug@lists.svlug.org on their mail-to rosters, so they barrage it with fakemail advertisements. Furthermore, all the MS-Windows trojans and viruses that operate their own MTAs from compromised desktop boxes likewise pick up that address (along with others) from the MS-Outlook / MS-Outlook Express address books, MSIE caches, etc., on compromised boxes, and generate their own fakemail addressed to it, hoping to hit more vulnerable Windows boxes.

All of that junkmail rains down endlessly upon the smaug@lists.svlug.org address. Fakemail being what it is, the "From:" header will be forged. Many of the faked "From:" addresses will have been stolen in the same fashion as smaug@lists.svlug.org's was, or scraped from mailto: links on Web pages, etc.

The poor autoresponder doesn't know this, and knows only that its sworn duty is to generate an outbound reply message, addressed to the "From:" address of each message received at smaug@lists.svlug.org. Statistically speaking, almost 100% of that arriving mail will be fakemail with forged addresses. So, in generating mail to forged addresses in many cases belonging to innocent parties, the autoresponder pretty much doubles the volume of spam.

The autoresponse, in this case, is functionally the same as a bounce message: It's generated after acceptance of the original mail, and is sent via a fresh SMTP delivery attempt to the claimed, apparent sender on the basis of the original's (often forged) headers.

Approach B:

SVLUG's admin elects to, instead, use some sort of MTA rule that, instead of operating after acceptance of the original mail, causes the MTA to issue a custom "error message" of the type I quoted above, e.g., 

   551  User has moved; please try <smaug@lists.scruz.org>.
   551  5.5.1 Bad destination mailbox address.  User moved.

This is not issued as an e-mail, but rather is fed back directly to the delivering MTA as part of the SMTP conversation. Thus, it is guaranteed to be reaching the actual sending SMTP process, as it doesn't credulously rely on "From:" headers. The "error message" in question is an example of a Delivery Status Notification (DSN), part of the SMTP standard.

It's the responsibility of the delivering MTA to do the right thing with DSN responses differing from the normal "250 Message Accepted". You will sometimes have seen those, sent back to you by your own or some other MTAs. For instance, DSNs may inform the sender of the original message that it could not be delivered due to a temporary or permanent problem, and/or whether or not and for how long delivery attempts will continue. There's always both a three-digit numeric error code and an explanatory English text accompanying it.

It's important to realise that a DSN (even one reporting permanent delivery failure) is not a bounce, and to realise the advantage: The DSN doesn't have to rely on inherently unreliable "From:" headers, but rather is issued to the absolutely-guaranteed-correct delivering machine right at the time of attempted delivery. A 5xx DSN tells the delivering machine "Absolutely no. Die, spammer, die." A 4xx DSN tells the delivering machine "Try again later." (This is sometimes used for teergrubing of machines trying to drop off heinously blatant spam -- stringing them along for repeated redelivery attempts to waste their time and mail-queue space.)

In no case does the DSN, in itself, cause the generation of secondary (collateral) spam, the way bounces -- and autoresponder messages -- do. Of course, the SMTP mail process that is attempting delivery could turn around and do something inappropriate with a DSN, such as drop it into an e-mail and send it somewhere harmful, depending. Here's a different scenario where that might happen:

This is something that might have happened while my friend Richard Couture's myrddin.imat.com machine was still serving as my backup MX (mail exchanger). Notice that a backup MX functions as a relay (or "Forwarder"), being one of that list I posted earlier, of machines caught in the crossfire of the spam war because they're forwarders.

The piece of fakemail gets accepted by myrddin.imat.com, which then attempts to open a new SMTP connection to linuxmafia.com, the end-destination host. (This is the essence of relaying or forwarding.) linuxmafia.com, during the SMTP delivery attempt, tests the incoming message stream and determines it to be forged fakemail of high "spamicity", and issues this DSN directly to myrddin.imat.com as a way of refusing delivery: 

550 Message Refused:  Medical/medicine/pharmacy/prescription drugs spam.
550 .
550 Verify: rick-verify@linuxmafia.com
550 Contact: postmaster@linuxmafia.com
550 .
550 Sorry, your message has been rejected because
550 its body text/content is prohibited for the
550 above reason.
550 .
550 We apologise if you have sent a legitimate
550 message and it has been blocked.  If this is
550 the case, please re-send adding verified- to
550 the beginning of the e-mail address of each
550 recipient.  If you do this, your message will
550 get through these restrictions.
550 .
550 If your message has been incorrectly blocked,
550 please let us know at the above contact address.

...and then my machine closes the connection.

The myrddin.imat.com machine is potentially in a bit of a bind, because it's already accepted the mail for delivery (forwarding). Many old-school admins of such forwarders/relays, still caught in that 1980s "all mail is sacred" mindset I referred to, will have their machines mindlessly put the DSN into an e-mail and attempt to send it back to the apparent sender -- e.g., to the virus-forged address of Heather Stern <star@starshine.org>, in this case. In which case, Heather gets yet another piece of secondary spam. (Whee!)

Of course, nowhere is it inscribed in stone that DSNs must be sent back to apparent senders of transparently fraudulent e-mails. Also, myrddin.imat.com's admin should have thought about that dilemma before accepting mail uncritically for redelivery. To the extent harm was done, it's on his watch.

Taking a slightly less harsh perspective, this sort of situation is why I put an end to backup MXing, and would use it only on machines implementing the exact same anti-spam policy as the primary MX -- specifically to avoid that problem.

 

Rick is [2010: was] a member of The Answer Gang.

[BIO] Rick has run freely-redistributable Unixen since 1992, having been roped in by first 386BSD, then Linux. Having found that either one sucked less, he blew away his last non-Unix box (OS/2 Warp) in 1996. He specialises in clue acquisition and delivery (documentation & training), system administration, security, WAN/LAN design and administration, and support. He helped plan the LINC Expo (which evolved into the first LinuxWorld Conference and Expo, in San Jose), Windows Refund Day, and several other rabble-rousing Linux community events in the San Francisco Bay Area. He's written and edited for IDG/LinuxWorld, SSC, and the USENIX Association; and spoken at LinuxWorld Conference and Expo and numerous user groups.

His first computer was his dad's slide rule, followed by visitor access to a card-walloping IBM mainframe at Stanford (1969). A glutton for punishment, he then moved on (during high school, 1970s) to early HP timeshared systems, People's Computer Company's PDP8s, and various of those they'll-never-fly-Orville microcomputers at the storied Homebrew Computer Club -- then more Big Blue computing horrors at college alleviated by bits of primeval BSD during UC Berkeley summer sessions, and so on. He's thus better qualified than most, to know just how much better off we are now.

When not playing Silicon Valley dot-com roulette, he enjoys long-distance bicycling, helping run science fiction conventions, and concentrating on becoming an uncarved block.

Copyright © 2005, Rick Moen, rick@linuxmafia.com.

This article was first published in issue 115 of Linux Gazette, June 2005.