[sf-lug] (forw) [conspire] DMARC follies

[Rick Moen]

----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----

Date: Tue, 21 Jul 2020 12:51:51 -0700
From: Rick Moen <rick at linuxmafia.com>
To: "paulz at ieee.org" <paulz at ieee.org>
Cc: conspire at linuxmafia.com
Subject: [conspire] DMARC follies (was: verbose way I've yet seen seen say
	'Ayn Rand was an idiot' Re: Federales in Portland?

Quoting Paul Zander (paulz at ieee.org):

> On a slightly different topic,  I haven't gotten any emails from Rick
> in a long time.  Many other Conspire emails, but none from Rick.Has
> something gone astray with my email / black list ??

CC'ing you now (which I normally wouldn't do on a mailing list).

OK, this is going to be a bit of a stemwinder, because it requires
background.

tl;dr:  I had fully implemented DMARC.  DMARC sucks epically.  I have 
reverted and killed that sucker.  Should be better as soon as DNS
propagates.


Longer version:

About a month ago-ish, I followed bad advice from (where else?) the
SF-LUG mailing list from someone who felt some imagined deficiency in my
server's functionality owed to the claimed fact that linuxmafia.com
'doesn't have a DMARC record'.  The person claiming this is said to be a
sysadmin, yet that person didn't actually check to _see_ if I publish a
DMARC record -- not directly.  That person used a Web site that probed
my DNS and informed him that linuxmafia.com lacks such a record.  If I
recall correctly, Jim Stockford panicked about linuxmafia.com's Mailman
disabling delivery to a bunch of subscribers because of DMARC validation
errors of _their_ (the senders') domains.  He assumed in error that
there must be a problem at linuxmafia.com, ignored my explanation, and
consulted with the third-party sysadmin, who lazily fired up a Web page,
got a report on linuxmafia.com, and said 'Oh, there's the problem.
linuxmafia.com lacks a DMARC record.'


My DNS _had_ had one for years.  It looked exactly like this:

_dmarc          IN      TXT     "DMARC: tragically misdesigned since 2012.  Check our SPF RR, instead."

Directly checking this or any other DNS record can be done using
standard tools like /usr/bin/dig:

$ dig -t txt _dmarc.linuxmafia.com @ns1.linuxmafia.com +short
"DMARC: tragically misdesigned since 2012.  Check our SPF RR, instead."
$

I readily admitted to Jim (and the third-party sysadmin) that this
record is deliberately non-compliant with the DMARC spec -- because 
I regard DMARC as botched, ergo I wanted to publish a vote of no
confidence in this DNS/SMTP extension.  But it's incorrect to claim that 
there was no DMARC record.


'What's DMARC?', you ask.

DMARC is Phase Two of an initiative from Yahoo to curb SMTP forgery 
through extensions to e-mail and DNS.  It provides a protocol for 
the owner/operator of a SMTP-sending domain to publish two things:
information about what IP addresses are deemed valid senders of the
domain's mail, and a crypto-based means of receiving SMTP sites to
verify that a sender's content within an SMTP message hasn't been
altered en-route.

The Yahoo-provided protocol for doing the latter thing, attesting to
the mail contents being uncorrupted, is via a Yahoo-originated crypto
protocol called DKIM (DomainKeys-Identified Mail), which was Phase One
of Yahoo's plan.  DomainKeys was an earlier attempt at that same task,
now obsoleted by its DKIM replacement.

Problem 1 of 2:  DKIM was botched.  Its signing of messages was so badly
designed that any message that transits a mailing list (such as one
operated by GNU Mailman or Sympa) arrives at its destination (the
subscriber) no longer passing DKIM validation of the message contents.
Thus, DKIM was by incompetent design mailing-list-hostile.  I
accordingly deliberately ignored DKIM as it started to become popular.

Problem 2 of 2:  When Yahoo engineered DMARC as an umbrella protocol
(explained below), judging by your experience and that of some other
commenters, it seems to have somehow done even more damage.  I don't
know all of what is going wrong in this area, but I know enough that I'm
tossing DMARC again.

In its definition, the DMARC specs say that it is _supposed_ to rest 
entirely on either/both of DKIM or SPF.  SPF is an earlier and
simpler, non-Yahoo extension to DNS and SMTP, where you have a DNS
record that says 'Please accept as genuinely from this domain SMTP
only if it arrives from the following IPs.'  SPF achieves its modest
aims quite well.  I like SPF, and have implemented it fully since its 
origination, some time around 2002.


The SPF record looks like this:

linuxmafia.com.  IN      TXT     "v=spf1 ip4:96.95.217.99 -all"

That is very simple.  It says 'Check the SMTP envelope on the arriving
mail.  If the envelope claims it's from linuxmafia.com, but the
delivering SMTP IP address isn't 96.95.217.99, then it's SMTP forgery 
of my domain, and please reject it.'

Because of the way mailing list manager (MLM) software (Mailman, Sympa,
others) works, transiting mail via MLM reflectors to subscribers 
does _not_ violate SPF.  


After Jim Stockford raised the ruckus with the third-party sysadmin, 
I made one change.  Instead of the aforementioned...

_dmarc          IN      TXT     "DMARC: tragically misdesigned since 2012.  Check our SPF RR, instead."

...I put in a _real_ DMARC record:

_DMARC          IN      TXT     "v=DMARC1; p=reject; sp=reject; pct=100; aspf=r; fo=s; ri=86400; rua=mailto:hostmaster at linuxmafia.com; ruf=mailto:hostmaster at linuxmafia.com"

Let's go through the right-hand side:

  v=DMARC1 :  This is version 1 of the DMARC spec
  p=reject :  Policy is reject if the mail fails validation.
  sp=reject : Policy for subdomains is the same.
  pct=100  :  Apply this regime to 100% of mails.
  aspf=r   :  Alignment mode for SPF should be reject.  (Reject mail
              if it fails SPF.)
  fo=s     :  Failure-reporting option is: Send failure report if 
              the message fails SPF.
  ri=86400 :  Reporting interval is 86400 seconds, or one day
  rua=mailto:hostmaster at linuxmafia.com : Report URI for aggregate 
              feedback is via mail to hostmaster at linuxmafia.com
  ruf=mailto:hostmaster at linuxmafia.com : Report URI for failure
              information is mail to hostmaster at linuxmafia.com.

So, basically this DMARC says 'Use my SPF record.'  Since I don't publish
or implement DKIM signing, that half of the DMARC spec goes unimplemented,
which is a valid configuration.


So, what is _supposed_ to be happening is that, if mail-receiving sites that
receive linuxmafia.com mail check for a DMARC record, they find one that
says 'Hey, refer to my SPF record', and the results ought to be the 
same as just with SFP.  Yet, based on your remarks and others, it appears 
that some sites like the one that operates your ieee.org account are 
rejecting rick at linuxmafia.com's mail, at least when it gets bounced through 
the MLM at linuxmafia.com.  That's pretty broken.  Without spending 
endless hours debugging, I think I'm justified in blaming Yahoo's cruddy
DMARC spec or at least the implementation of it.  Therefore, I've just
done the obvious, and um-implemented DMARC:

$ dig -t txt _dmarc.linuxmafia.com @ns1.linuxmafia.com +short
"DMARC: tragically misdesigned since 2012.  Check our SPF RR, instead."
$

You should now get at least one copy of this mail (direct vs.
MLM-mediated) -- although it will take a while for my banishing of
DMARC to propagate to other DNS servers.  Please acknowledge.  And
thanks for bringing this to my attention.


Oh, about the mass-rejection that occurred of a bunch of messages sent
out to the SF-LUG mailing list:  Jim Stockford noticed these because
he's (_allegedly_) listadmin of sf-lug at linuxmafia.com.  A bunch of
subscribers all at once got their subscription delivery disabled because
of excessive bounce score.  As I explained to Jim, this happened because
a frequent poster, who has an @yahoo.com posting address, sent several
posts.  When those transited the sf-lug at linuxmafia.com mailing list, 
and were retransmitted to subscribers, any subscribers' receiving domains 
that check DKIM found that the original (yahoo.com) subscriber's DKIM 
crypto signature no longer checked out -- owing to DKIM having been
botched and being MLM-hostile.  So, those domains rejected the
subscribers' copies of the yahoo.com poster's mail, which delivery
failure got reported back to the linuxmafia.com SMTP server, which
notified Mailman, which incremented the receiving subscribers' bounce
scores each time that happened.  Once bounce score rises too high in a
given period, Mailman disables delivery of the subscriber's mail.


Yahoo got informed immediately when it rolled out DKIM that it was
breaking all the mailing lists in the world -- specifically that
it would cause any MLM subscriber who's posting from a domain with a
strongly asserted DMARC/DKIM policy to drive up _other_ subscribers'
bounce scores (if their SMTP servers check DKIM on arriving mail), 
in addition to those subscribers not being able to see the poster's
posting (because of the subscriber copy getting rejected for failing
DKIM validation).

Yahoo didn't care.  Its response was that all the MLM software in the
world would simply need to be rewritten to accomodate DKIM signing.

Very recent Mailman versions include an optional kludge to 'munge' 
(fool with) the From: and Reply-To: headers on any mail from a sending
domain with a strongly asserted DMARC/DKIM policy, to work around the
DKIM brain damage.  My server doesn't yet run a recent Mailman version,
so I cannot yet enable that workaround.

Grrr.


_______________________________________________
conspire mailing list
conspire at linuxmafia.com
http://linuxmafia.com/mailman/listinfo/conspire

----- End forwarded message -----