[sf-lug] List posting missing? Well, let's see ...

[Michael Paoli]

List posting missing?  Well, let's see ...
So ... the question/claim occasionally comes up,
someone says they sent to the list, but it didn't get posted to the
list.  Well, I've got an email I sent to the list where /maybe/ that
happened.  Could moan at Rick over it, but why hassle him ... I do also
have access so I can actually check.

So ... first we start with relevant information.
Most relevant - and far too often what folks fail to include,
is full header information.  Rather than start with all of that,
let's start with the generally most useful bits thereof ... and I
didn't receive a bounce or anything like that, so far just have what
was presumably/allegedly sent ... don't yet even know for certain if
it even made it to Rick's linuxmafia.com host.

So, header bits ... rather than start with all, let's start with
just what's generally most highly useful and relevant, and generally
highly handy for uniquely identifying in logs.
Most notably these pieces of information:
Message-ID: <20210320130024.16699ms3hj4g15ys at webmail.rawbw.com>
Date: Sat, 20 Mar 2021 13:00:24 -0700
From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
Actually, the envelope FROM (not header From:) is generally more
useful/important, but the email of that and header From:
are typically the same (at least for most non-spam email and such),
and in this case they are in fact the same.  In some types of
forwarded email, the From: header might be different than the
envelope FROM, but in such cases, the envelope FROM is also
generally shown in other header field(s), e.g. Sender:, hence
the generally useful request to include ALL headers, as other
relevant needed bits may be in other header field(s).

Oh, also the envelope RCPT TO is quite important, and the To:
and related headers ... but simpler case here, both envelope
RCPT TO and header To: are sf-lug at linuxmafia.com

And, some additional related information.  I was in no hurry, I
figured I could wait and see if it bounced, or other traffic on
that list, or other lists from that host worked.  I did also
look at the on-line archive, and it also wasn't showing there.
I did also subsequently send to another list on that same host,
and that one went through fine - which would seem likely to
rule out any type of general mailman failure issue.  I was
thinkin' of potentially sending to the test list, but ...
I took the lazy/efficient route - and without putting test
email on non-test list ... just waited for other/additional list
traffic and ... fine - at least for another list on same host,
so that would be redundant with test list.  I didn't do direct
from my own SMTP sending server/client (maybe someday, but ...),
so, don't have logs/information from that, so I don't even know
for sure if it hit or attempted to hit the linuxmafia.com
host (and, unless folks get a "bounce" email or other such
failure, that's typically the case with most folks sending to
such lists).

But, I do have access to peek on linuxmafia.com ... so,
let's have a looksee.

Don't have sudo, but ...
$ hostname && ls -ld /var/log/exim*/
linuxmafia.com
drwxr-s--- 2 Debian-exim adm 4096 Mar 20 06:57 /var/log/exim4/
$
Least privilege principle ... I probably only need group adm to read
the relevant logs.
$ ls -dgn /var/log/exim*/
drwxr-s--- 2 4 4096 Mar 20 06:57 /var/log/exim4/
$
So ... the GID for group adm is 4.
$ id
uid=1004(mpaoli) gid=1004(mpaoli) groups=1004(mpaoli)
$ su - root -c 'exec perl -e '\''chdir(q(/));$(=4;$)=q(4  
1004);$<=1004;$>=1004;exec {q(/bin/bash)} (q(-bash));'\'''
Password:
linuxmafia:/$ id
uid=1004(mpaoli) gid=4(adm) groups=1004(mpaoli)
linuxmafia:/$ PS1='$ '; set -o vi
$ cd /var/log/exim4
$ ls -ltrd $(find . -type f ! -mtime +1 -print)
-rw-r----- 1 Debian-exim adm  83858 Mar 19 06:45 ./rejectlog.2.gz
-rw-r----- 1 Debian-exim adm  99119 Mar 19 06:51 ./mainlog.2.gz
-rw-r----- 1 Debian-exim adm 535195 Mar 20 06:53 ./rejectlog.1
-rw-r----- 1 Debian-exim adm 725401 Mar 20 06:56 ./mainlog.1
-rw-r----- 1 Debian-exim adm 169935 Mar 20 20:07 ./rejectlog
-rw-r----- 1 Debian-exim adm 378508 Mar 20 20:07 ./mainlog
$ fgrep -l 20210320130024.16699ms3hj4g15ys at webmail.rawbw.com mainlog rejectlog
mainlog
$ fgrep 20210320130024.16699ms3hj4g15ys at webmail.rawbw.com mainlog
2021-03-20 13:00:44 1lNhlq-00063n-2w SA: Action: scanned but message  
isn't spam: score=1.1 required=4.0 (scanned in 18/18 secs |  
Message-Id: 20210320130024.16699ms3hj4g15ys at webmail.rawbw.com). From  
<Michael.Paoli at cal.berkeley.edu> (host=shell1.rawbw.com  
[198.144.192.42]) for sf-lug at linuxmafia.com
2021-03-20 13:00:44 1lNhlq-00063n-2w <= Michael.Paoli at cal.berkeley.edu  
H=shell1.rawbw.com [198.144.192.42] U=root P=esmtp S=42863  
id=20210320130024.16699ms3hj4g15ys at webmail.rawbw.com
$
So, ... looks like the SMTP server ... exim4 on the linuxmafia.com host,
accepted the email.
Let's compare that to later one that made it to (conspire) list and was
both accepted and sent to the list:
$ fgrep -l -i Michael.Paoli at cal.berkeley.edu mainlog rejectlog
mainlog
$ sed -ne '1,/^2021-03-20 13:00:44/d;/^2021-03-20  
13:00:44/d;/Michael\.Paoli at cal\.berkeley\.edu/p' mainlog | head -n 3
2021-03-20 19:05:13 1lNnSf-0000UY-Sl SA: Action: scanned but message  
isn't spam: score=-2.6 required=4.0 (scanned in 11/11 secs |  
Message-Id: 20210320190500.16202y9tz98lm5e8 at webmail.rawbw.com). From  
<Michael.Paoli at cal.berkeley.edu> (host=shell1.rawbw.com  
[198.144.192.42]) for conspire at linuxmafia.com
2021-03-20 19:05:13 1lNnSf-0000UY-Sl <= Michael.Paoli at cal.berkeley.edu  
H=shell1.rawbw.com [198.144.192.42] U=root P=esmtp S=2046  
id=20210320190500.16202y9tz98lm5e8 at webmail.rawbw.com
2021-03-20 19:05:16 1lNnSu-0000Un-4D SA: Action: Not running SA  
because SAEximRunCond expanded to false (Message-Id:  
1lNnSu-0000Un-4D). From <conspire-bounces at linuxmafia.com>  
(host=localhost [127.0.0.1]) for
$
I cut off tail end of that 3rd log line, as it included subscribers sent
to (including also myself, hence the line matched).
So, looks like in the nominal case, two log entries coming in,
presumably a bit of SpamAsassin (SA) analysis, then handed off to
mailman, and then more log lines as it's sent out (I just showed one of
many of those, including the one that happened to also include my email
list, as a list subscriber).

So ... still doesn't quite explain what happened to the posting I
attempted to the SF-LUG list.
Let's check a bit more ... anything funky with my subscriber status
that may have caused issue (e.g. if it dropped me from the
list somehow, and treated it as from one not subscribed to the list)
... that all looks good, still subscribed, not in digest mode, delivery
enabled, etc.
So, back to those first pair of log messages ...
1lNhlq-00063n-2w
guessing that's some identifier in exim4's queuing/processing of the
message.

Let's see what else I can check.
I also still have access to the list administrator password for the
sf-lug list ... is it held in queue for some reason?
Checking there and find ... yep, it's held in queue ... but ... why?
Ah, ... here we go, for reason(s) ...:
Reason: Message body is too big: 41345 bytes with a limit of 40 KB

Let's see ... what did I have for the text file I used for body ...
$ stat -c %s disk_is_cheap
40519
$
SMTP use \r\n for line endings, so ...
$ todos < disk_is_cheap | wc -c
41541
$ ... well, somewhere in that range, approximates the 41345 bytes
mailman is complaining about.  Might also be some character encoding
bits that slightly change the size too.

Let's see if I can squeeze that down a wee bit ...
$ echo '(40000/41345)*40519' | bc -l
39200.87072197363647322609
$
If I get my original file down around or bit under that size, then it
likely makes it through ... and oh, yes, any listadmin could'a checked
on these mailman bits of it ... Jim? ...

Anyway.  Shall make it reasonably smaller and resend, and thus not
bother the listadmin(s) further on the matter.
... and, have trimmed that wee bit, so shall sent that bit after this.

And yeah, I probably should've checked the listadmin stuff first -
forgot I still had access to that.  (And no, I'm not exactly
volunteering to be listadmin for the sf-lug list ... but if/when nobody
else is able/willing to do it ... well, other than me and Rick, guess I
can be a "listadmin of last (or next to last) resort").

Also, speaking of which, any additional folk(s) want to volunteer to
(also) be listadmin?  I'm sure Jim could do with additional assistance
on it (and Rick would probably like to avoid being burdened with it
any more than necessary), so ... who else wants to step up to the plate?