[sf-lug] relocate SF-LUG list from linuxmafia.com (on linuxmafia.com) to lists.sf-lug.org (on BALUG VM)?

[Michael Paoli]

> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: Re: [sf-lug] relocate SF-LUG list from linuxmafia.com (on  
> linuxmafia.com)	to lists.sf-lug.org (on BALUG VM)?
> Date: Thu, 3 Jun 2021 11:14:53 -0700

> Quoting Michael Paoli (Michael.Paoli at cal.berkeley.edu):
>
>> And ... survey closed and results:
>> https://www.surveymonkey.com/stories/SM-MGTQRS5C/
>> 12 responses (4.6% of 259 list members)
>> 3 (25%) move SF-LUG list to sf-lug at lists.sf-lug.org and archives to
>>         lists.sf-lug.org (hosted on BALUG virtual machine)
>> 9 (75%) keep SF-LUG list at sf-lug at linuxmafia.com and archives on
>>         linuxmafia.com (hosted on linuxmafia.com)
>> http://linuxmafia.com/pipermail/sf-lug/2021q2/015296.html
>>
>> Certainly no overwhelming mandate to move it, or even close,
>> so, shall remain where it is.
>
> However, you might want to create a setup on lists.sf-lug.org with a
> periodically refreshed copy of the mailing list's cumulative mbox and a
> copy of /var/lib/mailman/lists/sf-lug/config.pck.  ISTR that a
> functional mailing list can be quickly migrated using just those two
> files.  (I might have misremembered a detail or two[1], but we could
> figure that out, if you're interested.)

Definitely not a bad idea ... :-)
http://linuxmafia.com/pipermail/sf-lug/2021q2/015301.html

"But" ... ;-) ...

Us sysadmins(/listadmins), not speaking in entirety categorically, but
at least statistically majority (more so than not ... or anecdotally
so?  And thus we've got a "reputation" to live up to / expectations to
meet?) are ...
lazy^Wefficient.  So ... relative (non-)criticality, timing, priorities,
etc. ...

Notably at least "important"(-ish), if not critical, is that the list be
backed up, and in manner in which its functionality (or at least most
critical/important portions thereof) can be restored.  And that is
actually pretty well covered.

As far as relevant bits of covering that - most important, the archive
of all the list postings - in raw mbox format (or something that can
reasonably easily be converted to that) - and have that well covered,
and also the list membership - also have that well covered.  Those two
alone, being reasonably well covered, makes it such that the list could
be restored or migrate to just about anything list-wise, without too
much difficulty.  Beyond that, there's various individual setting fiddly
bits (sometimes even binary on/off bits/flags) for individual members,
and also for the list itself - good/useful to also have those and have
those backed up - but also not so critical.  Also, most of those are
quite Mailman specific, and many even quite Mailman 2.x specific.
But lets come back to those.

First the archive(s) and membership.  You, Rick, do some on-site backups
of those (at least including the raw mbox format archive), and I believe
also the roster and probably also the user and list settings (for
Mailman 2.x I think most of that is in one single file).  There's also
some miscellaneous bits like any customizations used on some of the web
pages, etc., I'm guestimating you've also got that covered.  In any case
at least the mbox archive file and roster are the most
important/critical, and the remainder also nice to have, but by no means
crucial.  In addition to your backups, there's also the backups I do for
SF-LUG.  On the matter of SF-LUG's list - both archive (raw mbox format)
and roster (list of members) - I cover that about daily (and thanks for
your assistance on also making that possible).  Both are generally
backed up overnightly - from linuxmafia.com to the BALUG VM (which
also hosts sf-lug.org, etc.).  In addition to those backups, I not only
"copy" the file/data over (the mbox file is actually updated more
efficiency via public rsync - thanks again to Rick making that
available), but more than just copy, I also use RCS - so I generally not
only have "copy", but generally to granularity of daily, have full
version history - at least going back fair number of years now - so
there's not "just" backup, but effectively years of daily versions
backed up and available.  And in addition to that, the BALUG VM's data
is also periodically replicated and also further backed up and backed up
to yet another off-site (relative to both linuxmafia.com and the BALUG
VM).  So, that's pretty good coverage (and more folks can also
additionally backup - and the archive is especially easy to do that on
since it's made publicly available).  Anyway, that's pretty good
coverage.

So ... as for additional fiddly bits (e.g. in that .pck (Python pickle
format)), etc. ... and also lazy^Wefficient ...
That file - and other fiddly bits - would be a "nice to have", but far
from crucial/critical.  The SF-LUG list has about 259 members:
http://linuxmafia.com/pipermail/sf-lug/2021q2/015296.html
And, as I'm typing/drafting this, at current, from most recent
detected changes on the daily "backup" ...
$ hostname && id && pwd -P && wc -l sf-lug_roster && ls -ld --full-time
sf-lug_roster
balug-sf-lug-v2.balug.org
uid=29774(sflug) gid=29774(sflug) groups=29774(sflug)
/home/sflug/sf-lug_roster
259 sf-lug_roster
-r-------- 1 sflug sflug 8176 2021-05-17 14:00:26.000000000 +0000  
sf-lug_roster
$
That's still sitting at exactly 259 members and from the mtime, hasn't
changed (I preserve the older mtime when the data has not changed).
So, since the top-of-the-month reporting that data is thus far
unchanged.  So, much version change history - that's detail way beyond
mere backup(/restore) requirements/data ... but nice to have.

Anyway, lazy^Wefficient and additional fiddly bits.
Let's take a worstish scenario.  Let's say a largeish but not too big
highly charged asteroid crashes to Earth hitting your roof antenna and
creating a localized EMP that takes out linuxmafia.com and its on-site
backups, so the SF-LUG impact is localized to about only that.  Okay -
then I've got archive and roster (even including "full name" at that),
and - already having Mailman 2.x infrastructure and such (or even if by
then it was only 3.x), could have list up and running and operational,
with all the archives and members again, probably within about 48 hours
- maybe sooner if I really wanted, or bit longer (say up to a week) if I
was particularly busy and/or unmotivated at the time - not exactly a
major disaster as far as recovered to state.  And sure, probably under
different domain and URLs - but not a biggie.  And some fiddly bits
(e.g. user preferences) would be lost ... but, in the grand scheme of
things, also not a biggie.  It's also not like we're talkin' high
hundreds, or thousands or more users might be moderately inconvenienced
in not having their setting preferences restored - and would need to
themselves go in and changes those if they wanted something other than
the defaults.  But I'd guestimate probably over 90% are using defaults
anyway ... so we'd probably only be slightly inconveniencing what -
like about 26 users?  I don't think that's a biggie.  And even if all
the details had been backed up, the domain would likely necessarily
change - even if that change were not to be permanent - just to get
things back up and running quite a bit sooner again, and avoid any
entanglements/dependencies with linuxmafia.com. and any restore efforts
that may or may not be going on there around the same time frame.
Anyway, I think that's a not too horrible recovery scenario.  And, most
scenarios that would take out linuxmafia.com and its on-site backups
and also the BALUG VM availability and/or its restore capabilities
(backups) would also in most probability remove or knock out the
overwhelming majority of list members - in which case SF-LUG list
restore would probably be relatively moot.

"But" ... more on lazy^Wefficient and more of those fiddly bits.
LUGs and the like, time, priorities, etc.
If I were going to imminently or "soon" be migrating the SF-LUG
list - especially from where it is to BALUG VM, or to be doing so
around same time I'll (eventually) be migrating other lists on
the BALUG VM (notably thus far BALUG.org lists, though some other
lists may join that, e.g. BUUG.org, BAD.debian.net, maybe/likely at
least eventually BerkeleyLUG.com's list), well, it would be most
lazy^Wefficient to migrate SF-LUG list around same time - it would
mostly just be iterating one more list to add to migration (to the
BALUG VM or from Mailman 2.x to 3.x).  Doing other/additional,
e.g working out .pck stuff at times other than about the same as that,
not as lazy^Wefficient, as the further apart in time, the more of a
one-off - and thus less lazy^Wefficient(/easy) that becomes.  And,
so - and also with SF-LUGers not exactly majorly pushing for a migration
http://linuxmafia.com/pipermail/sf-lug/2021q2/015298.html
I'm in no rush to prep it to be further "migration ready" for a more (or
closer to) seamless migration process.  It can get folded into such
process(es)/procedures, and in more lazy^Wefficient manner, when it's
closer to darn good and ready to be migrated, or folding into or
piggybacking on other migrations happening at/around same time.
So, in general, the closer in timing, the more lazy^Wefficient,
the further apart in time, the more work and less efficient.

lazy^Wefficient ... and "procrastination".  So, procrastination, or
doing it somewhat later, for, e.g., systems administration, is also
very much lazy^Wefficient ... at least to a point.  Let's say one has
some certain project.  Let's say it needs happen at/around some specific
future date - like some major migration or upgrade.  Okay.  Well,
planning/preparing far too early in advance is inefficient/wasteful -
things may need to change in the plans, maybe the update/upgrade gets
outright cancelled or becomes moot, folks will forget, at least in part,
and be less familiar with all the planned details, tools that were set
up, data backed up and prepared becomes stale, things may need to be
updated to adjust for changes in how things will need or want to be
done, etc.  However, waiting 'till too late, is also not good - e.g.
things get rushed, folks get stressed, mistakes get made, things don't
go nearly so smoothly, issues/problems need to be fixed, etc.  And ...
somewhere between is the "just right" Goldilocks sweet spot.
Sufficiently well in advance that everything is well prepared and tested
and there's enough wiggle room / slop built in for contingencies and
unplanned, etc., so all is well prepared and tested when things are to
be implemented, and everything is implemented well and smoothly ... and
it's also not prepared too far in advance that folks forget and become
unfamiliar with what they'd prepared for and much earlier documented the
migration/upgrade procedure on, etc.  Anyway, so there's also that.  :-)

lazy^Wefficient ... and LUG and related time/priorities.  Yeah, sure,
the .pck file ... also having that backed up to off-site, sure, wouldn't
be a bad thing - but it's also bit more work to set up, as it contains
some sensitive-ish data (entire roster, user's shouldn't be important
Mailman passwords (but users being users ...)), etc., so wouldn't just
be a put it up publicly and grab it thing, but bit more work to securely
transfer (e.g. like the roster - that's securely encrypted before
transfer, so that full data is never exposed in the clear to the public
or Internet at large).  And, in the meantime, many items that, at least
I, see as higher LUGs (in general, not SF-LUG specific) priorities, e.g.
and not necessarily at all in this particular order - but probably all
higher priority than getting SF-LUG list's lesser fiddly bits:
BUUG & BAD lists
stabilization/support/maintainability/backups/redundancy (and web sites
too - though though do have backups - but the list archives and rosters
mostly don't - at least nothing off-site that I'm aware of),
there's the (probable) unGoogling of BerkeleyLUG's list,
there's significant work/improvements to be done to the BALUG VM's
email and list infrastructure - notably exim4 and improving anti-spam
(should quite improve much of the list bits that can be automated -
notably reject at SMTP level post attempts by those not authorized to
post to lists), being fully Mailman 3.x ready (have it installed -
partly/kind'a/mostly - but still have to complete that and fully
configure it and start testing and work out 2.x to 3.x migration plans).
Anyway, there's at least all that and much more (like when should we do
the next BALUG meeting, and will it include physical, and if so where,
and will we have speaker/presentation - and will it be yours truly yet
again - in which case I ought prepare some materials so it's better
talk/presentation/demonstration, rather than me just rambling/talking
about what I can on the topic off-the-top-of-my-head), etc.

Heh, ... not to mention life & non-LUG stuff, but ... whatever.

So ... were the/"my" LUGs & such priority list more clear/open/empty or
had fewer higher priority items ... yeah, that .pck might be at or lots
closer to the top of the list ... and over some time it may well make
its way up there (e.g. after much of the email and Mailman stuff is in
yet better shape) ... but as it presently stands ... eh ... not that
close to the top of the list ... yet.  lazy^Wefficient ... yup.  :-)

Oh, and lest folks fret that "oh my gosh, then you're that much further
away from having that covered".  Uhm, ... so long as linuxmafia.com is
up, I've got access and could always snag a copy ... but it also becomes
relatively quickly out-of-date ... though probably most of the data
doesn't change all that frequently.  Also, a lot of relevant pieces
(though not necessarily data) do exist.  E.g. not too many years back,
did a Mailman 2.x to 2.x migration - notably DreamHost.com (ugh, more
like NightmareHost, but whatever) to self-hosted ... and no way at all
they'd give access to the .pck file (even raw archive mbox file was like
pulling teeth to get that from them) ... so ... to migrate those lists,
the largest of which well over 800 members - to make it as smooth and
seamless as possible ... and did have web GUI listadmin access ... well,
most of those fiddly bits (about all except users' passwords) could be
sucked out via web GUI ... so ... I did that ... and along with Perl's
WWW::Mechanize, etc. automated/tooled the heck out of that where
feasible and it made sense ... notably many hundreds of users, short of
their password, migrated over all of their setting preferences.  Anyway,
still have all that code ... so it can always be used again - and
especially for 2.x to 2.x migration, though with some
adaptations/additions, may also be highly suitable for 2.x to 3.x
migration.  Oh, and at least in part, even better than that ... egad,
Web GUI scraping and such?  Well, when one must, but ... .pck - more
recently have quite the code to deal with that - at least in significant
part.  Developed that recently for use in anti-spam - most notably to
reject at SMTP time attempts to post to list where the address so
attempting isn't authorized to post to that list.  Anyway, the code that
pulls that data straight out of the .pck - including not only roster for
list, but also which members do/don't have the moderator bit/flag set on
them ... yeah, ... that and all the other fiddly bits, for the most part
not too hard to pull out of there and use.  And that'll likely also come
in super handy for 2.x to 3.x migration - so expecting I'll likely be
further developing that.

So, ... yep, ... .pck ... very good - even excellent idea, and not bad
at all to (also fully) cover that.  But ... lazy^Wefficient,
time/priorities/resources 'n all that ... okay, added to my LUG
queue/list ... but, at least presently, not that close to the top.

But hey, if someone else wants to take on getting that securely and
regularly backed up to off-site (at least relative to linuxmafia.com),
sure as heck don't let me stand in your way!  There are also even
resources on the BALUG VM available, if one might want/prefer to get it
to there (e.g. if one didn't already have a better more suitable place
or wasn't able to easily set such up).

Hmmm ... there are some "to do" and the like list(s)/page(s)/section(s)
on the BALUG wiki ... maybe I ought get around to doing some
updating/organizing/consolidating - to make the more current more
visible ... e.g. general list(s) and various LUG(s) and such and across
multiple LUG(s) and such in some cases, and more general overall stuff
"vs." bits that I'm particularly interested in and willing and able to
work on ("vs." bits I/we may need or want other folks to do or other
folks may be more interested in and willing to do).  Or not so much -
becomes yet one more thing to do and update and (attempt to) maintain.

"But that took so long to explain 'n all, why didn't you just" ...
Because there's always "one more thing".  Hop to that "one more thing",
then there's "But why didn't you do *my* one more thing, or *this* one
more thing, or when will you get to *that* one more thing."  There's a
whole lot 'o "one more thing"s in queue / on list, etc.  So ... on the
list, but ... and sometimes more appropriate to explain why that "one
more thing" isn't immediately being hopped to, but rather added
somewhere among the list of many "one more thing"s to do.

> That would be a useful exercise as both disaster planning and as an
> exercise in planned migration of a Mailman 2.x mailing list.  (Of
> course, it's also true that Mailman 2.x is now EOL and increasingly
> moldy, and we all need to think about whether to migrate to 3.x or
> something else.  Anyway, my other point is that developing one's
> expertise with Mailman 2.x in 2021 is like getting better at Visicalc
> and WordStar.)
>
>
> [1] For example, I cannot remember offhand whether the membership roster
> is encoded into config.pck or stored somewhere else.  config.pck is a
> Python 'pickled' data file, so not plaintext.)
>
> Hmm, let's see:
>
> linuxmafia:/var/lib/mailman# bin/dumpdb lists/sf-lug/config.pck | less
> [output omitted]
>
> Yes, the roster is indeed kept in there, and lots of other interesting
> details including each subscriber's per-subscriber membership password.
> So, I'm pretty sure _all_ aspects of the mailing list state are
> captured in, and recoverable upon migration from, the config.pck file
> and the cumulative mbox.