[sf-lug] relocate SF-LUG list...etc. etc.

[aaronco36]

So the trivial command 'wc -w' (see 'man 1 wc' or ref [1]) run on the 
non-snippet portion of Rick M's previous subject post of [2] shows that 
there are approximately 215 [mostly] concisely-used words there.
OTOH, the same 'wc -w' run on Michael P's non-snippeted portion of [3] and 
repeated near the bottom below shows a thirteenfold increase to over 2800 
words.

Definitely tl;dr [4] !!
Summarized points of [3] -- perhaps even bulleted ??
And no initial search engine results _currently_ found for the apparently 
widespread use of "lazy^Wefficient" within the quite extensive 
bottom-quote :-\

(btw, 'wc -w' run on this message up to here shows <115 words ;-} )

-A


============================================================
References
============================================================
[1]https://www.man7.org/linux/man-pages/man1/wc.1.html
[2]http://linuxmafia.com/pipermail/sf-lug/2021q2/015301.html
[3]http://linuxmafia.com/pipermail/sf-lug/2021q2/015304.html
[4]https://www.urbandictionary.com/define.php?term=tl%3Bdr
============================================================


Bottom-quoting non-snippeted portion of [3]:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Definitely not a bad idea ... :-)
http://linuxmafia.com/pipermail/sf-lug/2021q2/015301.html

"But" ... ;-) ...

Us sysadmins(/listadmins), not speaking in entirety categorically, but at 
least statistically majority (more so than not ... or anecdotally so? 
And thus we've got a "reputation" to live up to / expectations to meet?) 
are ... lazy^Wefficient.  So ... relative (non-)criticality, timing, 
priorities, etc. ...

Notably at least "important"(-ish), if not critical, is that the list be 
backed up, and in manner in which its functionality (or at least most 
critical/important portions thereof) can be restored.  And that is 
actually pretty well covered.

As far as relevant bits of covering that - most important, the archive of 
all the list postings - in raw mbox format (or something that can 
reasonably easily be converted to that) - and have that well covered, and 
also the list membership - also have that well covered.  Those two alone, 
being reasonably well covered, makes it such that the list could be 
restored or migrate to just about anything list-wise, without too much 
difficulty.  Beyond that, there's various individual setting fiddly bits 
(sometimes even binary on/off bits/flags) for individual members, and also 
for the list itself - good/useful to also have those and have those backed 
up - but also not so critical.  Also, most of those are quite Mailman 
specific, and many even quite Mailman 2.x specific. But lets come back to 
those.

First the archive(s) and membership.  You, Rick, do some on-site backups
of those (at least including the raw mbox format archive), and I believe
also the roster and probably also the user and list settings (for
Mailman 2.x I think most of that is in one single file).  There's also
some miscellaneous bits like any customizations used on some of the web
pages, etc., I'm guestimating you've also got that covered.  In any case
at least the mbox archive file and roster are the most
important/critical, and the remainder also nice to have, but by no means
crucial.  In addition to your backups, there's also the backups I do for
SF-LUG.  On the matter of SF-LUG's list - both archive (raw mbox format)
and roster (list of members) - I cover that about daily (and thanks for
your assistance on also making that possible).  Both are generally
backed up overnightly - from linuxmafia.com to the BALUG VM (which
also hosts sf-lug.org, etc.).  In addition to those backups, I not only
"copy" the file/data over (the mbox file is actually updated more
efficiency via public rsync - thanks again to Rick making that
available), but more than just copy, I also use RCS - so I generally not
only have "copy", but generally to granularity of daily, have full
version history - at least going back fair number of years now - so
there's not "just" backup, but effectively years of daily versions
backed up and available.  And in addition to that, the BALUG VM's data
is also periodically replicated and also further backed up and backed up
to yet another off-site (relative to both linuxmafia.com and the BALUG
VM).  So, that's pretty good coverage (and more folks can also
additionally backup - and the archive is especially easy to do that on
since it's made publicly available).  Anyway, that's pretty good
coverage.

So ... as for additional fiddly bits (e.g. in that .pck (Python pickle 
format)), etc. ... and also lazy^Wefficient ... That file - and other 
fiddly bits - would be a "nice to have", but far from crucial/critical. 
The SF-LUG list has about 259 members: 
http://linuxmafia.com/pipermail/sf-lug/2021q2/015296.html
And, as I'm typing/drafting this, at current, from most recent detected 
changes on the daily "backup" ...

$ hostname && id && pwd -P && wc -l sf-lug_roster && ls -ld --full-time
sf-lug_roster
balug-sf-lug-v2.balug.org
uid=29774(sflug) gid=29774(sflug) groups=29774(sflug)
/home/sflug/sf-lug_roster
259 sf-lug_roster
-r-------- 1 sflug sflug 8176 2021-05-17 14:00:26.000000000 +0000
sf-lug_roster
$
That's still sitting at exactly 259 members and from the mtime, hasn't 
changed (I preserve the older mtime when the data has not changed). So, 
since the top-of-the-month reporting that data is thus far unchanged. 
So, much version change history - that's detail way beyond mere 
backup(/restore) requirements/data ... but nice to have.

Anyway, lazy^Wefficient and additional fiddly bits.
Let's take a worstish scenario.  Let's say a largeish but not too big 
highly charged asteroid crashes to Earth hitting your roof antenna and 
creating a localized EMP that takes out linuxmafia.com and its on-site 
backups, so the SF-LUG impact is localized to about only that.  Okay - 
then I've got archive and roster (even including "full name" at that), and 
- already having Mailman 2.x infrastructure and such (or even if by then 
it was only 3.x), could have list up and running and operational, with all 
the archives and members again, probably within about 48 hours - maybe 
sooner if I really wanted, or bit longer (say up to a week) if I was 
particularly busy and/or unmotivated at the time - not exactly a major 
disaster as far as recovered to state.  And sure, probably under different 
domain and URLs - but not a biggie.  And some fiddly bits (e.g. user 
preferences) would be lost ... but, in the grand scheme of things, also 
not a biggie.  It's also not like we're talkin' high hundreds, or 
thousands or more users might be moderately inconvenienced in not having 
their setting preferences restored - and would need to themselves go in 
and changes those if they wanted something other than the defaults.  But 
I'd guestimate probably over 90% are using defaults anyway ... so we'd 
probably only be slightly inconveniencing what - like about 26 users?  I 
don't think that's a biggie.  And even if all the details had been backed 
up, the domain would likely necessarily change - even if that change were 
not to be permanent - just to get things back up and running quite a bit 
sooner again, and avoid any entanglements/dependencies with 
linuxmafia.com. and any restore efforts that may or may not be going on 
there around the same time frame. Anyway, I think that's a not too 
horrible recovery scenario.  And, most scenarios that would take out 
linuxmafia.com and its on-site backups and also the BALUG VM availability 
and/or its restore capabilities (backups) would also in most probability 
remove or knock out the overwhelming majority of list members - in which 
case SF-LUG list restore would probably be relatively moot.

"But" ... more on lazy^Wefficient and more of those fiddly bits.
LUGs and the like, time, priorities, etc.
If I were going to imminently or "soon" be migrating the SF-LUG
list - especially from where it is to BALUG VM, or to be doing so
around same time I'll (eventually) be migrating other lists on
the BALUG VM (notably thus far BALUG.org lists, though some other
lists may join that, e.g. BUUG.org, BAD.debian.net, maybe/likely at
least eventually BerkeleyLUG.com's list), well, it would be most
lazy^Wefficient to migrate SF-LUG list around same time - it would
mostly just be iterating one more list to add to migration (to the
BALUG VM or from Mailman 2.x to 3.x).  Doing other/additional,
e.g working out .pck stuff at times other than about the same as that,
not as lazy^Wefficient, as the further apart in time, the more of a
one-off - and thus less lazy^Wefficient(/easy) that becomes.  And,
so - and also with SF-LUGers not exactly majorly pushing for a migration
http://linuxmafia.com/pipermail/sf-lug/2021q2/015298.html
I'm in no rush to prep it to be further "migration ready" for a more (or
closer to) seamless migration process.  It can get folded into such
process(es)/procedures, and in more lazy^Wefficient manner, when it's
closer to darn good and ready to be migrated, or folding into or
piggybacking on other migrations happening at/around same time.
So, in general, the closer in timing, the more lazy^Wefficient,
the further apart in time, the more work and less efficient.

lazy^Wefficient ... and "procrastination".  So, procrastination, or doing 
it somewhat later, for, e.g., systems administration, is also very much 
lazy^Wefficient ... at least to a point.  Let's say one has some certain 
project.  Let's say it needs happen at/around some specific future date - 
like some major migration or upgrade.  Okay.  Well, planning/preparing far 
too early in advance is inefficient/wasteful - things may need to change 
in the plans, maybe the update/upgrade gets outright cancelled or becomes 
moot, folks will forget, at least in part, and be less familiar with all 
the planned details, tools that were set up, data backed up and prepared 
becomes stale, things may need to be updated to adjust for changes in how 
things will need or want to be done, etc.  However, waiting 'till too 
late, is also not good - e.g. things get rushed, folks get stressed, 
mistakes get made, things don't go nearly so smoothly, issues/problems 
need to be fixed, etc.  And ... somewhere between is the "just right" 
Goldilocks sweet spot. Sufficiently well in advance that everything is 
well prepared and tested and there's enough wiggle room / slop built in 
for contingencies and unplanned, etc., so all is well prepared and tested 
when things are to be implemented, and everything is implemented well and 
smoothly ... and it's also not prepared too far in advance that folks 
forget and become unfamiliar with what they'd prepared for and much 
earlier documented the migration/upgrade procedure on, etc.  Anyway, so 
there's also that.  :-)

lazy^Wefficient ... and LUG and related time/priorities.  Yeah, sure,
the .pck file ... also having that backed up to off-site, sure, wouldn't
be a bad thing - but it's also bit more work to set up, as it contains
some sensitive-ish data (entire roster, user's shouldn't be important
Mailman passwords (but users being users ...)), etc., so wouldn't just
be a put it up publicly and grab it thing, but bit more work to securely
transfer (e.g. like the roster - that's securely encrypted before
transfer, so that full data is never exposed in the clear to the public
or Internet at large).  And, in the meantime, many items that, at least
I, see as higher LUGs (in general, not SF-LUG specific) priorities, e.g.
and not necessarily at all in this particular order - but probably all
higher priority than getting SF-LUG list's lesser fiddly bits:
BUUG & BAD lists
stabilization/support/maintainability/backups/redundancy (and web sites
too - though though do have backups - but the list archives and rosters
mostly don't - at least nothing off-site that I'm aware of),
there's the (probable) unGoogling of BerkeleyLUG's list,
there's significant work/improvements to be done to the BALUG VM's
email and list infrastructure - notably exim4 and improving anti-spam
(should quite improve much of the list bits that can be automated -
notably reject at SMTP level post attempts by those not authorized to
post to lists), being fully Mailman 3.x ready (have it installed -
partly/kind'a/mostly - but still have to complete that and fully
configure it and start testing and work out 2.x to 3.x migration plans).
Anyway, there's at least all that and much more (like when should we do
the next BALUG meeting, and will it include physical, and if so where,
and will we have speaker/presentation - and will it be yours truly yet
again - in which case I ought prepare some materials so it's better
talk/presentation/demonstration, rather than me just rambling/talking
about what I can on the topic off-the-top-of-my-head), etc.

Heh, ... not to mention life & non-LUG stuff, but ... whatever.

So ... were the/"my" LUGs & such priority list more clear/open/empty or 
had fewer higher priority items ... yeah, that .pck might be at or lots 
closer to the top of the list ... and over some time it may well make its 
way up there (e.g. after much of the email and Mailman stuff is in yet 
better shape) ... but as it presently stands ... eh ... not that close to 
the top of the list ... yet.  lazy^Wefficient ... yup.  :-)

Oh, and lest folks fret that "oh my gosh, then you're that much further 
away from having that covered".  Uhm, ... so long as linuxmafia.com is up, 
I've got access and could always snag a copy ... but it also becomes 
relatively quickly out-of-date ... though probably most of the data 
doesn't change all that frequently.  Also, a lot of relevant pieces 
(though not necessarily data) do exist.  E.g. not too many years back, did 
a Mailman 2.x to 2.x migration - notably DreamHost.com (ugh, more like 
NightmareHost, but whatever) to self-hosted ... and no way at all they'd 
give access to the .pck file (even raw archive mbox file was like pulling 
teeth to get that from them) ... so ... to migrate those lists, the 
largest of which well over 800 members - to make it as smooth and seamless 
as possible ... and did have web GUI listadmin access ... well, most of 
those fiddly bits (about all except users' passwords) could be sucked out 
via web GUI ... so ... I did that ... and along with Perl's 
WWW::Mechanize, etc. automated/tooled the heck out of that where feasible 
and it made sense ... notably many hundreds of users, short of their 
password, migrated over all of their setting preferences.  Anyway, still 
have all that code ... so it can always be used again - and especially for 
2.x to 2.x migration, though with some adaptations/additions, may also be 
highly suitable for 2.x to 3.x migration.  Oh, and at least in part, even 
better than that ... egad, Web GUI scraping and such?  Well, when one 
must, but ... .pck - more recently have quite the code to deal with that - 
at least in significant part.  Developed that recently for use in 
anti-spam - most notably to reject at SMTP time attempts to post to list 
where the address so attempting isn't authorized to post to that list. 
Anyway, the code that pulls that data straight out of the .pck - including 
not only roster for list, but also which members do/don't have the 
moderator bit/flag set on them ... yeah, ... that and all the other fiddly 
bits, for the most part not too hard to pull out of there and use.  And 
that'll likely also come in super handy for 2.x to 3.x migration - so 
expecting I'll likely be further developing that.

So, ... yep, ... .pck ... very good - even excellent idea, and not bad at 
all to (also fully) cover that.  But ... lazy^Wefficient, 
time/priorities/resources 'n all that ... okay, added to my LUG queue/list 
... but, at least presently, not that close to the top.

But hey, if someone else wants to take on getting that securely and 
regularly backed up to off-site (at least relative to linuxmafia.com), 
sure as heck don't let me stand in your way!  There are also even 
resources on the BALUG VM available, if one might want/prefer to get it to 
there (e.g. if one didn't already have a better more suitable place or 
wasn't able to easily set such up).

Hmmm ... there are some "to do" and the like list(s)/page(s)/section(s) on 
the BALUG wiki ... maybe I ought get around to doing some 
updating/organizing/consolidating - to make the more current more visible 
... e.g. general list(s) and various LUG(s) and such and across multiple 
LUG(s) and such in some cases, and more general overall stuff "vs." bits 
that I'm particularly interested in and willing and able to work on ("vs." 
bits I/we may need or want other folks to do or other folks may be more 
interested in and willing to do).  Or not so much - becomes yet one more 
thing to do and update and (attempt to) maintain.

"But that took so long to explain 'n all, why didn't you just" ... Because 
there's always "one more thing".  Hop to that "one more thing", then 
there's "But why didn't you do *my* one more thing, or *this* one more 
thing, or when will you get to *that* one more thing."  There's a whole 
lot 'o "one more thing"s in queue / on list, etc.  So ... on the list, but 
... and sometimes more appropriate to explain why that "one more thing" 
isn't immediately being hopped to, but rather added somewhere among the 
list of many "one more thing"s to do.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

aaronco36 at sdf.org
--