[sf-lug] relocate SF-LUG list...etc. etc.
aaronco36 at SDF.ORG
Sun Jun 6 08:37:01 PDT 2021
So the trivial command 'wc -w' (see 'man 1 wc' or ref ) run on the
non-snippet portion of Rick M's previous subject post of  shows that
there are approximately 215 [mostly] concisely-used words there.
OTOH, the same 'wc -w' run on Michael P's non-snippeted portion of  and
repeated near the bottom below shows a thirteenfold increase to over 2800
Definitely tl;dr  !!
Summarized points of  -- perhaps even bulleted ??
And no initial search engine results _currently_ found for the apparently
widespread use of "lazy^Wefficient" within the quite extensive
(btw, 'wc -w' run on this message up to here shows <115 words ;-} )
Bottom-quoting non-snippeted portion of :
Definitely not a bad idea ... :-)
"But" ... ;-) ...
Us sysadmins(/listadmins), not speaking in entirety categorically, but at
least statistically majority (more so than not ... or anecdotally so?
And thus we've got a "reputation" to live up to / expectations to meet?)
are ... lazy^Wefficient. So ... relative (non-)criticality, timing,
priorities, etc. ...
Notably at least "important"(-ish), if not critical, is that the list be
backed up, and in manner in which its functionality (or at least most
critical/important portions thereof) can be restored. And that is
actually pretty well covered.
As far as relevant bits of covering that - most important, the archive of
all the list postings - in raw mbox format (or something that can
reasonably easily be converted to that) - and have that well covered, and
also the list membership - also have that well covered. Those two alone,
being reasonably well covered, makes it such that the list could be
restored or migrate to just about anything list-wise, without too much
difficulty. Beyond that, there's various individual setting fiddly bits
(sometimes even binary on/off bits/flags) for individual members, and also
for the list itself - good/useful to also have those and have those backed
up - but also not so critical. Also, most of those are quite Mailman
specific, and many even quite Mailman 2.x specific. But lets come back to
First the archive(s) and membership. You, Rick, do some on-site backups
of those (at least including the raw mbox format archive), and I believe
also the roster and probably also the user and list settings (for
Mailman 2.x I think most of that is in one single file). There's also
some miscellaneous bits like any customizations used on some of the web
pages, etc., I'm guestimating you've also got that covered. In any case
at least the mbox archive file and roster are the most
important/critical, and the remainder also nice to have, but by no means
crucial. In addition to your backups, there's also the backups I do for
SF-LUG. On the matter of SF-LUG's list - both archive (raw mbox format)
and roster (list of members) - I cover that about daily (and thanks for
your assistance on also making that possible). Both are generally
backed up overnightly - from linuxmafia.com to the BALUG VM (which
also hosts sf-lug.org, etc.). In addition to those backups, I not only
"copy" the file/data over (the mbox file is actually updated more
efficiency via public rsync - thanks again to Rick making that
available), but more than just copy, I also use RCS - so I generally not
only have "copy", but generally to granularity of daily, have full
version history - at least going back fair number of years now - so
there's not "just" backup, but effectively years of daily versions
backed up and available. And in addition to that, the BALUG VM's data
is also periodically replicated and also further backed up and backed up
to yet another off-site (relative to both linuxmafia.com and the BALUG
VM). So, that's pretty good coverage (and more folks can also
additionally backup - and the archive is especially easy to do that on
since it's made publicly available). Anyway, that's pretty good
So ... as for additional fiddly bits (e.g. in that .pck (Python pickle
format)), etc. ... and also lazy^Wefficient ... That file - and other
fiddly bits - would be a "nice to have", but far from crucial/critical.
The SF-LUG list has about 259 members:
And, as I'm typing/drafting this, at current, from most recent detected
changes on the daily "backup" ...
$ hostname && id && pwd -P && wc -l sf-lug_roster && ls -ld --full-time
uid=29774(sflug) gid=29774(sflug) groups=29774(sflug)
-r-------- 1 sflug sflug 8176 2021-05-17 14:00:26.000000000 +0000
That's still sitting at exactly 259 members and from the mtime, hasn't
changed (I preserve the older mtime when the data has not changed). So,
since the top-of-the-month reporting that data is thus far unchanged.
So, much version change history - that's detail way beyond mere
backup(/restore) requirements/data ... but nice to have.
Anyway, lazy^Wefficient and additional fiddly bits.
Let's take a worstish scenario. Let's say a largeish but not too big
highly charged asteroid crashes to Earth hitting your roof antenna and
creating a localized EMP that takes out linuxmafia.com and its on-site
backups, so the SF-LUG impact is localized to about only that. Okay -
then I've got archive and roster (even including "full name" at that), and
- already having Mailman 2.x infrastructure and such (or even if by then
it was only 3.x), could have list up and running and operational, with all
the archives and members again, probably within about 48 hours - maybe
sooner if I really wanted, or bit longer (say up to a week) if I was
particularly busy and/or unmotivated at the time - not exactly a major
disaster as far as recovered to state. And sure, probably under different
domain and URLs - but not a biggie. And some fiddly bits (e.g. user
preferences) would be lost ... but, in the grand scheme of things, also
not a biggie. It's also not like we're talkin' high hundreds, or
thousands or more users might be moderately inconvenienced in not having
their setting preferences restored - and would need to themselves go in
and changes those if they wanted something other than the defaults. But
I'd guestimate probably over 90% are using defaults anyway ... so we'd
probably only be slightly inconveniencing what - like about 26 users? I
don't think that's a biggie. And even if all the details had been backed
up, the domain would likely necessarily change - even if that change were
not to be permanent - just to get things back up and running quite a bit
sooner again, and avoid any entanglements/dependencies with
linuxmafia.com. and any restore efforts that may or may not be going on
there around the same time frame. Anyway, I think that's a not too
horrible recovery scenario. And, most scenarios that would take out
linuxmafia.com and its on-site backups and also the BALUG VM availability
and/or its restore capabilities (backups) would also in most probability
remove or knock out the overwhelming majority of list members - in which
case SF-LUG list restore would probably be relatively moot.
"But" ... more on lazy^Wefficient and more of those fiddly bits.
LUGs and the like, time, priorities, etc.
If I were going to imminently or "soon" be migrating the SF-LUG
list - especially from where it is to BALUG VM, or to be doing so
around same time I'll (eventually) be migrating other lists on
the BALUG VM (notably thus far BALUG.org lists, though some other
lists may join that, e.g. BUUG.org, BAD.debian.net, maybe/likely at
least eventually BerkeleyLUG.com's list), well, it would be most
lazy^Wefficient to migrate SF-LUG list around same time - it would
mostly just be iterating one more list to add to migration (to the
BALUG VM or from Mailman 2.x to 3.x). Doing other/additional,
e.g working out .pck stuff at times other than about the same as that,
not as lazy^Wefficient, as the further apart in time, the more of a
one-off - and thus less lazy^Wefficient(/easy) that becomes. And,
so - and also with SF-LUGers not exactly majorly pushing for a migration
I'm in no rush to prep it to be further "migration ready" for a more (or
closer to) seamless migration process. It can get folded into such
process(es)/procedures, and in more lazy^Wefficient manner, when it's
closer to darn good and ready to be migrated, or folding into or
piggybacking on other migrations happening at/around same time.
So, in general, the closer in timing, the more lazy^Wefficient,
the further apart in time, the more work and less efficient.
lazy^Wefficient ... and "procrastination". So, procrastination, or doing
it somewhat later, for, e.g., systems administration, is also very much
lazy^Wefficient ... at least to a point. Let's say one has some certain
project. Let's say it needs happen at/around some specific future date -
like some major migration or upgrade. Okay. Well, planning/preparing far
too early in advance is inefficient/wasteful - things may need to change
in the plans, maybe the update/upgrade gets outright cancelled or becomes
moot, folks will forget, at least in part, and be less familiar with all
the planned details, tools that were set up, data backed up and prepared
becomes stale, things may need to be updated to adjust for changes in how
things will need or want to be done, etc. However, waiting 'till too
late, is also not good - e.g. things get rushed, folks get stressed,
mistakes get made, things don't go nearly so smoothly, issues/problems
need to be fixed, etc. And ... somewhere between is the "just right"
Goldilocks sweet spot. Sufficiently well in advance that everything is
well prepared and tested and there's enough wiggle room / slop built in
for contingencies and unplanned, etc., so all is well prepared and tested
when things are to be implemented, and everything is implemented well and
smoothly ... and it's also not prepared too far in advance that folks
forget and become unfamiliar with what they'd prepared for and much
earlier documented the migration/upgrade procedure on, etc. Anyway, so
there's also that. :-)
lazy^Wefficient ... and LUG and related time/priorities. Yeah, sure,
the .pck file ... also having that backed up to off-site, sure, wouldn't
be a bad thing - but it's also bit more work to set up, as it contains
some sensitive-ish data (entire roster, user's shouldn't be important
Mailman passwords (but users being users ...)), etc., so wouldn't just
be a put it up publicly and grab it thing, but bit more work to securely
transfer (e.g. like the roster - that's securely encrypted before
transfer, so that full data is never exposed in the clear to the public
or Internet at large). And, in the meantime, many items that, at least
I, see as higher LUGs (in general, not SF-LUG specific) priorities, e.g.
and not necessarily at all in this particular order - but probably all
higher priority than getting SF-LUG list's lesser fiddly bits:
BUUG & BAD lists
stabilization/support/maintainability/backups/redundancy (and web sites
too - though though do have backups - but the list archives and rosters
mostly don't - at least nothing off-site that I'm aware of),
there's the (probable) unGoogling of BerkeleyLUG's list,
there's significant work/improvements to be done to the BALUG VM's
email and list infrastructure - notably exim4 and improving anti-spam
(should quite improve much of the list bits that can be automated -
notably reject at SMTP level post attempts by those not authorized to
post to lists), being fully Mailman 3.x ready (have it installed -
partly/kind'a/mostly - but still have to complete that and fully
configure it and start testing and work out 2.x to 3.x migration plans).
Anyway, there's at least all that and much more (like when should we do
the next BALUG meeting, and will it include physical, and if so where,
and will we have speaker/presentation - and will it be yours truly yet
again - in which case I ought prepare some materials so it's better
talk/presentation/demonstration, rather than me just rambling/talking
about what I can on the topic off-the-top-of-my-head), etc.
Heh, ... not to mention life & non-LUG stuff, but ... whatever.
So ... were the/"my" LUGs & such priority list more clear/open/empty or
had fewer higher priority items ... yeah, that .pck might be at or lots
closer to the top of the list ... and over some time it may well make its
way up there (e.g. after much of the email and Mailman stuff is in yet
better shape) ... but as it presently stands ... eh ... not that close to
the top of the list ... yet. lazy^Wefficient ... yup. :-)
Oh, and lest folks fret that "oh my gosh, then you're that much further
away from having that covered". Uhm, ... so long as linuxmafia.com is up,
I've got access and could always snag a copy ... but it also becomes
relatively quickly out-of-date ... though probably most of the data
doesn't change all that frequently. Also, a lot of relevant pieces
(though not necessarily data) do exist. E.g. not too many years back, did
a Mailman 2.x to 2.x migration - notably DreamHost.com (ugh, more like
NightmareHost, but whatever) to self-hosted ... and no way at all they'd
give access to the .pck file (even raw archive mbox file was like pulling
teeth to get that from them) ... so ... to migrate those lists, the
largest of which well over 800 members - to make it as smooth and seamless
as possible ... and did have web GUI listadmin access ... well, most of
those fiddly bits (about all except users' passwords) could be sucked out
via web GUI ... so ... I did that ... and along with Perl's
WWW::Mechanize, etc. automated/tooled the heck out of that where feasible
and it made sense ... notably many hundreds of users, short of their
password, migrated over all of their setting preferences. Anyway, still
have all that code ... so it can always be used again - and especially for
2.x to 2.x migration, though with some adaptations/additions, may also be
highly suitable for 2.x to 3.x migration. Oh, and at least in part, even
better than that ... egad, Web GUI scraping and such? Well, when one
must, but ... .pck - more recently have quite the code to deal with that -
at least in significant part. Developed that recently for use in
anti-spam - most notably to reject at SMTP time attempts to post to list
where the address so attempting isn't authorized to post to that list.
Anyway, the code that pulls that data straight out of the .pck - including
not only roster for list, but also which members do/don't have the
moderator bit/flag set on them ... yeah, ... that and all the other fiddly
bits, for the most part not too hard to pull out of there and use. And
that'll likely also come in super handy for 2.x to 3.x migration - so
expecting I'll likely be further developing that.
So, ... yep, ... .pck ... very good - even excellent idea, and not bad at
all to (also fully) cover that. But ... lazy^Wefficient,
time/priorities/resources 'n all that ... okay, added to my LUG queue/list
... but, at least presently, not that close to the top.
But hey, if someone else wants to take on getting that securely and
regularly backed up to off-site (at least relative to linuxmafia.com),
sure as heck don't let me stand in your way! There are also even
resources on the BALUG VM available, if one might want/prefer to get it to
there (e.g. if one didn't already have a better more suitable place or
wasn't able to easily set such up).
Hmmm ... there are some "to do" and the like list(s)/page(s)/section(s) on
the BALUG wiki ... maybe I ought get around to doing some
updating/organizing/consolidating - to make the more current more visible
... e.g. general list(s) and various LUG(s) and such and across multiple
LUG(s) and such in some cases, and more general overall stuff "vs." bits
that I'm particularly interested in and willing and able to work on ("vs."
bits I/we may need or want other folks to do or other folks may be more
interested in and willing to do). Or not so much - becomes yet one more
thing to do and update and (attempt to) maintain.
"But that took so long to explain 'n all, why didn't you just" ... Because
there's always "one more thing". Hop to that "one more thing", then
there's "But why didn't you do *my* one more thing, or *this* one more
thing, or when will you get to *that* one more thing." There's a whole
lot 'o "one more thing"s in queue / on list, etc. So ... on the list, but
... and sometimes more appropriate to explain why that "one more thing"
isn't immediately being hopped to, but rather added somewhere among the
list of many "one more thing"s to do.
aaronco36 at sdf.org
More information about the sf-lug