[sf-lug] Linux distributions' package ... take advantage of if feasible (was: Malware on PyPI repository)

[Michael Paoli]

> Linux distributions' package
> take advantage of if feasible

Yes, generally if, and, as to the extent feasible.  And be cautious and
prudent if, when, and to the extent one goes around or circumvents stuff.
But, sometimes a bit of that might be warranted, however that should
generally be minimized as feasible.  After all, there are reasons you're
running a distro, right?  I presume you didn't roll your own by by doing
Linux From Scratch or the like.

So, reminds me bit of a certain story ... I forget some of the details -
was over half a dozen years ago ... in any case ...

So, at the time, I was doing work at a fairly large company, various
tasks and projects - essentially working there as a *nix sysadmin.
Don't need to name the company, but ... large US power/utility
company that occasionally blows things up or sets them on fire,
has twice gone bankrupt, and in recent years has pled guilty to multiple
felony charges.  In any case, they had a particular technical challenge
to solve, which they handed off to me.

So, the challenge was, they were running a certain somewhat older
Operating System (OS) version ... I don't recall the exact version,
but let's say Red Hat N ... then still supported, but, but a bit old, with
N+1 being the current major version.  Well for various technical reasons, it
wasn't feasible form them to run N+1 - at least for the particular target
server installations.  And, they had an application that required Perl,
and certain Perl modules, that they needed to port to that platform.
It was at the time running on Solaris.  They had everything they needed
to port it over ... except for some of the Perl fiddly bits.
Notably they required some certain Perl modules.  Well, there was a
Perl module or two, which Red Hat and their various supported and
more-or-less-reasonably supported respositories just didn't make one or
two of the needed Perl modules available.  So ... how to solve that,
and make it reasonably supportable.

One of the suggestions was just grab all the relevant source for Perl
and all the needed modules, and compile it, and have it locally
installed.  I thought that a relatively bad idea, given the size and
complexity of the code - as there will be security issues and important
bugs to be fixed.  That might be roughly equivalent to "just download
the source code for that current modern GUI web browser, compile it
yourself and locally install it ... and keep maintaining it yourself
for security and important bugs to be fixed."  Uhm, yeah, not the
way to go.

So ... I rather took a hybrid approach ... and at that, a forward-compatible
hybrid approach.  The modules that weren't supported by Red Hat's
repositories - I downloaded those - and only those - was only a module
or two ... think it may have only been one?  In any case, I compiled
that/those modules - and only those bits - nothing else.  And,
furthermore, I compiled them for that existing Red Hat N environment and
such, so those modules could be fully used with that ... with one
slight twist.  I didn't install them in the standard Red Hat N location.
Rather I installed them in a specific local location, so they wouldn't be
on Perl's @INC by default, but they could be included and used by
adjusting @INC.  And I didn't even change the default @INC.
Instead, what I did, is I set up the program that needed the module(s)
to test if it was able to load them ... and only if it failed to load
them, would it then adjust @INC, so that it would then be able to find
the local module(s).  And by so doing, if that Red Hat were ever upgraded,
possibly even major version upgrade, or those modules otherwise became
supported and added through standard Red Hat repositories, the program
would find and use those standard Red Hat supported ones - rather than
the local ones.

So ... almost all of Perl would be running the Perl and modules from the
Red Hat repositories.  The only Perl bits not from that would be the
local modules - and those would only be used from the local versions if
they also/still didn't exist from the standard Red Hat installed modules
and their standard default locations.  This had key advantages of
greatly reducing the volume of independent code to be maintained (just
one or two quite small modules), while allowing Red Hat to cover and
support the vast overwhelming majority of all the Perl bits.  And it
also had the forwards-compatibility that if in future, those modules
where provided by Red Hat and installed, the program would automatically
default to using those, and not even have need or reason to load
the local ones or even search where they were installed.

And, other interesting side bit.  I did this over about 2 or 3 days,
while also working on various other items.  After I'd tested it,
completed and demonstrated it, wee bit later manager approached me,
and informed me, essentially quite saying, "Uhm, ... we brought in
a team of 3 contractors/consultants to figure out and resolve that
challenge so we could reasonably port that software over, ... they
worked on it for an entire month ... and weren't able to come up
with any solution that worked."

Anyway, as feasible, stay within one's distro's package management
system and repos.  If when there's warranted need to go outside of
that, try to keep it as minimal as feasible.

> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: Re: [sf-lug] Malware on PyPI repository
> Date: Sun, 5 Dec 2021 18:59:46 -0800

> I tried to articulate the nuances on that problem years ago, when I was
> one of _Linux Gazette's_ main editors , at
> http://linuxmafia.com/~rick/weatherwax.html#1 :
>
>   All of the above are problems normally addressed (and the burden of
>   solving them, shouldered) by Linux distributions' package maintainers,
>   so that you won't have to. It's to your advantage to take advantage of
>   that effort, if feasible. The memory of when a thousand Linux sysadmins,
>   circa 1993, would need to do all of that work 999-times redundantly, is
>   still fresh to us old-timers: We call those the Bad Old Days, given that
>   today one expert package maintainer can instead do that task for a
>   thousand sysadmins. And yes, sometimes there's nothing like such a
>   package available, and you have no reasonable alternative but to grab
>   upstream source tarballs — but the disadvantages justify some pains to
>   search for suitable packages, instead.