[conspire] fyi: Understanding the bin, sbin, usr/bin , usr/sbin split

[Rick Moen]

Quoting Nick Moffitt (nick at zork.net):

> Well, /opt was for a long time the GNU ghetto on proprietary unixes.  It
> was an optional parallel universe of system utilities and core
> libraries, bolted on as a bag of freedom on the side.
> 
> By the late 90s /opt had already taken on more of the connotation as the
> proprietary software ghetto on GNU systems.  Oracle dumped itself in
> /opt, and so did a lot of other big bolt-on apps for hefty Unix systems.
> So I like to look at /opt as a field I'm proud to leave fallow for as
> long as possible.

I somehow missed the protohistory about it being the longtime home of
the GNU tools.  It's been a long time, but I think Star Office was the
first thing that really brought that to my attention, FWIW (back when it
was still from Star Division).

> That said, I think that the complaints against making initrds
> self-sufficient are largely coming from people who don't trust distro
> kernel packages.  If we really do move to "everything you need is in the
> initrd" as a model, this stuff will get automated away via dpkg triggers
> and grub management scripts.

I like that idea -- and it may be the logical direction.  The notion of
using things like Parted Magic seems appealing on the surface, but
there's really nothing like having maintenance-mode utilities that you
are absolutely certain are appropriate to the local system because they 
are literally part of it.  (I tend, FWIW, to keep a separate 1GB
filesystem, normally unmounted but with /etc/fstab entry as /recovery,
that has a duplicate minimal Debian system, just as an additional 
recovery option.  That separate system, of course, likewise needs to 
be maintained.)

For the present, the appeal of a local set of such tools within /bin,
/sbin, and /lib is that it's a time-tested solution to a real problem
that works and is simple with known characteristics.  In a stressful
situation such as 'major filesystems refuse to mount at startup time', 
you don't want something clever and tricky.  You want it to be
bog-standard, familiar, present and working every time, and dead simple.


> What I find more interesting is the lore that /sbin was created to store
> static copies of critical binaries during a time when dynamic linking
> was still experimental and error-prone.  We now think of it as "sysadmin
> binaries" and the FHS codifies this somewhat, but legend has it that
> there used to be full copies of ls and sh and so forth in there.
> 
> Matthew Garrett pointed out in https://lwn.net/Articles/477534/
> > The / and /usr divide artificially split the platform in two, without
> > any clear semantics as to which subset of functionality you can
> > actually depend on. The world has got more complicated than it was
> > when the worst case was /usr being on an NFS server connected by
> > ethernet. Something has to change.
> 
> This final statement leaves him a lot of room to maneuver.  Pottering
> has proposed a solution, but Garrett just identified a problem.  It
> seems like Lennart would be the more constructive one in that situation,
> but he hasn't made his case in a way that convinces people yet.  He
> needs to define the problem he intends to solve a little better, or it
> will just seem like yet another thing that he wants to rewrite from
> scratch.

Yes, that seems a fair-minded summary.

In general, the seat-of-pants semantic distinction that was
traditionally observed between what's OK to be in /usr and what must not 
has tended to pass the functionality test every time it's been critical
on systems I've relied on.  (I'm trying to avoid sweeping statements; 
thus I'm speaking only about systems I've relied on.)  E.g., when there
was a problem mounting local /usr or NFS-mediated /usr, the remaining
utilities in /bin and /sbin included binaries for fsck, mount, umount,
etc. that were either statically linked or at minimum linked only to
libs in /lib rather than /usr/lib.  So, when I hear people like Lennart
and the freedesktop.org people saying 'separate /usr has been broken for
years', my first reaction is:  Bullshit.  Mine isn't.  If yours is,
consider fixing your damn bugs instead of claiming the problem is
obsolete and should not be fixed.

But even though the distinction between what's OK to be in /usr and what
must not be has seemed stable and functional enough for real-world 
diagnosis, repair, backup, and restore in my experience, Matthew is
probably right about it being poorly defined.

> Personally, I think it's a really minimal subset of people who still
> keep /usr on a separate filesystem from /.  It's pretty niche, these
> days, and a bit retro.  

Belt-and-suspenders people who not only do that but keep a separate,
normally unmounted /recovery partition are presumably fewer still.  If I
wanted to be gratuitously snide, I'd say we're the sysadmins as opposed
to prototype coders.