[sf-lug] multichown (SF-LUG & BALUG: System OS upgrades)

[Michael Paoli]

[include: "SF-LUG & BALUG: System OS upgrades" (less quotes) within
Subject: header for folks to follow on this topic]

As mentioned at Noisebridge Wednesday evening, multichown:

From: Michael Paoli
Sent: Wednesday, December 21, 2011 8:02 AM
Subject: quite useful systems administration tool for fixing up ID
   conflicts: multichown

Got UID/GID conflicts?  Want to correct them reasonably?  Need to
preserve that existing system's data?  (not feasible to just blow it
all away and start from scratch).
Have a look a multichown - utility I wrote some years ago.
It's very handy for doing most of the "heavy lifting" in correcting
UID/GID alignments.
It's rather like chown(1), but much more capable of handling quite
complex non-overlapping ID reassignments in a single pass (for
untangling overlapping ID reassignments, at most two passes would be
used with separate specifications - e.g. if GID 110 needs to become GID
127, but also GID 102 needs to become GID 110 after the 110 to 127
changes are done (note the 110 overlap), then two passes are used (in
the hairiest of cases, one can use temporary otherwise unused GIDs
and/or UIDs as part of intermediary mappings - and that's probably even
recommended when the number of overlaps is large)).

Very recent use in action.  :-)
I installed
Debian GNU/Linux 6.0.3 "Squeeze" - Official amd64
on my new personal laptop.
I also wanted to merge in data from old personal laptop.
After doing the installation, between new installation and old laptop,
I had UID and GID conflicts.  There probably was some way I might've
prevented that during the install - but I didn't think about that until
later (Debian upgrades work so well I almost never do a fresh install
to "upgrade" a system, but instead to the actual upgrade procedure) ...
and I didn't want to reinstall again - so ... multichown to the rescue.
  Essentially:
... and also showing steps relevant for actual production environments:
reboot to single user mode, or securely and reversibly lock out all
except essential administrative account(s) for doing the conversion,
do the ID analysis to figure out what needs to be changed and how
(examining /etc/passwd, /etc/group, and generally also recommended to
examine UIDs and GIDs on actual filesystems),
use one pass, or if needed two passes, of multichown, with suitable
arguments, to make the UID/GID changes to all the applicable files,
use vipw, vigr, pwconv, grpconv, etc., to suitably update /etc/passwd,
/etc/group, and their corresponding shadow files,
reboot, reenable accounts, reboot

Below is example scripts I used.  The bit about "/etc/group only" was
comment to myself, about some changes to be applied to /etc/group, but
where there were no corresponding group ownerships of files on the
filesystems to be changed.
In this case, I was doing the remapping on the applicable new
filesystems, to match the older (being merged in) data.  I did it that
way, rather than vice versa, as that older data already matches to
additional system(s) where I wish to keep the UIDs and GIDs and also
associated names consistently aligned.

The scripts I executed (two of 'em, to do it in two passes, due to one
overlapping ID change - mutlichown is stateless - thus it can be safely
interrupted and later restarted - that's also why it can't directly
handle overlapping ID reassignments - but that can be handled manually
with just two passes).

# expand -t 4 < fixem1
#!/bin/sh
multichown=/var/tmp/blackie/home/m/michael/src/IDs/multichown
"$multichown" \
     --force \
     --recursive \
     0,0,103,115,105,117,107,120,109,126,110,127 \
     7,7,109,126 \
     100,122,101,122 \
     101,115,103,115 \
     102,123 \
     103,124,106,124 \
     104,119,107,120 \
/ /boot /home /usr /var

#,,108,125 #/etc/group only
#,,111,128 #/etc/group only

#second pass:
#0,0,102,110
# expand -t 4 < fixem2
#!/bin/sh
multichown=/var/tmp/blackie/home/m/michael/src/IDs/multichown
"$multichown" \
     --force \
     --recursive \
     0,0,102,110 \
/ /boot /home /usr /var
#

The bits further below give more information on multichown's general
operation, options/arguments, and some related utilities/materials.

todo:
wishlist item: add capability to handle GID remappings regarless of  
UID(s) (e.g. '*,*,fromGID,toGID' to remap GID(s) unconditionally  
regardless of UIDs)
fix the known spellos/typos in the text bits, etc.

references/excerpts:

multichown - works somewhat similar to chown(1), but handles complex
non-overlapping multiple UID/GID remappings, and does so in a single
pass. The primary design criteria has been do to UID/GID remappings
where large numbers of UIDs/GIDs need to be aligned, e.g. to LDAP, on
systems where the UID/GID ownerships aren't already suitably aligned,
and to do so in an efficient manner (e.g. for large filesystems that
take many hours to traverse, all the changes are made in a single
pass). The --help gives a pretty good overview of its functionality:
$ multichown --help
usage: multichown [--force|-f] [--help|-h|-?] [--nochange|-n]  
[--noxdev|--nomount] [--recursive|-r|-R] [--verbose [level]|-v  
[level]] IDspec [IDspec ...] pathname [pathname ...]
     --force|-f
         Be forceful - bypass some checks/restrictions.
     --help|-h|-?
         Help - provide some basic usage information, overrides other options.
     --nochange|-n
         Change nothing, but say what would otherwise be done.
     --noxdev|--nomount
         Descend directories on other filesystems.
     --recursive|-r|-R
         Recursively descend directories.
     --verbose [level]|-v [level]
         Be verbose, optionally specifying verbosity level:
             -1 not verbose (default if --verbose not used)
              0 report items changed (and not changed if --nochange)
                (default if --verbose used without level)
              1 0 and argument processing and report items not changed
              2 1 and include diagnostics on options and items processed
              3 2 and more diagnostics on options
     IDspec - ID specification, specifcation of set of ID changes as follows:
         fromUID,toUID[,fromGID,toGID ...]
         each ID must be specified as a decimal integer, any UID owned by
         fromUID will be lchown()ed to be owned by toUID, additionally, if
         and as specified, if the owning GID is fromGID, it will have the
         group ownership changed to the immedately following specified
         toGID
     pathname - pathname(s) to examine

Some of the funky invocation in the script is to deal with working
around some flakey (broken) perl installations on systems where more
functional installations of perl also exist.

news:1156663711.574609.98570 at i3g2000cwc.googlegroups.com
http://groups.google.com/groups/search?as_umsgid=1156663711.574609.98570%40i3g2000cwc.googlegroups.com
http://mail.pm.org/pipermail/oakland/2006-August/001885.html
http://www.rawbw.com/~mp/perl/