[conspire] 1:2.1.29-1+deb10u5? Re: upgrade-in-place to Mailman 2.1.30 and want to test Mailman3?
Michael Paoli
michael.paoli at berkeley.edu
Thu Mar 21 05:46:57 PDT 2024
So ...
Here's what I'm proposing, and the the "Reader's Digest" version of
plan to get to:
o Debian mailman 1:2.1.29-1+deb10u5 (has effective DKIM work-arround)
and also along the way:
o get linuxmafia.com to a (LTS) supported Debian release
o get linuxmafia.com to 64-bit (Debian's "amd64" architecture)
o better stabilize linuxmafia.com's (virtual) hardware
o start getting at least some initial use of LVM going on linuxmafia.com
LVM also planned, at least in part for guido, but that may come
earlier and/or later - more pressing needs for LVM with the above will
first hit on linuxmafia.com.
So, not covered/shown in the below, and what would be applicable for
linuxmafia.com:
o for prod(uction) (linuxmafia.com), first each major version upgrade on
linuxmafia2 (non-prod), then essentially likewise on prod.
o when stepping through that for prod, linuxmafia2 will mostly be done
like it was "for real" for prod, notably carefully following proper
upgrade procedures, carefully merging configuration changes, etc.
Main difference between prod and non-prod, is non-prod to be blocked
throughout on outbound TCP port 25 (except 127.0.0.0/8 and ::1).
Also, procedures show be followed to generally maximize uptime of
services on prod (Debian's upgrade documentation also pretty well
documents how to do this).
Anyway, that's the general idea. Additionally, some more snapshotting
will be done - most notably for linuxmafia2, so any major missteps in
upgrade process can be very much rolled back. But yeah, not like that
for prod ... because live data and all that ... and rolling everything
back would be bad (lost data and/or replay of stuff such as emailings
and list postings, etc.). So, prod mostly roll forward, not back, and
following the successful steps first well completed for non-prod.
Also, step-wise validation - as feasible, for both non-prod and prod.
Notably reasonably well checking, as feasible, functionality of at least
important services. In general things should mostly remain available
and mostly not break.
And, on the OS side, the 32-->64 bit cross-grading - I'm sure I could do
that more smoothly than last time through, so may do a few more
iterations of that and have that procedure better smoothed out before
applying to prod.
And prof-of-concept for expediency mostly went with maintainer's version
of config files - as opposed to the time consuming careful manual merge
at each major version upgrade (but will need to do that for prod).
Anyway, here's what the "Reader's Digest" version looks like, as it was
quite successfully applied to non-prod. And for the "for real", can
probably also better optimize some steps and such, but this basically
has been fully done to linuxmafia2 and works:
$ ssh guido 'cat mailman_plan.info'
(umask 077 && cd / && tar -cf - etc | xz -9 > etc.tar.xz)
/etc/apt/sources.list -->
deb http://archive.debian.org/debian/ squeeze main contrib non-free
deb http://archive.debian.org/debian/ squeeze-lts main contrib non-free
apt-get -y update
apt-get -y remove etckeeper # repeatedly gets in way during major upgrades,
note also not purged, so all the config data still there - can reinstall
package later.
apt-get -y upgrade
apt-get -y dist-upgrade
/etc/apt/sources.list -->
deb http://archive.debian.org/debian/ wheezy main contrib non-free
deb http://archive.debian.org/debian/ wheezy-backports main contrib non-free
apt-get -y update
apt-get -y --force-yes upgrade # go with maintainer's versions
apt-get -y --force-yes install linux-image-686
apt-get -y --force-yes dist-upgrade
apt-get -y --force-yes -f install
cd / && shutdown -r now
apt-get -y purge linux-image-2.6.22-3-686 linux-image-2.6.26-2-686
... transition from 32 bit to 64 bit
dpkg --add-architecture amd64
/etc/apt/sources.list -->
deb [arch=amd64] http://archive.debian.org/debian/ wheezy main contrib non-free
deb [arch=amd64] http://archive.debian.org/debian/ wheezy-backports
main contrib non-free
deb http://archive.debian.org/debian/ wheezy main contrib non-free
deb http://archive.debian.org/debian/ wheezy-backports main contrib non-free
apt-get -y update
cd / && shutdown -h now
VM: upgraded RAM & CPU:
<memory unit='KiB'>1048576</memory>
<currentMemory unit='KiB'>1048576</currentMemory>
<cpu mode='host-passthrough' check='none' migratable='on'/>
apt-get -y --force-yes install linux-image-3.2.0-4-amd64:amd64
vi /boot/grub/menu.lst
reboot
Now running 64 bit kernel:
Linux 3.2.0-4-amd64 #1 SMP Debian 3.2.78-1 x86_64 GNU/Linux
apt-get -d -y --force-yes install dpkg:amd64 gcc-4.7-base:amd64
libbz2-1.0:amd64 libc6:amd64 libgcc1:amd64 liblzma5:amd64
libselinux1:amd64 zlib1g:amd64 apt:amd64 tar:amd64
libapt-pkg4.12:amd64 libstdc++6:amd64
dpkg --add-architecture i386
apt-get -d -y --force-yes install libgcc1:amd64 gcc-4.7-base:amd64
(cd /var/cache/apt/archives/ && dpkg -i
libc6_2.13-38+deb7u10_amd64.deb libgcc1_1%3a4.7.2-5_amd64.deb
gcc-4.7-base_4.7.2-5_amd64.deb)
(cd /var/cache/apt/archives/ && dpkg -i libbz2-1.0_1.0.6-4_amd64.deb
liblzma5_5.1.1alpha+20120614-2_amd64.deb libselinux1_2.1.9-5_amd64.deb
zlib1g_1%3a1.2.7.dfsg-13_amd64.deb)
(cd /var/cache/apt/archives/ && dpkg -i tar_1.26+dfsg-0.1_amd64.deb
dpkg_1.16.18_amd64.deb)
(cd /var/cache/apt/archives/ && dpkg -i
libapt-pkg4.12_0.9.7.9+deb7u7_amd64.deb libstdc++6_4.7.2-5_amd64.deb
apt_0.9.7.9+deb7u7_amd64.deb)
dpkg -l |
#||/ Name Version Architecture Description
perl -e '
use Data::Dumper;
my %h=();
while(<>){
local @_;
chomp;
@_=split;
$#_ >= 4 or next;
my($stat,$name,$vers,$arch,$desc)=@_;
$stat eq q(ii) && ( $arch eq q(i386) || $arch eq q(amd64) ) or next;
#print(join(q( ),$stat,$name,$vers,$arch),"\n");
$name =~ s/:(?:i386|amd64)$//;
if(!exists($h{$name})){
$h{$name}={$vers => {$arch => undef}};
#print Dumper(%h); exit(0);
}elsif(!exists($h{$name}{$vers})){
$h{$name}{$vers}={$arch => undef};
#print("$name $vers ", Dumper($h{$name}{$vers})); exit(0);
}elsif(!exists($h{$name}{$vers}{$arch})){
$h{$name}{$vers}{$arch}=undef;
#print("$name $vers ",Dumper($h{$name}{$vers})); exit(0);
};
}
#print Dumper(%h); exit(0);
for $name (keys %h){
#print("$name\n");
for $vers (keys $h{$name}){
#print("$name $vers\n");
if(exists($h{$name}{$vers}{q(i386)})){
if(!exists($h{$name}{$vers}{q(amd64)})){
# have i386 but not amd64, need amd64:
print("$name:amd64=$vers\n");
}else{
# have i386 and amd64
#print("DUAL $name $vers\n");
}
};
}
}
'
/etc/apt/sources.list -->
deb [trusted=yes] file:/media/cdrom7 wheezy main
deb http://archive.debian.org/debian/ wheezy main contrib non-free
deb http://archive.debian.org/debian/ wheezy-backports main contrib non-free
deb [arch=i386] http://archive.debian.org/debian/ wheezy main contrib non-free
deb [arch=i386] http://archive.debian.org/debian/ wheezy-backports
main contrib non-free
apt-get -y update
make these changes for better firewall setup with ifup/ifdown:
rm /etc/ip6tables.up.rules /etc/iptables.up.rules
and for
/etc/network/if-{pre-up,post-down}.d/ip{,6}tables
echo 'H4sIALlL9GUCA+2WXUvjQBSGc51fccQ7YZr5TCV4U2IVF7qVGvHClZI2I420SZhMlf33O/GjYsS2
K36w7HlyMWFyYCYhz3smvyZVWVuSlXdFJwvyKrTpZK5r7wOhjq6U96OjPVKhlMek6ioqJW3qmGRK
eEC9L2BZ29QAeKYs7bq6Tc//UXZ3gkleBPXMD+rmZvUDEKNrWxoNBwfwazzuD5Px2N+FY11ok1qd
weQ3PBfX6a2GW9YRnX0oCzhbFjBwn5VRYDJSKuL7wCmX/t51Prfa+NHJz9PzBHpx3D9N4JJG9MqP
joaji97osDU7PE9elcbDweAkcduJy0U118121q36tH0PaZO3/f94/Tf7H4pn//m9/zTk6P/3+P83
+m9pP1Nr7XdFjNNw2wTgNGIhk1unwNPqmAJv+G80WVaf1fy38F+xVf93USCc/0IK9B/7/5v9n/Tg
cXoHSAZRxALmliAV2GkFZPEwkKwqjQWugNzAqP+jHydu0ugbPbXkLrczyKeLKiRNFVkWRqfTWfM6
8B+dL176/yn6b/ZfhCv/lQob/wWe/7H/b+r/rRRgvNuhzRVwpR7u3pcI7w8EPGogCIIgCIIgCIIg
CIIgCIIgCPL9/AHqu5WcACgAAA==' | base64 -d | gzip -d | (cd /etc/network/ &&
tar -xf -)
... transition from 32 bit to 64 bit
dpkg --remove-architecture i386
get wheezy full upgraded set-wise through to completion of dist-upgrade
reconfigure grub2 for serial
--- /etc/default/grub 2024-03-14 07:00:04.000000000 -0700
+++ - 2024-03-15 02:56:49.149236788 -0700
@@ -7 +7 @@
-GRUB_TIMEOUT=5
+GRUB_TIMEOUT=15
@@ -9,2 +9,2 @@
-GRUB_CMDLINE_LINUX_DEFAULT="quiet"
-GRUB_CMDLINE_LINUX=""
+GRUB_CMDLINE_LINUX_DEFAULT=""
+GRUB_CMDLINE_LINUX="consoleblank=0 console=tty0 console=ttyS0,9600n8"
@@ -18,0 +19,4 @@
+GRUB_SERIAL_COMMAND='serial --unit=0 --speed=9600'
+GRUB_TERMINAL_INPUT='serial console'
+GRUB_TERMINAL_OUTPUT='serial gfxterm'
+GRUB_DISABLE_OS_PROBER=true
update-grub
apt-get -y install linux-image-amd64
reboot
/etc/apt/sources.list -->
deb http://archive.debian.org/debian/ jessie main contrib non-free
apt-get -y update
block systemd:
/etc/apt/preferences.d/99init
echo 'H4sIAAAAAAACA+3SwU7DMAwG4Jz7FL4NkDq1W1e03ThwRX2FrDVgUZKRpBWTeHiSAYcCYtIkkJD+
7+JajpM46notRoL6VUVUV9UhRp9jUcfvsqqqerFcrlalKsqiXlwqKtQfGHzQjkg5a398hmP1f+r6
eddro4NYs6Gr0UpHgxET2HTckZj4On1/KJO9Jb/3gR+7PMZxnk16029ErTa0Zdo5O0rq3+43kx56
SemY1uatdZw1un3QdzxdlTUSNxzZ+XTsRUrzxol1EuJ+eZllJ9162nVjA1O41+GjTJ1lb2aBHD8N
4nh68bP3bOa/Gfutdj7/Ms7xSRQAAAAAAAAAAAAAAAAAAAAAAMCJXgF1hTjQACgAAA==' |
base64 -d | gzip -d | (cd /etc/apt/preferences.d/ && tar -xvf -)
apt-get -y upgrade
apt-get -y dist-upgrade
reboot
apt-get -y purge linux-image-3.2.0-4-amd64
/etc/apt/sources.list -->
deb http://archive.debian.org/debian/ stretch main contrib non-free
apt-get -y update
apt-get -y upgrade
apt-get -y dist-upgrade
reboot
apt-get -y purge linux-image-3.16.0-6-amd64
/etc/apt/sources.list -->
deb http://archive.debian.org/debian/ buster main contrib non-free
deb http://archive.debian.org/debian/ buster-updates main contrib non-free
deb http://security.debian.org/ buster/updates main contrib non-free
deb-src http://security.debian.org/ buster/updates main contrib non-free
apt-get -y update
out of space on /var/lib /dev/md1 /dev/sdb6
unmounted that and copied the contents to /var/lib (on /var)
and commented /var/lib out of /etc/fstab
apt-get -y install lvm2
mdadm --stop /dev/md1
mdadm --zero-superblock /dev/sdb6
comment md1 out from /etc/mdadm/mdadm.conf
blockdev --setrw /dev/sda3 /dev/sdb8
rm /etc/init.d/ro-partitions /etc/rcS.d/S16ro-partitions
changed partition type of /dev/sda3 and /dev/sdb8 to 8e Linux LVM
pvcreate /dev/sdb6 /dev/sda3 /dev/sdb8
vgcreate vg00 /dev/sdb6 /dev/sda3 /dev/sdb8
created lv, moved /var/lib content to that,
updated /etc/fstab, and mounted that on /var/lib
apt-get -y upgrade
apt-get -y dist-upgrade
and now out of space on /usr /dev/sda9, moving that to LVM ...
edited /etc/network/if-{pre-up,post-down}.d/ip{,6}tables changing
/sbin to /usr/sbin and restore to legacy-restore
changed partition type of /dev/sda9 to 8e Linux LVM
created LV, copied data to it, updated /etc/fstab, mounted LV on /usr
apt-get -y dist-upgrade
reboot
apt-get -y purge linux-image-4.9.0-13-amd64
shutdown -h now
redid the virtual hardware:
start with base of default built around (first) disk image, then:
add 2nd and 3rd disks (the 3rd is "temporary" - for large caching of
/var/cache/apt/archives/ contents notably to both avoid repeated
downloads, and more importantly have relevant packages handy even
if/when network isn't available)
change first disk to SCSI,
add watchdog hardware,
add 2nd ethernet interface,
change MAC addresses on ethernet interfaces
boot - virtual hardware seems to work fine
dpkg -r sa-compile # sa-compile too broken, even after purge + reinstall
dpkg --purge linux-image-3.16.0-6-amd64
dpkg -r libapache2-mod-php5
apt-get -y install texlive-font-utils
apt-get -y purge linux-image-3.16.0-6-amd64
apt-get -y purge linux-image-4.9.0-13-amd64
Now at good clean buster Debian 10(.13) state (notwithstanding do still
have lots of obsolete packages and configuration of many removed
packages), except one more glitch to work out on LVM initialization.
Would've been automatic if originally installed with LVM, but
still having to do a manual vgchange -a y to boot (notably find and
mount /usr). So, probably just wee configuration tweek to find and fix
then that should be good. While figuring that out,
lets see if can do high I/O xfer over ssh to linuxmafia2 storage
without now crashing.
Checked, and with the varous VM changes and OS changes,
watchdog is still using correct (virtual) hardware device.
High I/O does still trigger some (virtual) SCSI reset events
on the VM, but no kernel Oops, and no issues seen at physical (guido)
level. Wrote 5GiB, twice, + sync&&sync after, and some other concurrent
I/O on the VM, and other than the aforementioned resets, no issues.
So, looks like the changes in VM hardware probably bought some fair bit
of stability. May be worth doing likewise to linuxmafia.com (prod)
sooner, rather than later. Shall see if linuxmafia2 remains quite
stable in those regards.
minissdpd looks to be much more the nuisance than not, so
stopping and disabling:
/etc/init.d/minissdpd stop
update-rc.d minissdpd disable
but leaving it installed in case actually desired later.
apt-get -y autoremove
dpkg-reconfigure linux-image-4.19.0-21-amd64
/etc/apt/sources.list -->
deb http://deb.debian.org/debian/ buster main contrib non-free
deb-src http://deb.debian.org/debian/ buster main contrib non-free
deb http://security.debian.org/ buster/updates main contrib non-free
deb-src http://security.debian.org/ buster/updates main contrib non-free
apt-get -y update
apt-get -y upgrade
apt-get -y dist-upgrade
apt-get -y install linux-image-amd64
reboot
apt-get -y purge linux-image-4.19.0-21-amd64
and added:
ls -l /etc/initramfs-tools/scripts/local-premount/vgchange_-a_y
-rwxr-xr-x 1 root root 29 Mar 21 04:34
/etc/initramfs-tools/scripts/local-premount/vgchange_-a_y
#!/bin/sh
vgchange -a y || :
dpkg-reconfigure linux-image-4.19.0-26-amd64
And that's sufficient to prevent the boot from dropping out to
intramfs and needing a manual vgchange -a y
apt-get -y install etckeeper
$
More information about the conspire
mailing list