[sf-lug] sf-lug.mbox ... rsync ... now gently backed up overnightly

[Michael Paoli]

Rick - Thanks

regarding:
> Date: Sun, 1 Feb 2015 01:15:56 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: sf-lug <sf-lug at linuxmafia.com>
> Subject: Re: [sf-lug] sf-lug.mbox ... wget ... rsync? :-)
> Message-ID: <20150201091556.GU10984 at linuxmafia.com>
>
> Quoting Michael Paoli (Michael.Paoli at cal.berkeley.edu):
>
>> Is http://linuxmafia.com/pipermail/sf-lug.mbox/sf-lug.mbox
>> also available via such rsync access (and if so could you let me/us
>> know the rsync path to it), or if it isn't could you make it so
>> and let us know?  Thanks for your consideration on this.
>
> OK, added.  Pretty good idea!
>
> $ rsync linuxmafia.com::
> sf-lug          SF-LUG cumulative mbox
> $
> :r /etc/rsyncd.conf
> read only = yes
> [sf-lug]
>         path = /var/lib/mailman/archives/private/sf-lug.mbox/
>         comment = SF-LUG cumulative mbox
>         list = yes

see also:
http://linuxmafia.com/pipermail/sf-lug/2015q1/010666.html

And now gently rsync(1)ed overnightly.  Still have bit more rotation
stuff to add, but it's off to a fair start (and manually tested for
first run).
And this host to which it's rsync(1)ed, is itself backed up to off-site
on a regular basis approximately monthly.

The rsync(1)ed target copy (and older copies) available at:
http://www.sf-lug.org/sf-lug.mbox
http://www.sf-lug.org/sf-lug.mbox.BAK/

And yes, I thought it a good idea - after I'd done the very first
initial wget to the sflug host, seemed to me rsync would be the way to
go to maintain it - avoid the wget --continue hazards, and go easy on
server bandwidth.  Was only today or yesterday (time, priorities ...
besides, had already at least gotten one recent copy) I managed to check
and notice you already had public rsync access to the host - so seemed
natural to then ask about such access to the file (which is public
anyway).

Still have some more rotation/thinning to add to the basic script, but
what it thus far has is covered a bit further below (stripped of comments
and blank lines, for brevity).

Below, whitespace within [] is generally one space and one tab.
PS1='# ' PS2='> '

# hostname
sflug
# id
uid=0(root) gid=0(root) groups=0(root)
# crontab -l
07 3 * * * /usr/local/bin/sf-lug.mbox_rsync+rotate
# < /usr/local/bin/sf-lug.mbox_rsync+rotate expand -t 4 |
> sed -ne '1p;2,${;/^[  ]*[^    #]/p;}'
#!/bin/sh
set -e
umask 022
backup_and_staging_dir=/var/www/sf-lug/sf-lug.mbox.BAK
target_dir=/var/www/sf-lug/
rsync_opts='--quiet --checksum --times --sparse --partial  
--ignore-times --compress-level=9 --bwlimit=5'
cd "$backup_and_staging_dir"
cp -p sf-lug.mbox sf-lug.mbox.tmp
rsync $rsync_opts rsync://linuxmafia.com/sf-lug/sf-lug.mbox sf-lug.mbox.tmp
mtime=$(stat -c '%Y' sf-lug.mbox)
if [ -f sf-lug.mbox."$mtime" ]; then
     cmp sf-lug.mbox sf-lug.mbox."$mtime" &&
     rm sf-lug.mbox
else
     mv sf-lug.mbox sf-lug.mbox."$mtime"
fi
mv sf-lug.mbox.tmp sf-lug.mbox
ln -f sf-lug.mbox "$target_dir"/sf-lug.mbox
find sf-lug.mbox.* -type d -prune -o -type f -mtime +366 -exec rm \{\} \;
#