[sf-lug] lists.sf-lug.org draft config settings (was: done*: SF-LUG & MX records ... @lists.sf-lug.org ...)

[Rick Moen]

I wrote:

> I've spend the last couple of hours banging my head up against many
> details of Mailman's virtual domain support and Apache's Location
> stanzas and special provisions for Mailman CGIs, trying to make
> test2 at lists.sf-lug.org work right.  I can provide details if there is
> interest.  
> 
> However, it doesn't fully work -- almost but not quite....

At Michael Paoli's offlist suggestion, Mailman and Apache2 particulars
follow.


Mailman
-------

Base config directory for GNU Mailman on Debian is
/var/lib/mailman/Mailman/ .  The main conffile is  mm_cfg.py, which is 
implemented as a symlink to /etc/mailman/mm_cfg.py, in accordance with
Debian Policy about system configuration files.  However, FYI, some
significant settings are in Defaults.py, which is _not_ symlinked to
/etc.

Here is /mm_cfg.py plumbing to support virtual domains.  Note
commented-out lines for the (former) virtual hosting of four mailing
lists on lists.linuxgazette.net:

---<begin snippet>---

#-------------------------------------------------------------
# Uncomment this if you configured your MTA such that it
# automatically recognizes newly created lists.
# (see /usr/share/doc/mailman/README.Exim4.Debian)
# MTA=None   # Misnomer, suppresses alias output on newlist

#-------------------------------------------------------------
# Uncomment if you use Postfix virtual domains, but be sure to
# see /usr/share/doc/mailman/README.Debian first.
# MTA='Postfix'
MTA = 'Postfix'
POSTFIX_ALIAS_CMD = '/bin/true'
POSTFIX_MAP_CMD = 'chgrp Debian-exim'
#POSTFIX_STYLE_VIRTUAL_DOMAINS = ['linuxmafia.com', 'lists.linuxgazette.net']
POSTFIX_STYLE_VIRTUAL_DOMAINS = ['linuxmafia.com', 'lists.sf-lug.org']

#VIRTUAL_HOSTS = {'lists.linuxgazette.net':'lists.linuxgazette.net'}
VIRTUAL_HOSTS = {'lists.sf-lug.org':'lists.sf-lug.org'}
VIRTUAL_HOST_OVERVIEW = 1

---<end snippet>---

As a point of interest, the "MTA='Postfix'" line and other references to 
Postfix merely mean Mailman will be using the Postfix _method_ of
communicating with the system MTA.  The system MTA on my host is
(and remains) Exim4, not Postfix.


With merely the above plumbing, it is possible to successfully 
create mailing list 'test at lists.sf-lug.org' by doing:

# /var/lib/mailman/bin/newlist test2 at lists.sf-lug.org

(Note for novices and sudo-obsessed Ubuntistas:  In this context, the
'#' isn't a comment prefix, but rather indicate use of root-user
authority, whereas '$' would indicate execution as a non-root user.
This is a Unix convention, so learn it.)

Running newlist _does_ create a functional mailing list!  As thus
created, it exists solely within the lists.sf-lug.org namespace, a point
to which we'll return when we discuss Apache2, below.  At this point,
the mailing list thus created is 'functional' in that you can
communicate with it normally via the e-mail command interface, and 
my testing confirmed that I was able to successfully send it mail,
i.e., because I was not subscribed my test mail landed in Mailman's 
admin queue for it.  Had I been subscribed, I expect the mail would have 
gone out.

Of course, hardly anyone uses Mailman's e-mail interface to command
Mailman to do things, even though you _can_ accomplish pretty much any 
action that way, including subscribing and unsubscribing.  Almost
everyone uses the Web interface, which is fueled by a number of Python
scripts running as CGIs to populate the Mailman-specific Web pages.
Thus we now turn to Apache2:


Apache2
-------

Debian intends for the administrator to keep Apache2's config tree 
as a set of individual configuration fragments in
/etc/apache2/sites-available/ .  The ones to be used get pointed to 
by symbolic links in related directory /etc/apache2/sites-enabled/ .
This logic is driven by top-level files, which it's intended that you 
(normally, mostly) do _not_ fool with after initial system setup:

/etc/apache2/apache2.conf
/etc/apache2/httpd.conf
/etc/apache2/ports.conf
/etc/apache2/magic
/etc/apache2/envvars

In particular, apache2.conf includes this bit that makes Apache 
heed the symlinks in /etc/apache2/sites-enabled/ and, via them, the 
individual configuration fragments in /etc/apache2/sites-available/:

# Include the virtual host configurations:
Include /etc/apache2/sites-enabled/

So, because this exists in /etc/apache2/sites-available/ ...

000-default -> /etc/apache2/sites-available/default

...Apache2 pulls its local configuration on my server from file
/etc/apache2/sites-available/default .

Now, a long time ago, I was taken by surprise during a major
Debian upgrade by the sudden unavailability of package 'apache', which 
was Apache httpd 1.3.x.  This was because Apache 1.3.x had been
end-of-lifed and I hadn't paid adequate attention.  Relatively 
then-new Apache2 series (packaged by Debian as 'apache2') had a
significantly different configuration file format, and there was no 
upgrade path.

So, this was a situation where my Web server was down, and I suddenly
realised it would remain down until I wrote from scratch, manually, an
Apache2 configuration corresponding to my Apache 1.3.x one.  (The other
alternative of fetching to obsolete, unmaintained 'apache' package from
the previous Debian release and limping along until I could write an
apache2 configuration at greater leisure, did not occur to me in this
stressful and time-constrained situation.)

So, I wrote a functional /etc/apache2/sites-available/default file 
in a tearing hurry, hacked on it clumsily until it worked, and then
stopped.

Because Apache 1.3.x used a single conffile, that was the approach I
emulated by default, but this is not the _correct_ way to work within
Debian's intended administrative framework for Apache2.  The correct way 
is to create a separate configuration fragment in sites-available for
each site, and then make a symlink from sites-enabled to each such 
site's configuration fragment that Apache is to make operational.
I did not do that because I was in a hurry and didn't stop to read
/usr/share/doc/apache2-doc/README.Debian.gz or other Debian-specific
docs.

At some point, I _should_ have broken all of this down into Debian-style
fragments, refactoring the configuration -- but doing so involves
fooling around with a running production server, which has obvious
risks, so ironically it becomes unattractive to do what one knows one
ought to do.

There are ways out of that dilemma, outside the scope of current
discussion.


Here are the relevant additions to my single configuration file
where all local sites are defined, /etc/apache2/sites-available/default :

<VirtualHost lists.sf-lug.org:80>
    DocumentRoot /var/www/sf-lug
</VirtualHost>

<Directory /var/www/sf-lug/>
    Options Indexes MultiViews FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>


That is based very closely on these commented-out lines from the 
pre-2010 Mailman namespace for the four _Linux Gazette_ mailing lists:

#<VirtualHost lists.linuxgazette.net:80>
#    DocumentRoot /var/www/gazette
#</VirtualHost>

#<Directory /var/www/gazette/>
#    Options Indexes MultiViews FollowSymLinks
#    AllowOverride All
#    Order allow,deny
#    Allow from all
#</Directory>


With the cited additions (and restarting Apache2), it was possible to
reach Mailman page http://lists.sf-lug.org/mailman/listinfo and
http://lists.sf-lug.org/mailman/listinfo/test2 (before I removed mailing
list test2 at the end of my attempts at debugging).  

Using administrative credentials to log into the admin WebUI interface for 
mailing list test2 at lists.sf-lug.org brings up the admin CGI pages
within http://lists.sf-lug.org/mailman/admin/, but
any attempt to save changes results in a 404 on a slightly-wrong
URL like http://lists.sf-lug.org/cgi-bin/mailman/admin/test2
(which should have been http://lists.sf-lug.org/mailman/admin/test2).
Thus, changes cannot yet be made, or subscription / unsubscription
requests processed, via the WebUI unless this glitch gets chased down.

As Michael Paoli is a highly trusted user on my system, I will be
enabling full administrative rights for him for a while, so he can have
a go at this, time and interest permitting.


Oh, and by the way, _if_ this gets debugged, then the remaining problem
is migrating the current sf-lug at linuxmafia.com back-postings archive,
membership roster, and mailing list settings to new mailing list
sf-lug at lists.sf-lug.org.  There is no 'just rename everything to use a
different FQDN' magic wand.  At a glance, it appears that I would
construct a new mailing list within the lists.sf-lug.org namespace, 
enter manually into the admin WebUI settings drawn from the existing
one, copy over the existing cumulative mbox of prior postings, run
/var/lib/mailman/bin/arch to generate new HTML archives, dump the
existing mailing list's roster to a text file, and use the same utility to 
load that roster into the new mailing list.  Oh, this would assign new,
random subscription passwords to everyone.  Then, I would somehow
disable the old mailing list while testing the new one, and make sure
everyone knows about the switchover and doesn't keep trying to use the
old address.

(There may be shortcut ways to copying the existing mailing list's
configuration over, but I haven't yet discovered them, let alone tested
them.)

Some of this may prove simpler than first appears, after experimentation
and testing.

Of course, the way to make everything easy (or, I should say 'easy') 
is to not do the work, and expect someone else to solve all the
difficulties you're not even aware of.