htdig-tips

Gergely Madarasz gorgo@caesar.elte.hu
Mon, 6 Sep 1999 12:30:00 +0200 (METDST)

On Sat, 4 Sep 1999, Grace, Terry wrote:

> I'm using htDig to index my list archives. Works great for the
> publicly accessible archives but since it locates documents via
> spidering, the only way to index the private archives is to turn them
> into public archives.
>
> My thought to get around this is on the private archives is to have
> htDig replace /pipermail/ with /mailman/private/ in the url. Easily
> done but in order for it to work via a script I need to have a method
> of turning the list public while it's indexing and then back to
> private when it's finished.
>
> Anyone have any ideas?

use the local_urls: option in your htdig.conf

Les Niles les@2pi.com
Wed, 22 Sep 1999 14:41:09 -0700

Wilma is a search engine and web interface built on top of MHonArc
and glimpse. (ftp://ftp.hpc.uh.edu/pub/wilma/) It works nicely
with mailman. I set it up to build its archive in the usual
mailman/archives/private/list_name/ directory, fetching the
messages from the list_name.mbox/list_name.mbox file. (Actually,
a link to that file with a month-specific name, since I've got
wilma set up to make month-by-month archives.) I haven't delved
into how pipermail is wired into mailman so I just left it all
there; there's no problem with it and wilma stepping on each other,
as long as wilma is configured to ignore the pipermail HTML
archives.

When users access the archives they do so through the "private" cgi
program, via a URL something like
http://mysite.com/mailman/private/list_name/9909/ (for the Sept 1999
archives), which means that mailman's password security mechanism gets
invoked just like for the pipermail archives. It worked surprisingly
easily.

Igor S. Livshits igorl@life.uiuc.edu
Fri, 20 Aug 1999 17:34:23 -0500

Hi,

Here's a short patch that will allow one to add a search block to the
index.html pages created by the Mailman archiver:

-=>diff HyperArch.py HyperArch.py.orig
372d371
< %(search)s
525,540d523
<
< # Provision for archive searching
< try:
< search_file_path= os.path.join(mm_cfg.PREFIX,
< 'archives/private',
< self.maillist._internal_name,
< 'search.html')
< search= open(search_file_path, 'r')
< except IOError:
< d["search"] = '''\
< 
< '''
< else:
< d["search"] = search.read()
< search.close()
<

Simply patch HyperArch.py and add a search.html file to your
~mailman/archives/private/<listname>/ directory. As the archive gets
rebuilt, the contents of the search.html file will get included just
above the index table of the index.html file in that directory.

This provides a fairly painless way to add an interface for ht://Dig
searches and similar.

This is my first attempt at Python code; so, if there is a better way
to accomplish the same, I'd love to hear about it.

Nigel Metheringham Nigel.Metheringham@VData.co.uk
Fri, 08 Sep 2000 09:48:26 +0100

In contrast to Mr Lawrence I am using the inbuilt pipermail archiver
with htdig, which seems work passably well.

You *do* want to add this patch which prevents the indexer indexing the
tables of contents and other cruft which then gives you lost of false
positives on searches.
http://sourceforge.net/patch/index.php?func=detailpatch&patch_id=100867
&group_id=103

An example of this form of configuration can be seen at
http://www.exim.org/

- follow the Mailing Lists link
http://www.exim.org/maillist.html

Jack Valko jack@valko.org
Sun, 27 Aug 2000 00:55:37 -0700

I've created a HOWTO on including SWISH-E searching into list archives.
It's available at http://www.valko.org/mailmanswish-e/. The changes are
rolled into my 1.1 installation, and I'll upgrade it to 2.0 as soon as
it comes out of beta.

Drop me a line if you have comments or questions,

<html>
<head>
<title>ht://Dig WWW Search</title>
</head>

<h1><a href="http://www.htdig.org"><IMG SRC="/htdig/htdig.gif"
align=bottom alt="ht://Dig" border=0></a> WWW Site Search</H1>

This search will allow you to search the contents of
all the publicly available WWW documents at this site.
To obtain help in searching, click <a href="searchhelp.html">here</a>.
To see all that has been searched before, click
<a href="/cgi-bin/htdig.log">here</a>.

<br>

<font size-1>Match:

<select name=method>
<option value=and>All
<option value=or>Any
<option value=boolean>Boolean
</select>

Format:
<select name=format>
<option value=builtin-long>Long
<option value=builtin-short>Short
</select>

Sort by:
<select name=sort>
<option value=score>Score
<option value=time>Time
<option value=title>Title
<option value=revscore>Reverse Score
<option value=revtime>Reverse Time
<option value=revtitle>Reverse Title
</select>

Which site:
<select name=restrict>
<option value="">All Eskom Web Servers
<option value="http://www.eskom.co.za/">External Web Site
<option value="http://intranet.eskom.co.za/">Intranet Web Server
<option value="http://lists.eskom.co.za/pipermail">ESKOM Mailing List Archives
</select>

<br>

Matches per page:
<select name=matchesperpage>
<option value=10>Ten
<option value=20>Twenty
<option value=30>Thirty
<option value=40>Forty
<option value=50>Fifty
</select>

<br>

Search:
<input type="text" size="30" name="words" value="">
<input type="submit" value="Search">

</form>

<hr noshade size=4>
For any queries regarding the search engine, e-mail
the <a href="mailto:richarde@eskom.co.za">searchmaster</a>.

</body>
</html>