Hypermail is a program that takes a file of mail messages in UNIX mailbox format and generates a set of cross-referenced HTML documents. Each file that is created represents a separate message in the mail archive and contains links to other articles, so that the entire archive can be browsed in a number of ways by following links. Archives generated by Hypermail can be incrementally updated, and Hypermail is set by default to only update archives when changes are detected.
Each HTML file that is generated for a message contains (where applicable):
the subject of the article,
the name and email address of the sender,
the date the article was sent,
links to the next and previous messages in the archive,
a link to the message the article is in reply to, and
a link to the message next in the current thread.
In addition, Hypermail will convert references in each message to email addresses and URLs to hyperlinks so they can be selected. Email addresses can be converted to mailto: URLs or links to a CGI mail program.
To complement each set of HTML messages, four index files are created which sort the articles by date received, thread, subject, and author. Each entry in these index files are links to the individual articles and provide a bird's-eye view of every archived message.
Hypermail was originally developed and designed by Tom Gruber for Enterprise Integration Technologies (EIT) in Common Lisp. It was later rewritten in C by Kevin Hughes while at EIT. Hypermail is now being maintained by Kent Landfield <email@example.com>.
To see what Hypermail can do, take a look at these Hypermail-produced archives:
Usage: hypermail [options]
-a URL : URL to other archives
-b URL : URL to archive information
-c file : Configuration file to read in
-d dir : The directory to save HTML files in
-i : Read messages from standard input
-l label : What to name the output archive
-m mbox : Mail archive to read in
-M : Use metadata
-n listaddr : The submission address of the list
-o keyword=val: Set config item
-p : Show progress
-s htmlsuffix : HTML file suffix (.html, .htm, ..)
-t : Use Tables
-T : Use index tables
-u : Append all input messages
-v : Show configuration variables only
-V : Show version information and exit
-x : Overwrite previous messages
-0 number : Delete messages
-1 : Read only one mail from input
-L lang : Specify language to use (de en es sv fi fr is pl)
Using the flags -h, or -? with Hypermail will display this usage summary.
To tell Hypermail what mailbox to read in, use the -m option. If articles will be sent to Hypermail through standard input, use the -i option. Note that the -m and -i options can't be used together! By default, Hypermail will look for a file called mbox to read its articles in from.
The -d option specifies the directory to put the HTML files and index files that are created into. If the directory doesn't exist, a new one will be created with the name that is specified. If the -d option isn't used, Hypermail will look for a directory with the same name as the mailbox or will create one if needed.
example 1: hypermail -m "wu-ftpd" -d "/wu-ftpd"
example 2: cat "/var/spool/mail/wu-ftpd" | hypermail -i
This example reads the articles in wu-ftpd and will save the output in the /wu-ftpd directory.
This reads the file /var/spool/mail/wu-ftpd from standard input and will save the output in a directory called wu-ftpd in the same directory Hypermail was run from.
Note that Hypermail can only read messages in the UNIX mailbox format! Such archives are typically RFC 2822 mail messages appended to each other that look similar to this:
From firstname.lastname@example.org Mon Jan 1 00:01:30 1994
Date: Mon, 1 Jan 1994 00:01:15 PDT
Subject: Hello, world!
Hi, everyone, just saying hello!
From email@example.com Mon Jan 1 00:02:00 1994
Date: Mon, 1 Jan 1994 00:01:45 PDT
The messages are typically separated by lines in this format:
From firstname.lastname@example.org Fri Jul 1 00:18:20 1994
The -c option tells Hypermail to read in settings from a configuration file. By default, the program will attempt to read settings from a file called .hmrc in the user's home directory if it exists.
In the configuration file, variables are set in the following manner:
variable = number
variable = "string"
The complete set of variables that Hypermail recognizes is described in the Configuration Options section.
The -l option tells Hypermail what to call the archive - the name that is specified will be in the title of the index pages so users know what sort of messages are being archived.
The -a option includes a link labelled "Other mail archives" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to pointers to other mail archives. By default, this will be a pointer to the parent directory in which the archive files reside.
The -b option includes a link labelled "About this archive" in the index pages to any specified URL. This way users who are looking at the archive have the opportunity to go to information about the archive.
example: hypermail -l "WU-FTPD Development Archives"
In the index files for the archive, the above setting will produce something like this:
The -x option tells Hypermail to explicitly overwrite any previous HTML files that may exist. Use this option only when it is desirable to completely rewrite the entire archive.
The -u option tells Hypermail to add message(s) to the end of the existing HTML file archive and integrate them into it by links and cross-references. All archive index files will be regenerated to include the new message.
Hypermail used to require that you only send it one message at a time when
using the -u option, but it should now work reasonably when
given mailboxes containing multiple messages.
When using the -u option, don't send any messages that
Hypermail has already processed. If you want Hypermail to recognize that
some messages are old messages that shouldn't be added to the archive again,
send it a mailbox with a complete set of messages and avoid the
example 1: cat "one.letter" | hypermail -i -u -d "/wu-ftpd/mail-archives"
example 2: hypermail -u -m "one.letter" -d "/wu-ftpd/mail-archives"
example 3: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives" -x
example 4: hypermail -m "mailbox" -d "/wu-ftpd/mail-archives"
This tells Hypermail to take the article it receives from standard input
and integrate it with the archive under the wu-ftpd/mail-archives
directory. If no archive exists, a new one will be created with the specified
letter as the first file of the archive.
This does the same thing, except that the letter is read in from a file that contains only that letter.
With these options, Hypermail will read in the articles from
mailbox and write over any existing files in the
wu-ftpd/mail-archives directory if they exist. If no archive
exists, a new one will be created.
With these options, Hypermail will read in the articles from
mailbox and only write new articles - it will not overwrite
any existing archive files.
Note that no matter what options are specified, the index files are always
rewritten. The date when Hypermail was last run is included in index pages,
so it's easy to tell when the archive was last updated.
The -p option shows a progress report as Hypermail reads in and writes out messages - the number of files that Hypermail is reading and writing and the file names of the directory and files created are shown. This information is written to standard output.
The -v option shows the configuration variables and their values that Hypermail would use if it was run with the same configuration file and command line options. This is useful when starting up a new list or modifying a list configuration file. Once the information is displayed, Hypermail terminates and not actual processing occurs.
The -V option prints the Hypermail version information. Once the information is displayed, Hypermail terminates and not actual processing occurs.
The -0 option list message numbers that should be deleted
from the html archive. The mbox is not changed.
It is equivalent to the delete_msgnum
In the configuration file, variables must be in lowercase and separated by their values with an equals (=) sign. Blank lines and lines beginning with the # character are skipped:
mbox = "/home/john/my_mailbox"
filemode = 0600
While the example uses quotes ("), they is not required when used in the configuration file.
Below is a list of the more important configuration variables.
For a complete list, see hmrc.html.
HM_LABEL "label name"
Define this as the default label to put in archives.
This will create a link in the archived index pages to the specified
URL. Define as "NONE" to omit such a link.
See also custom_archives.
HM_HMAIL "list submission address"
This is the email address used to send a new message to a hypermail archive. "NONE" means don't use it. Since this is different for each hypermail archive, you should probably leave it set to "NONE" here, and let it be specified at runtime by command-line parameters in the list specific configfile.
See also newmsg_command
This is the default directory that Hypermail will look for when creating and updating archives. If defined as "NONE", the directory name will be the same name as the mailbox read in.
This is the default mailbox to read messages in from. Define this with
a value of "NONE" to read from standard input as the
A string to be stripped from all subject lines. Helps
unclutter mailing lists which add tags to subject lines.
HM_FOLDER_BY_DATE = "strftime-date-format"
This string causes the messages to be put in subdirectories
by date. The string will be passed to strftime(3) to generate
subdirectory names based on message dates. Suggested values are
"%y%m" or "%b%y" for monthly subdirectories, "%Y" for
yearly, "%G/%V" for weekly. Do not alter this for an existing
archive without removing the old html files. If you use this
and update the archive incrementally (e.g. with -u), you must
use the usegdbm option.
See also monthly_index.
Set this to On to display article received dates in
YYYY-MM-DD HH:MM:SS format. If used with the gmtime
option, a Z will be inserted between the DD and HH.
See also eurodate and dateformat.
This is a two-letter string specifying the default
language to use, or a longer string specifying a language
and locale. Set this the value of the language
table you wish to use when running and generating
archives. See also iso2022jp
Current supported languages, with their default locales:
de (de_DE) - German
en (en_US) - English
es (es_ES) - Spanish
fi (fi_FI) - Finnish
fr (fr_FR) - French
el (el) - Greek
gr (el_GR) - Greek
is (is_IS) - Icelandic
no (no_NO) - Norwegian
pl (pl_PL) - Polish
ru (ru_RU) - Russian
sv (sv_SE) - Swedish
The directory /usr/share/i18n/locales on many systems has the locale
codes that are available on that system.
HM_INCREMENT boolean_number Define as 1 to append all input messages to the end of existing archives.
Define as 0 for it to read a mailbox that corresponds to the entire
archive. If there are any existing html messages, it will figure out which
ones at the end of the mailbox are new, and add only those that haven't been
Set this to On to maintain a parallel mbox archive. The file
name defaults to mbox in the directory specified by -d or dir.
See also append_filename
HM_SHOWHTML 0, 1, or 2
Define as 1 to show the articles in a proportionally-spaced
font rather than a fixed-width (monospace) font. Setting this
option to 1 also tells Hypermail to attempt to italicize quoted
passages in articles.
Define as 2 for more complex conversion to html
similar to that in txt2html.pl.
Showhtml = 2 will normally produce nicer looking results than
showhtml = 1, and showhtml = 0 will look pretty dull, but
1 and 2 run risks of altering the appearance in undesired ways.
Set this to On to create fine-grained links from quoted
text to the text where the quote originated. It also improves
the threads index file by more accurately matching messages
with replies. Note that this may be rather cpu intensive (see
the searchbackmsgnum option to alter the performance).
This will create a link in the archived index pages to the specified URL. Define as "NONE" to omit such a link.
The address of the contact point that is put in the HTML header line
<LINK REV=made HREF=mailto:MAILTO>
The <LINK...> header can be disabled by default by setting HM_MAILTO to "NONE".
Defining this will cause Hypermail to generate an index menu in HTML table format at the top and bottom of each page.
Setting this variable to 1 will tell Hypermail to generate
a message index Subject/Author/Date listings using a table
format. Set to 0 if you want the standard Hypermail index
page look and feel.
Set this to the domainname you want added to a mail address appearing
in the RFC2822 field which lack a hostname. When the list resides on the
same host as the user sending the message, it is often not required of
the MTA to domain-ize these addresses for delivery. In such cases,
Hypermail will add the DOMAINADDR to the email address. If defined as
NONE, this feature is turned off.
HM_USEMETA [ 0 | 1 ]
This option allows you to use metadata to store the content type
of a MIME attachments and, later on, when a user browses the
attachment, send back this information in the HTTP Content-Type
header. When set to 1, the Content-Type header of a
MIME attachment will be stored in a metadata file. Let us say that
the MIME attachments for a message are stored in directory
att-num. The metadata for those attachments will
then be stored in directory att-num/.meta. If a
MIME attachment is stored in file att-file, its
metadata will be stored in file att-file.meta. This
convention is directly compatible with the Apache server handling of
Defining this variable as 1 will reverse-sort the article entries in the date and thread index files by the date they were received. That is, the most recent messages will appear at the top of the index rather than the other way around.
Define path as the path to a file containing valid HTML formatting statements that you wish to included at the top of every message page. Hypermail will print this file as the header of the message so make sure it contains <HTML>, <HEAD>, and <BODY> and other statements that suit your local customized needs.
See also ihtmlheaderfile,
This is the default configuration file to read settings in from. This
can only be specified as an environment variable. If the first character
is "~", Hypermail will look for the file under the current user's home
From the program's hard-wired internal defaults (specified in options.h),
From runtime environment variables,
From command-line options,
From the configuration file.
It is important to note that this is different from many programs. In Hypermail's
case what you put on the command line DOES NOT override what is
specified in the configuration file. This may change in the future.
Filenames: In the specified directory, articles will be read out in the order that they were read in from a mailbox or standard input. Filenames start at zero and increase in this fashion: 0000.html, 0001.html, 0002.html, etc. In the same directory:
date.html is the index of articles sorted by the date they were received by the system's mail daemon.
thread.html is the index of articles sorted by thread first, then the date they were received.
subject.html is the index of articles sorted by subject. Any "Re:" prefixes in front of subjects will have been stripped out.
author.html is the index of articles sorted by the first word of the author's name. If the author's name can't be determined, their email address will be substituted.
One of the above files will be called index.html and is the default index that users can go to when entering the archive.
Sorting: In the date and thread index files, note that these lists are sorted by the date the articles were received by the system's mail daemon, not by the date they were written on. The order of articles in the date index may not necessarily match the order in which the article files are written and linked together. Because of this, it is a good idea to make sure the mailbox is sorted by date with the most recent messages towards the bottom.
Running Hypermail automatically: All that's needed to start archiving email messages is to set up Hypermail to do incremental updates in your /etc/aliases file. Here's what an entry might look like (the last line is one unbroken line):
# WU-FTPD Mailing List Archives
wulist: "|/usr/local/bin/hypermail -i -u -d /wu-ftpd/mail-archive -l \"WU-FTPD Mailing List Archive\""
After adding the entry, make sure newaliases is run to update the
mail aliases. This entry will run Hypermail and update/create the archive whenever
a new message is received. Hypermail also works well as a cron job.
Because sendmail may run Hypermail as different users, you will
want to make sure that archive directories and files are made readable and writeable
by a trusted sendmail user (or read/writable by everyone if you can't do that)
when they are created. This will ensure that there will be no problems incrementally
updating the archive.
If you are running Linux kernel version 2.4 or higher,
looks like it provides another way to automate Hypermail.
Including HTML in messages: One can include formatted HTML in message bodies by enclosing the HTML with the <HTML> tag (in either uppercase or lowercase). This tag must be on a line by itself:
This text will not be parsed...
this text will be parsed as HTML.
This text will not be parsed...
There is no limit to how often the <HTML> tag can be used in an article.
If you are are looking for more information on Hypermail and its features and
developmental status, check out
SourceForge: Project Info - hypermail
and hypermail.org. Additional
documentation and the hypermail development list archives are available there.