This is my TODO list for procmail. My personal priorities are to maintain and improve the efficiency, security, robustness, and usability of procmail. Requests For Enhancement will probably go on the bottom of the list unless they help increase one of the above four. If they add bloat they'll probably drop off the list. If the task can reasonably be farmed out to some exterior program then I can almost guarantee you won't find it here. To quote the procmail README file:

Please note that this program essentially is supposed to be static; that means no extra features (honouring the VNIX spirit) are supposed to be added (though any useful suggestions will be appreciated and evaluated if time permits).

I have a separate page for the recording of "warts" and other "non-obvious behaviors". The current version of the procmail HISTORY file and SmartList HISTORY file, as included in the source snapshots, are also available. You can find links to snapshots of the current procmail and SmartList source trees on those pages, or you can browse the combined source tree.

Submissions for either this or the warts list and comments on the items already on it should be sent to <[email protected]>. Discussion about procmail and SmartList development and changes takes place on the procmail-dev and smartlist-dev mailing lists. To subscribe to either list, send an email message to <[email protected]> or <[email protected]> as appropriate with "subscribe" in the body.

Completed Items

Procmail

SmartList

DSN fix

The DSN (Delivery Status Notification) recognizer was broken by the CVS tabifier.

Bugs

unlocking occurs late

Unlocking a locallockfile is delayed until the next line is read. If a recipe ends on the last line of the /etc/procmailrc then the lockfile won't be unlinked until after procmail gives up its root privileges.

readmail() efficiency problems (part II)

mmap() a temp file

This requires a new level of abstraction on the handling of the message memory block. I've implemented this in the snapshot and it works, but the reading in of a new message could be more efficient. Or maybe not. This requires a few bugfixes, testing, and tuning.

formail parsing bugs

Formail's message and address parser should be checked for conformance with the latest messaging standards and against the draft message format document. It has been fixed to accept and strip whitespace before the colon, but there are probably other things to adjust. It also needs to be checked for proper handling of NULs in header fields. There was at least one bug caused by using strcpy() instead of memcpy().

SmartList install and alias issues

SmartList needs to be setgid as well as setuid when not installed setuid root. The docs should say so. Also, both procmail and SmartList documention should say more about how things change in the presence of smrsh.

SmartList procbounce doesn't handle multiple addresses

SmartList's procbounce needs to be updated to loop over multiple addresses when they have been extracted from a DSN.

Address confusion in SmartList (part II)

While SmartList accepts subscribe and unsubscribe requests that specify another address, rc.request checks for the address from the headers, not from the request itself. It also appears that multigram may have a problem with addresses that don't have a space between a double quote and an open angle bracket (unconfirmed).

Don't use network/socket libraries if they're not needed

initmake should try compiling a program with socket(), send(), and gethostbyname() and if it links without any network or socket libraries, then don't use them. This would fix both the problem of IRIX's broken -lnsl and -lsocket, and DYNIX's broken -linet.

Locking inefficiencies

The temp file doesn't need to be removed each time procmail tries to lock a mailbox. The retries should probably also be backed-off, either with an exponential or Fibonacci. Then again, maybe not.

TIMEOUT problems

TIMEOUT doesn't work as expected with backquote expansion and ? conditions if a shell is invoked. Procmail sends SIGTERM to the shell, but it doesn't get passed on to the child of the shell that's holding things up. The solution may be to have procmail put its kids in their own process groups, then use killpg() (or kill(-pid)) to get them all. How standard are process groups?

regexp matching bug

Procmail's regexp engine can be tricked into returning the wrong text in MATCH. While there's always another regexp which will match the same set of texts but set MATCH to the correct text, it's still a bug. Here's an example.

Bad default for w/W flag

By default, procmail ignores the status of programs that it invokes. This is usually the Wrong Thing. The default should be the same as the 'w' flag, and some other flag should tell procmail to ignore the exit status.

No "$@" in logged abstract

When passing "$@" to a command, the "Folder:" logged does not include any of arguments passed via the "$@"

Custom delimiter lossage

When using a custom message delimiter (like MMDF's ^A^A^A^A\n) procmail fails to escape the delimiter in incoming messages, resulting in corrupted mailboxes. Best current workaround is to put a recipe in the /etc/procmailrc file that reads something like

        :0 fw
        * ^A^A^A^A$
        |perl -pe 's:\001\001\001\001$:\002\002\002\002:'

The "^A"s in the condition need to be real control-A characters.

Control-M isn't whitespace

Every so often someone copies an rcfile from a Windows box and it ends up with CRs on the end of every line. They should be treated just like spaces and tabs are. As if, the results are really confusing.

Shell Expansion (part I)

Shell expansion of conditions via the '$' special treats double-quotes weirdly. They should not be considered special at all there.

Backslash-newline inconsistencies

Backslash-newline removal is almost completely inconsistent and should be straightened out some how, but without breaking anything that's in use.

Enhancements & Clarifications

GROUP_PER_USER at run-time?

Can GROUP_PER_USER be controlled at run-time by a variable set in the /etc/procmailrc file?

Unnecesarily complicated "From " logic

The logic that governs the generation of the "From " line is probably more complicated than it needs to be . In particular, could the -o flag be treated as if the invoker had written -f $USER?

new headers

Several new headers have been added by recent RFCs or are now in common usage by mail programs. Many of these should be added to formail's internal list for digest splitting.

mailbox format details

Each mailbox format has its own details. Messages in MH folders shouldn't have a "From " pseudo-header and newer versions of MH store the date that would have been there in a Delivery-Date: field. Maildir messages have no "From " pseudo-header and are supposed to have Delivered-To: field (but does that have to be added by the MTA to reflect aliases?). mbox message must have a "From " pseudo-header. Should procmail add a Delivery-Date: field and suppress the "From " pseudo-header on (all?) directory mailboxes?

Error messages

It would make debugging some errors much easier if the errno message was included in the log.

Filename/linenumber logging

On syntax errors and the like, procmail should log the name of rcfile in effect and the line number.

Documentation fixups

The suggestions for what to put in a .forward file are incorrect for sites with smrsh or exim, though the sendmail 8.10.0 version of smrsh appears to fix this.

uintmax_t for ultoan()

Currently, procmail assumes that unsigned long is the largest unsigned integral type. While this is currently a safe assumption for procmail's purposes, this changed with C90, the new version of the C standard. Probable solution: if uintmax_t exists, use it. Otherwise, continue to use unsigned long. I need to read C90 to make sure this'll do it.

non-sgid installation

If procmail is already installed as the local mailer (#ifndef CF_no_procmail_yet), then I think that recommend.c should skip the examination of the sgidness of /bin/mail, et al, and just consider the modes on the mailspool to determine whether procmail should be sgid mail. If the spool is mode 1777 and procmail is already in use, then I don't think procmail should be setgid.

Features

SmartList enhancements

Confirmation cookies (with Bible quotes on them?). The ability to send out digests in HTML format, thereby allow links from the digest TOC to individual messages and possible auto-inclusion of HTML versions of individual messages (would probably require perl to do the MIME and HTML correctly). A program for generating a sorted/threaded archive on the web (like the procmail archive found at http://www.xray.mpe.mpg.de/mailing-lists/procmail/). Other enhancements??? (Please send suggestions to me or smartlist-dev!)

SmartList documentation enhancements

There's a bunch of common requests that the SmartList documention could show how to do. For example, the documention could explain how to put "[listname]" tags in the subject lines of messages. The FAQ linked to from the homepage appears to cover this.

enhanced exitcodes for LMTP

Allow the return of full DSN style status codes when using LMTP. It would probably involve the overloading of EXITCODE so that assigning a text string would be a DSN code, while a simple number would be a EX_* code. This is mostly waiting on the main LMTP work.

formail mime splitting

It would be nice if formail could split MIME multipart messages. Currently people either use something like Perl to pull a full MIME parse, or cruft up some partial solution that may or may not work in the future. It would be nice if there was a middle solution that didn't require people to learn Perl, but worked correctly all of the time. This requires a big pile of thought on how it would work and interface with procmail.

OR flag

Should there be a flag that causes conditions to be ORed instead of ANDed? This would be a crutch for people who never studied logic and/or can't remember DeMorgan's laws, but it would be a useful crutch, given the number of times people ask "how do I 'or' conditions?"

levels of VERBOSEity [sic]

Should VERBOSE take a numeric value to indicate that some things should be logged but not others (e.g., scoring would be logged if $VERBOSE > 3, while condition matching would be logged if $VERBOSE > 2)?

LOCKRETRYCOUNT to mimic lockfile's -r option

This is probably a bad idea for most situations, but when you need it, it's a real pain to fake up. Hmm, perhaps procmail should unset/reset it before the implicit delivery to $ORGMAIL?

Make procmail IPng aware/compatible

This is really low priority, but as IPv6 becomes real, network-aware applications will need to be tweaked to handle both IPv4 and IPv6 (and done properly, any future protocols with in certain bounds). procmail is network-aware only in its talking to comsat/biff, but, well, some day it'll have to be updated. This is more a reminder until the time comes when most OS vendors are shipping IPv6-aware OSes

Make mailstat's output more customizable

To sure what this really means, but someone suggested it.

Shell Expansion (part II)

Shell expansion is stuck in the bourne shell days. It would be really handy to have some of the newer forms like ${var##word}.

Provide for run-time configuration of compile-time values

The values of interest are mainly the spool location and the value of the GROUP_PER_USER #define. The current idea is for procmail to always read another rcfile, /etc/procmail.conf, after it initializes the environment for the user but before reading /etc/procmailrc or $HOME/.procmailrc. Only variable assignments and variable capture recipes would be processed in that rcfile and a mismatched HOST would merely exit that rcfile without losing the message.

Rejected Changes

Don't send messages to comsat that specify pipes

It's kind of strange to send a message to comsat with a 'filename' field like |formail -I"Foo: bar" >>some-file. Currently, procmail simply leaves out the 'file position' field in the message that it sends to comsat. Should it just not send a biff message at all?

This isn't really doing any harm and may be useful for an extended comsat daemon, or if procmail's biff messages have been directed to a different port.

Eliminate LINEBUF

Instead of performing expansions in a fixed size buffer, dynamically extend the buffers as needed.

Making procmail dynamically size it's expansion buffers is not necessarily a Good Thing -- doing so may allow a remote (and untrusted) user to consume arbitrary amounts of a precious system resource. This is separate from the memory consumption brought about by the message itself, as tools already exist to control that (in the MTA).

regexp enhancements

The most widely suggested enhancements include braces ({n,m} counting), Perl-like \letter (\s, \t, \n, etc), better extraction operations, and locale support (hard to explain). Instead of hacking such things onto the current engine, it is likely that the entire engine will end up being replaced by another implementation that supports a more standard langauge. Backwards compatability will have to be maintained, of course.

${_} != $_

The former is the value of the underbar variable in the environment, the latter is the name of the current rcfile. No concensus emerged from the discussion on the development list on how to change this, so it is being left "as is".