OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: hmmmm....
From: Thomas Roessler (roesslerdoes-not-exist.org)
Date: Tue Sep 19 2000 - 13:23:37 CDT


On 2000-09-19 19:43:12 +0200, Brad Knowles wrote:

> It is my understanding that maildir with IMAP still requires file
> locking to work (because of the inherent multiple simultaneous
> processes that are accessing the mailbox) , and therefore is
> fundamentally incompatible with NFS.

I think you can actually work around this, at the cost of metadata
updates. (Note that this is merely a back-of-the-envelope idea;
what I'm proposing is not efficient, but it may be a starting point
for doing things reliably.)

The most important thing here is that you can do all the
modifications which don't affect the message file's content in a way
which doesn't require file locking (the actual locking happens on
the file system level). IMAP doesn't need to modify files'
contents.

The hard part about accessing maildir folders is not concurrent
access between MUA/imapd and the local delivery agent (that's
entirely harmless since the LDA doesn't get into the MUA's way), but
concurrent access of different MUAs which may delete or modify
messages under each others' feet.

The simplest way to deal with this is to regularly scan for updates,
using the unique part of the file name as a unique message-ID, and
appropriately updating structures. This works iff you can say "ups,
the message doesn't exist any longer" when the user wants to read it.

However, if the IMAP details in the back of my head are correct, an
EXPUNGE is not permitted/desirable as a reply to message-reading
commands from the client side. Thus, we need to make sure that we
have a copy of (or rather: a link to) each message available which
can be accessed when the message no longer exists in the "real"
folder.

To achieve this, extend the maildir structure by a working directory
for each imapd accessing the folder:

        Maildir/
                cur/
                new/
                tmp/

                work.$HOSTNAME.$TIMESTAMP.$PID/

Whenever imapd updates it's internal image of the maildir, it
creates a hard link in the work directory to the actual message file
in cur/ or new/. Since this kind of updates will happen at
opportunities where an EXPUNGE is permitted, failure to create the
link can be safely interpreted as a deleted message, and the
possibly necessary EXPUNGE can be sent to the client (or the server
can just forget about that message if the client didn't know about
it).

Since the message flags are stored in memory, the unique part of the
file name should be used for the backup link. Normally, imapd
accesses the "real" maildir. However, whenever a file to be
accessed no longer exists (and there is no file with the same unique
name, but a different flags part - checking for such a file can be
reduced to a couple of open() or stat() calls if walking the
directory in user space is undesirable), imapd has to access the
working copy of the file. At the next opportunity, the client is
told about the fact that the file was expunged, and the working copy
is unlinked.

The problem with this approach is obviously the fact that there are
lots of metadata updates involved.

Flames and all kinds of criticism are welcome.

-- 
Thomas Roessler                         <roesslerdoes-not-exist.org>