Victor.Duchovnimorganstanley.com
Date: Mon Feb 02 2004 - 12:10:59 CST

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Mon, 2 Feb 2004, Wietse Venema wrote:

> > One more observation, the "REMOVE" interface is also used by "cleanup"
> > when deleting queue files for rejected messages and by "pickup" when
> > deleting "maildrop" files. If the facility is to be used for archiving, it
> > should only archive messages that entered the incoming queue.
> >
>
> The code that implements "dont remove" is a good starting point.
> It would take a new boolean parameter and a little extra queue manager.
>

Yes, the "dont_remove" flag, would I think need to move up from REMOVE()
to mail_queue_remove() which would need an extra argument, set only when
removing a queue file all whose recipients are delivered or bounced.

> Apart from those details, is this idea of parking old mail in an
> "old mail" queue a practical option to archive mail, and what
> support is missing? The whole idea is so similar to the "HOLD"
> feature that both may benefit from any tooling developed for
> inspection etc.
>

I have already invested in tooling up hold queue management for a user
visible quarantine where they can request message release. Indeed archive
processing from a "save" queue is feasible, but generally one wants to
archive mail off the MTA.

So one needs a "reaper" process that embalms dead messages with extracted
envelope information and dispatches them for burial. If the dispatch is
submission via SMTP or other Postfix assisted delivery, one also needs to
avoid putting the archive messages in the "save" queue.

If REC_TYPE_DONE is now guaranteed to be recipients only, my original idea
of zero copy archive delivery becomes viable again... This can deliver
into large maildir hierachies on a NetApp server with a 100 TB of storage,
or forward to a remote SMTP, ...

The question is whether it is easier or more natural to archive using
mechanisms similar to always_bcc (but with the delivery agent sending a
multipart mixed with a text/plain envelope sender + list of recipients one
to a line, and message/rfc822 atachment containing the message content),
or whether it is easier to deploy a largely external to Postfix
re-injector for the saved queue files.

Certainly using the saved files (if they truly only represent delivered
messages rather than all files deleted by Postfix) is less invasive in
terms of changes to the delivery agents and queue manager, and perhaps
offers more flexibility.

If a trigger message could be sent to a multi-server when messages are
added to the "save" queue, a "pickup-like" SMTP injector could do zero
copy SMTP submission, or maildir delivery, etc. and then delete the
message from the "save" queue. If zero copy is not an issue, one could
just use a modified "pickup" to submit an encapsulated message to the
cleanup service of a second Postfix instance that is responsible for
archive content delivery.

I think the idea has merit and flexibility. The main obstacle is saving
just the right files, and arranging for optional wakeup triggers.

--
        Viktor.

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]