Victor.Duchovnimorganstanley.com
Date: Sat Jun 28 2003 - 10:45:28 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Sat, 28 Jun 2003, Wietse Venema wrote:

[ Paragraph moved up ]
> For example, I would implement archival with just a little delivery
> agent that spits out the envelope before spitting out the content
> as message/rfc822, and then injects the result into the queue just
> like Postfix normally forwards mail internally. It costs a few
> more cycles, but it is trivial to implement and most importantly,
> it does not require surgery of existing core infrastructure.
>

Exactly! This is the design goal, to do extremely efficient archiving out
of the same queue file via suitable delivery agents without cloning the
message. For wart-free behaviour, I do not want archive recipients to be
visible in error/delay notices. I would also like archive deliveries to
always have soft_bounce = yes.

> You basically introduce multiple classes of recipients in the same
> queue file. One recipient class has pre-empts delivery to another
> class; one class produces DSNs and the other doesn't. Since the
> whole concept does not exist, major surgery is required in the
> queue manager and in the way that bounce reports are being handled.
>

Is not some of this already necessary to implement REDIRECT (DIVERT) in a
wart-free fashion? (Log completeness/consistency, once per message
delivery, original recipient handling, ...) It is possible that REDIRECT
becomes simpler as a special recipient than as a flag, because delivery
looks much like delivery to any other recipient.

Also currently REDIRECT pre-empts always_bcc. If one is using always_bcc
for poor-man's archiving, the non-archiving of REDIRECT messages may be a
problem.

We have some klunky divert/screen capabilities in our content filter. They
are used for handling persistent email harrassment and compliance
investigations. "Divert" reroutes mail from harrassers to corporate
security for evidence collection in cease/desist orders, "screen" is
almost obviated by "always_bcc", except that "screen" is invisible to the
sender (no delay or error notices).

On further thought I don't think we need orthogonal recipient classes,
just a few combinable recipient flags:

- A flag to supress sender notification. (ARCHIVE, DIVERT, SCREEN)

- A flag to suppress hard bounces (per recipient soft_bounce) (ARCHIVE,
  SCREEN, DIVERT)

        archive_soft_bounce = yes/no
        screen_soft_bounce = yes/no
        divert_soft_bounce = yes/no

- A flag to pre-empt delivery of unflagged recipients (DIVERT)

- A flag to deliver despite pre-empt (ARCHIVE)

> Each feature comes at a cost. I wonder if the cost of this one is
> justified, considering that almost no-one will be using it.
>

The financial services industry has stiff regulatory requirements for
archiving. Other industries may have similar requirements. So while I
don't know how much use REDIRECT will get, archiving will have a
non-negligible community of users, be it a fraction of the entire user
base.

> Remember how the external content filter was implemented. It was
> not added as a different mail handling process internal to Postfix.
> Instead it uses all the existing infrastructure and external
> interfaces. The additional complexity and overhead is experienced
> only by those who actually want to use this functionality. The
> price is that mail changes queue file upon filtering. The benefit
> is that a lot of complexity never needed to be implemented.
>

Yes, indeed my initial idea for archiving was just the archive delivery
agent. The special notification semantics are perhaps scope creep, but I
do not want to expose archiving failures to the sender, and want a second
chance to archive even if delivery fails. Should it ever happen, it would
be embarrassing (if not outright costly) to have an email message from a
regulator return an archive failure bounce message...

> If other exotic features can be implemented by chaining delivery
> agents (including special-purpose delivery agents) then I would
> favor that over burdening regular mail delivery with a lot of code
> that almost no-one uses. Add new functions on the periphery if
> possible, instead of burdening the core with ever increasing
> complexity.
>

Fair enough, and consistently with this the archiving logic goes
mostly into the delivery agents.

If the queue manager merely kept track of flags, and passed them around to
"bounce" and the delivery agents, could the special bounce handling be
delegated also?

Perhaps just enhancing soft_bounce to work per-recipient, and suppressing
delay notices to soft_bounce recipients would get most of the job done.

The most complex mechanism in this space is in fact the already
implemented REDIRECT/DIVERT. ARCHIVE and SCREEN don't pre-empt other
deliveries and behave like ordinary recipients in every way except when
bouncing.

I am looking for design feedback, any suggestions? I am volunteering to
implement any sensible design in this space that meets with your
architectural goals and still accomplishes my archiving/screening
requirements.

--
        Viktor.

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]