OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Some questions on postfix, SUMMARY
From: Wietse Venema (wietseporcupine.org)
Date: Fri Mar 03 2000 - 09:44:01 CST

Karimov, Rashid:
> > > 1. On a slow system, fork()/exec() pair will be slow :)
> >
> > Have you tried this? Unless you run a dog of an operating system
> > you should be able to fork/exec a hundred or more times a second.
> > That's more than enough, given that it takes much more time to
> > write mail to disk.

I still need to see the argument that fork/exec is prohibitively
expensive. With the present queue organization of one message/file,
creating a file is orders or mangitude slower than spawning a
process.

> > There is the other variant, and that is to take a few Postfix
> > routines and write a sendmail-like program that injects directly
> > into the queue. However, that program would have to run set-uid
> > postfix, which is OK only in a tightly-controlled environment.
> >
> > If that is an option to consider, then I could write such a beast.
> > But it would be an optional thing, not recommended for general use.

> I think there is definitely a demand for a thing like this, may be
> even just a fast routine (thread-safe would be great) one can call:
>
> maildrop_it(** recepients, *message_buffer)

I prefer an open(), write() and close() routine instead. That allows
one to submit messages that are larger than is convenient to keep
in memory. Maybe it is because I am an old fart who grew up with
machines with less than a megabyte of main memory.

> that one can use in a custom message generation program. If it is
> a cross-platform (WinNT), it would be a real killer .

I have no idea. My programming experience is limited to UNIX VMS
DOS and a little bit of Windoze.

> Some of our messages are generated on WinNT (don't ask me why ! :),
> and if an NT system can just drop off a message into NFS-mounted maildrop
> queue, it would ease the transition.

The Postfix queue routines expect UNIX file system semantics and
need file inode numbers in order to avoid file name collisions.
Choosing the wrong file name can result in loss of mail.

> And we are talking about lots of these messages - up to 1 Mil per event (!).

Queueing up a million files per hour is 300 files a second. With
the present queue organization of one message per file, that requires
a lot of disk spindles to share the load, and a persistent write
cache that allows the system to sort disk writes for optimal speed.

Since this spends most of the time in file creation overhead, it
could be done quicker with a different queue organization where
messages are appended to a log, say a thousand messages per log,
and where each log is processed sequentially.

But is a major change to Postfix and won't happen this week.

> - you mention in the docs, that there is very little chance that an
> arbitrary
> message dropped into maildrop can be considered a valid postfix message -
> how
> is it protected ? Is there a signature of some kind that only is generated
> when
> postfix's sendmail is properly called ? Is it the mail_stream_file() that
> generates the signature ?

BTW did you notice that your word-wrapping algorithm sucks?

Postfix queue files have a very specific format, and if the file
format does not match expectations, then they are rejected. The
format is enforced by the Postfix queue file writing routines.

> I would like to help you (and the community) with writing this routine (see
> above),
> and any input from the author or people-in-the-know would be appreciated.

It's not a problem for me to write a direct injection routine.
Explaining how to do it would take more time than writing it.

        Wietse