OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: idea: content filtering
From: Russ Allbery (rrastanford.edu)
Date: Fri May 12 2000 - 15:12:26 CDT

Wietse Venema <wietseporcupine.org> writes:

> It can be part of a separate resident server, so that the cost is twice
> the cost of Postfix (mostly disk I/O) plus the cost of the resident
> filter (mostly CPU, unless it causes the machine to swap).

Not sure why it would require disk I/O after it's started up? Unless the
message has to be written out to disk to be passed to the filter, which I
guess is a reasonable design. I personally would try to keep things in
memory as much as possible, but I'm possibly biased by previous experience
with news servers that always keep the entire message in memory while
they're processing it.

> I'm glad I am a multi-lingual person. However in this case it should be
> possible to handle the IPC in PERL. SMTP is easy if you don't have to
> talk to every broken mailer in the world. And because there is not a
> network involved, it is OK when PERL needs a bit of time before it
> replies to ".".

The main reason why I'd be inclined to write the IPC in C rather than Perl
is to take advantage of a general framework that allows the same code to
be reused for an embedded filter, a standalone separate daemon reading
things through pipes, or even a standalone separate daemon using shared
memory or some other faster mechanism than pipes. It's easier to build
that sort of thing in C, but it's a mite out of scope for Postfix proper.

The other reason, though, is speed. Perl errs on the side of safety and
flexibility, at the cost of speed, when determining its memory allocation
strategy, and reading messages into Perl tends to be rather wasteful in
terms of memory allocation overhead in my experience. With C, you have
tight control over your buffer allocation (at the cost of having to manage
it yourself directly) and can bless only and exactly what you want to pass
to Perl and make sure Perl leaves it alone and just reads it.

As an aside, though, there's already an external filtering interface
"specification" of sorts, since I believe both Cyclone and Diablo use the
same interface to an external filtering daemon. They didn't want to do
the work of embedding Perl directly into the server. 

-- 
Russ Allbery (rrastanford.edu)             <http://www.eyrie.org/~eagle/>