From: Ronald F. Guilmette (rfgmonkeys.com)
Date: Fri Dec 01 2006 - 16:27:55 CST

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

[[ My apologies to everyone on the list if you see this twice. I tried
   to post this messge to the list _before_ my "Enhancement Request #2"
   messge, but for some mysterious reason, THAT message seems to have
   made it out to the list promptly, but this one didn't. So I'm giving
   it a second try. ]]

I asked recently about Postfix-supported techniques for performing
"backscatter" rejection. Obviously, Postfix does have some capabilities
for doing this sort of thing, but from my perspective, the current
capabilities are not entirely optimal. (I should clarify here that
what I really need and want to do is to protect real live mailboxes,
which belong to real live people, including myself, from essentially
all "unsolicited" automated response messages, even while allowing
"valid" autoresponses... including bounces... in. For my purposes,
its not enough to simply protect old or retired domains.)

So anyway, I was doing some additional head-scratching about this, when
I came upon the SMTPD_POLICY_README file. I may have read this file
some long time ago, but if I did, then I rather entirely forgot about
this wonderful (policy sever) capability that's built into Postfix.
Anyway, I love it, because it's very very general, and allows an external
policy server to perform all sorts of interesting calculations, at all
different phases of the e-mail reception process. (Kudos, as always,
to Wietse!)

I'm thinking now about building a policy server of my own to do several
different interesting, and hopefully useful things.

First on the list is doing what I would consider to be intelligent back-
scatter rejection. The task here can be roughly broken up into two parts.
The first part is simply recognizing/differentiating those messages which
are automated response messages (a subset of which includes "bounces")
from all other messages. Part two is to look at the "sender" for each
automated message, and to try to determine if the sender... or more pro-
perly the sender's e-mail address... is one from which the current message
recipient might reasonably be expecting to receive some sort of an auto-
mated response message, e.g. the current recipient previously (and recently?)
sent a message to the address that's now sending back some sort of auto-
mated response message.

Both parts of this overall task would be greatly facilitated by the ability
to pass certain mail message headers from the current (incoming) message
from Postfix to the external policy server.

For example, in my own attempts to build a simple tool to differentiate
automated response messages from all other e-mail messages, I quickly
learned that performing this differentiation is no easy task. There is
little or no standardization among such messages. Although many of
these kinds of messages can be differentiated based upon the envelope
sender information (e.g. any envelope sender address beginning with
"MAILER-DAEMON") an unfortunately large percentage of automated response
messages cannot be recognized as such without actually searching for the
presence or absence of certain (static) header text, e.g.

        Content-Type: multipart/report;...
        User-Agent: Vacation/...
        X-MS-Embedded-Report:...
        From: Mail Delivery Subsystem...
        Subject: MDaemon Notification...

So obviously, a Postfix policy server cannot perform even the first step
in the two step process that I'd like to implement (for intelligent back-
scatter filtering) unless and until Postfix is ready, willing, and able
to pass the entire text of certain pre-specified headers from the current
message to the policy server, e.g. "Content-Type", "User-Agent", "From",
"Subject", etc.

Separately, in phase two, when a determination has already been made that
a given incoming message is in fact an automated response message, it would
then become useful... if not to say essential... to likewise be able to
pass the contents of certain headers from the current message to the
policy server. I am speaking of such headers as "From", "Sender",
"Resent-Sender" and so forth. Whereas automated response messages often
are accompanied by null envelope sender addresses... thus making determi-
nation of the true sender essentially impossible... the actual sender's
address can often be found in the various sender-related headers such
as "From" and/or "Sender". E-mail address values could be parsed out of
those mail headers (by a sufficiently intelligent policy server) and could
then be used as lookup keys against a data base of addresses to which the
current message recipient had previously (recently?) sent outbound mail.
In other words, given the capability to pass certain headers to the policy
server, the policy server could isolate any and all plausible "sender"
e-mail address, and it could then look up each of those in a data base
to see if the current message recipient had sent mail to any of those
addresses recently. If so, then the current automated response (bounce?)
message would be accepted by Postfix, otherwise it would be rejected as
an "unsolicited backscatter" message.

I can implement all of this stuff. Indeed, I already have a lot of the
code written to do it all. What I lack is the ability to simply tell
Postfix to pass the content of certain mail headers, e.g. "Subject",
"Content-Type", "From", "Sender", etc., etc... to my yet-to-be-built
policy server. So I'm requesting that (new) capability in Postfix.

I want to make it clear however that I believe that the (still hypothetical)
ability to pass certain specific mail headers to a Postfix policy server
could potentially have quite a lot of uses *in addition to* building a
good quality, intelligent backscatter filter. Just as one example, I'm
thinking that I'd like to build my own SPF policy server. But if I'm
going to build *that*, then I'd kind-of like to also make it smart enough
so that it can do other, similar sorts of things. To be specific, I'd
like to build a Postfix policy server implementation of the so-called
"DomainKeys" client validation scheme. I don't actually know much about
that at this point, but the one thing that I do know is that the whole
scheme seems to revolve around the use of a magical mail header called
the "DomainKey-Signature" header. So obviously, if one desired to con-
struct a Postfix policy server to implement DomainKeys, it would appear
to be helpful, if not to say outright necessary, to be able to pass at
least any "DomainKey-Signature" header that is present in the current
message to the policy server.

I feel sure that other people could probably dream up yet more uses for
the feature, if and when Postfix allows specific mail headers to be passed
to an external policy server. In short, I think that there are a number
of good arguments in favor of this exact type of enhancement, and I hope
and trust that Wietse will give it fair and ample consideration.

Regards,
rfg

P.S. While I was reviewing the protocol described in the SMTPD_POLICY_README
document today, I was struck by the (vague?) similarities between the pro-
tocol that Postfix currently uses to talk to external policy servers and
the HTTP protocol. At the very least, both seem to be request/response
oriented, and both seem to use a single blank line to terminate at least
the "headers" part of both the request message and also the response message.
And although it appears that Postfix generally likes to keep its connections
to its policy server(s) open continuously, it is my understanding that the
modern HTTP protocol allows for this sort of persistant connection also.

I guess what I'm driving at is that if Postfix's policy server protocol
were, hypothetically, someday to be conformed more closely to the HTTP
protocol, then some Interesting Possibilities might conceivably ensue.
(Then again, they might not.) I could certainly envision a scenario
wherein I or someone else might operate an HTTP-based Postfix policy
server CGI, via the operator's web server, and then make that available,
over the net, to other parties.

Well, it's just sort of an off-the-cuff idea. At this point, there's
no real reason, either good or otherwise, to do things that way... the
current protocol is perfectly fine and usable just as it is. I just
wanted to mention the interesting (if minimal) similarities between
the Postfix policy server protocol and HTTP, just in case nobody else
had noticed those.

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]