From: Gálos András (galosanetinform.hu)
Date: Thu Sep 08 2005 - 18:53:05 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Helo (or rather ehlo) once more in this subject,

After a few days of reading and trying and debugging, finally I could set up
a per user based checking with a help of an sql database.
Let me summarize what I found, I hope it may be useful for others too.

Now I have one table (called postfix_filters) in the database and four
fields in it (among others) that determine what actions to do with the
actual recipient, subdomain, domain, or user of an incoming email.

These fields, and their sql type:
        RBL -> enum('rbl0', 'rbl1')
        SenderAddressVerification -> enum('sav0', 'sav1')
        SpamAssassin -> enum('sa0', 'sa1')
        Clamav -> enum('clam0', 'clam1')

I or even the users themselfs can modify these values to tune what they want
and what they don't want to check with messages addressed to them.

There were no problem with RBL checks or sender address verification (SAV):
smtpd_recipient_restrictions = ...
        check_recipient_access mysql:/etc/postfix/spam_rbl.cf
        check_recipient_access mysql:/etc/postfix/spam_sav.cf

And for example in /etc/postfix/spam_sav.cf:
hosts = 127.0.0.1
user = postfix
password = mypassword
dbname = users
table = postfix_filters
select_field = SenderAddressVerification
where_field = Recipient
additional_conditions = AND Enabled = '1'

I created restriction classes according to the values in the database in
main.cf:
smtpd_restriction_classes = rbl0, rbl1
                              sav0, sav1
rbl0 = reject_unauth_pipelining
rbl1 = reject_rbl_client sbl-xbl.spamhaus.org
          reject_rbl_client dnsbl-1.uceprotect.net
          reject_rbl_client relays.ordb.org
          reject_rbl_client dnsbl.njabl.org
sav0 = reject_unauth_pipelining
sav1 = reject_unverified_sender

These worked well without any problem.
The reject_unauth_pipelining lines are dummy lines, the message can get
through them without any action.

The harder were the content filters to me.
I successfully use clamav for virus scanning and spamassassin for spam
tagging.
First I tried to use amavisd-new and its database lookups. I wanted it to
search my postfix_filters table and if it finds sa0 in the SpamAssassin
record it should bypass calling spamassassin, and when it finds clam0 in the
Clamav field, it should bypass calling clamav. After days I had to give it
up. Amavis's sql usage were very strange and unclear for me, and even more,
some spamassassin feature were not availabe when calling it throug
amavisd-new.
Therefore I decided to use amavisd-new to call clamav, and used spamassassin
as a separate content filter.
Then the trouble was, that now I have two content filters, but can call only
one from smtpd_recipient_restrictions, or smtpd_sender_restrictions, etc.
Only the last one was executed, according to
http://www.postfix.org/FILTER_README.html.
But I needed to call two filters.
I could resolve this by calling one of the filters (amavisd-new/clamav) when
the message arrives on port 25, and call the other filter (spamassassin),
when the same message arrives on port 10025.
Amavisd-new accepts messages on port 10024, and sends them back on port
10025 after passing them to clamav. This is the base, many howtos deal with
it. I needed to query the database first, and if the Clamav field contains
clam1, the message has to be passed to amavisd-new, and when this field
contains clam0, amavisd-new has to be bypassed.
So now I use:
Another line in smtpd_recipient_restrictions:
smtpd_recipient_restrictions = ...
        check_recipient_access mysql:/etc/postfix/clamav.cf

/etc/postfix/clamav.cf is similar to the /etc/postfix/spam_sav.cf file
above, but with the Clamav value in the select_field line.

Another two restriction classes, so another line in
smtpd_restriction_classes
smtpd_restriction_classes = ...
                              clam0, clam1

And their definition:
clam0 = check_recipient_access pcre:/etc/postfix/clamav_bypass
clam1 = check_recipient_access pcre:/etc/postfix/clamav

These pcre tables are catch all tables.
So /etc/postfix/clamav:
/.*/
    FILTER smtp-amavis:[127.0.0.1]:10024

And /etc/postfix/clamav_bypass:
/.*/
    FILTER smtp:[127.0.0.1]:10025

When the database contains clam1, the message is filtered through
amavisd-new via port 10024. Then amavisd-new sends back the message on port
10025 if it's clean. In other case the message is discarded.
When the database contains clam0, the message is sent directly to port
10025, just like as amavisd-new sends the virus checked message back, but
amavisd-new is completely bypassed this way.

This way I could reach that every message arrives on port 10025 regardless
if they are checked by amavisd-new/clamav or not.
And here comes the second content filter. I could define it here, on port
10025 in master.cf!
The smtp server on port 10025 has its own smtpd_recipient_restrictions
parameter, which is where I set up spamassassin. It ovverides the same
setting in main.cf.
By default (according to the howtos) it contains:
127.0.0.1:10025 inet n - n - - smtpd
        ...
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        ...

(But of course connections to port 10025 are firewalled out anyway from
everywhere except localhost.)

I have to insert "check_recipient_access mysql:/etc/postfix/spam_sa.cf"
here, but these lines can't contain spaces in master.cf. Therefore I defined
it in main.cf:
sa_check = check_recipient_access mysql:/etc/postfix/spam_sa.cf

And used this variable in master.cf:
127.0.0.1:10025 inet n - n - - smtpd
        ...
        -o smtpd_recipient_restrictions=$sa_check,permit_mynetworks,reject
        ...

There is no problem with that whitespace character in it.
And /etc/postfix/spam_sa.cf is similar to the above, it tells postfix which
field is to be querried from the database from the postfix_filters table.
The return values are sa0 or sa1 here (No match, no return value? See
below). Another two restriction classes are defined in main.cf:
smtpd_restriction_classes = ...
                              sa0, sa1

And their definition:
sa0 = reject_unauth_pipelining
sa1 = check_recipient_access pcre:/etc/postfix/spamassassin

The reject_unauth_pipelining is a dummy action here, just to let the message
through.
The /etc/postfix/spamassassin file contains:
/.*/
        FILTER spamfilter:

And the usual definition in master.cf, written in many howtos, not detailed
here.

Huh, seemingly they worked fine, I can adjust each values freely in a per
user basis, it's great!
But there were a few more things to adjust:

When a message initially arrives on port 25, before putting it into the
postfix queue (?) address rewriting takes place. This means my virtual
aliases table is queried, and the recipient address gets rewritten according
to it. It's bad for me, because the spamassassin filter sits on port 10025,
and the messages arrive there with the rewritten rcpt to address. This means
the postfix_filters table is bad, because the first filter is queried with
the original recipient, but the second filter with the rewritten recipient.
Therefore I had to turn off address rewriting on port 25, and had to turn it
on on port 10025. With this, the virtual aliases are queried after the two
filters, both of them are called with the original recipient.
Additionally there is not wrong to turn off address rewriting on port 25 in
this case, because every message goes to port 10025, not only those that
amavisd-new sends there.
So, in main.cf:
receive_override_options = no_address_mappings

And in master.cf at the port 10025 stanza the receive_override_options has
not to contain no_address_mappings, only this:
127.0.0.1:10025 inet n - n - - smtpd
        ...
        -o
receive_override_options=no_unknown_recipient_checks,no_header_body_check

There was one more issue with this setup I bumped into:
If the querry on the postfix_filters table gets back no result (no matching
record), so no rectriction class, the incoming message is bounced with an
unknown user error. Regardless if this address is in the virtual users table
or not, it can't get there, because there is now way defined what
restriction classes to use (or something...).
So, it's important to add a record for each hosted domains in the
postfix_filters table with the default values of these four switches. All
four turned on probably. If a user wants spam, another record will be
inserted into this table that contains rbl0, sav0, sa0, or whatever to user
wants.
So, for example if a friend's mailserver is blocked by an rbl, he can turn
off rbl checking temporarely, until that important message arrives.
If the select field contains the domain only, so not galosanetinform.hu but
only netinform.hu, this will affect all email addresses in the domain, for
example otherusernetinform.hu also. So the default values will be given
back to postfix if a user queried that is not explicitely listed in the
postfix_filters table.

Another question is that what happens with messages that come with multiple
RCPT TO recipients, and these recipients have different settings in the
postfix_filters table. For example one wants all the four checks (rbl, sav,
clamav, spamassassin), the other one doesn't want them at all.
I don't clearly know why, but telneting some test messages to my server it
turned out that all the checks are executed for all the recipients. It
doesn't matter which recipient comes first in the smtp session, all gets
checked.
And this is the correct way in this case, since the defaults are to use all
the checks.

So, that's all about my story. I far don't know everything about Postfix,
but I think I went many steps ahead knowing it in the last few days. I do
hope other postfix users can find useful things in this message.

Oh, and one more thing: the sql querries are very easily debuggable by
turning on mysql's querry logging. I don't know any way to see how hash
table lookups are done in the practise, but it's very interesting to examine
the queries in the sql log file, it greatly helps understanding how the
lookups work.

And the last note: I (among others) think there is no need to check outgoing
messages. I mean outgoing, that coming from $mynetworks, either created
locally by webmail for example, or coming from hosts that this server is a
smarthost of. If the permit_mynetwork line is executed before the
check_recipient_access lines in smtpd_recipient_restrictions, then the
outgoing messages will be accepted before these checks, and will be sent out
unchecked, but fast and resource-cheap.

And please correct this schema if needed, I'm sure there are many things to
correct in it.

And thank you for your previous help in this thread also!

Regards,
Andras Galos

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]