On Sat, Jan 06, 2001 at 08:04:10AM +0000, Ralf Hildebrandt wrote:
| On 6 Jan 2001 00:01:49 +0100, J.D. Bronson <jeffsparc-central.com> wrote:
| >can someone explain the difference between these:
|
| connect from smtpd_client
| HELO helo
| MAIL FROM:<smtpd_sender>
| RCPT TO:<smtpd_recipient>
| DATA
| header_checks
| body_checks
| .

Last month I posted a short doc discussing postfix's UCE.
I've learned a lot since then; here's an improved version.
comments and criticism welcome, of course.

------------------------------------------------------------

         My Understanding Of How UCE Actually Works
             v1.1, mengwongpobox.com 20010109

   http://www.postfix.org/uce.html exhaustively describes
   the syntax and semantics of several configuration
   options, but not their pragmatics. How come
   configuration values can appear in multiple places? What
   does it mean when they do?

Let's start by getting some preliminaries out of the way
before we move on to discussing the more complex UCE
restriction options.

First, a dress code requirement. If the smtp client fails
either of the following variables, it's thrown out.

        smtpd_helo_required = yes/no
    strict_rfc821_envelopes = yes/no

The header_checks and body_checks variables define access
maps, which we'll look at later.

    header_checks = maptype:mapname (usually regexp.)
      body_checks = maptype:mapname (usually regexp.)

Maps are lookup tables. Postfix supports several kinds of
maps (hash and regexp are the most common; see postconf -m),
and uses them for several functions, including alias maps
(which rewrite one address into another), transport maps
(which specify a transport for a given address), and access
maps, which tell postfix whether to permit or deny a given
smtp transaction, and can do other things besides, things
weird and wonderful, such as recursively calling other
restriction lists, but I'm getting ahead of myself.

The meat of postfix's UCE configurations are found in
parameters that are all named smtpd_*_restrictions.

Access maps are one possible kind of smtpd restriction.
there are others.

Think of a restriction as a function that returns one of OK,
REJECT, or PASS. If you're reminded of router access-lists,
good: you've recognized the pattern. Some restrictions only
return OK or PASS, some return only REJECT or PASS, and some
are empowered to return a verdict either way. If the
restriction doesn't have a strong feeling one way or the
other, it just returns a PASS, and postfix goes on to
evaluate the next restriction. All restrictions may PASS
except check_relay_domains, which has to return either OK or
REJECT. For that reason people generally set it as the last
restriction to be evaluated.

The restrictions are grouped into lists and evaluated in
order within each list.

If the restriction returns either OK or REJECT, the
list short-circuits and immediately concludes with that
value.

If a list concludes in REJECT, the smtp client will be denied.

The default result for all lists is OK.

Here's a symbolic representation:

  listA { r0; r1; r2; }
  listB { r2; r3; r4; }
  listC { r0; r2; r3; r4; r5; r6; }
  listD { r2; r5; r6; r7; }

Note that later lists have access to more and more
restrictions:

  listA may run r0, r1, and r2.
  listB may run all of listA's, plus r3 and r4.
  listC may run all of listB's, plus r5 and r6.

>>> Why "plus all of the previous list's restrictions"?

  We'll get to that later. For now just keep in mind that
  each list has certain restrictions that belong to it; in
  addition to those, it can run other restrictions that
  belong to the preceding lists.

>>> So, what are the lists really called?

  listA: smtpd_client_restrictions
  listB: smtpd_helo_restrictions
  listC: smtpd_sender_restrictions
  listD: smtpd_recipient_restrictions

The order is important. They are evaluated in that order,
and they "inherit" restrictions in that order.

If all the lists return OK, the client is 354 invited to
DATA the message over, and then after "." completion,
header_checks and body_checks run to give postfix one last
chance to reject the message instead of saying "250 OK:
queued as whatever."

>>> So, what are the functions available to each list?

All lists can run the following generic restrictions:

  permit (default when postfix runs off the end of a list)
  reject
  reject_unauth_pipelining

smtpd_client_restrictions can include all of the above, plus:

  check_client_access maptype:mapname
  permit_mynetworks
  reject_unknown_client
  reject_maps_rbl

smtpd_helo_restrictions can include all of the above, plus:

  check_helo_access maptype:mapname
  reject_invalid_hostname
  reject_unknown_hostname
  permit_naked_ip_address
  reject_non_fqdn_hostname

smtpd_sender_restrictions can include all of the above, plus:

  check_sender_access maptype:mapname
  reject_unknown_sender_domain
  reject_non_fqdn_sender

smtpd_recipient_restrictions can include all of the above, plus:

  check_recipient_access maptype:mapname
  permit_auth_destination
  permit_mx_backup
  reject_non_fqdn_recipient
  reject_unauth_destination
  reject_unknown_recipient_domain
  check_relay_domains (should be placed last, because it doesn't PASS)

smtpd_etrn_restrictions is a restriction list which isn't
really targeted at controlling UCE and therefore only gets a
passing mention here. it can run the helo, client, and
generic sets of restrictions, but not the sender or
recipient restrictions.

Restrictions are evaluated in the order they are listed.

If none of the restrictions has returned a REJECT, postfix
invites the client to send the data over. After the data
has been received and the client has sent a ".", postfix
applies header_checks and body_checks. This is your last
chance to reject the message.

Note: if you specify maptype:mapname without saying
check_*_access before it, the * is resolved according
to the smtpd_*_restrictions list that it appears in.

Note: while a restriction may result in a rejection early on
in the SMTP exchange, postfix waits until RCPT has been
received to send back a rejection code.

>>> What's the point of running a restriction in a later
    block? Shouldn't all client-related restrictions belong
    inside the client_restrictions list, and so on?

Let's do an example. Suppose you're big on spam-defense,
and so you've turned on reject_maps_rbl. The gauntlet
you've erected looks like this:

client: { reject_maps_rbl }
helo: { }
sender: { }
recipient: { permit_mynetworks, check_relay_domains }

You're happily rejecting any client who's listed in the RBL,
DUL, and RSS provided by MAPS. Now, MAPS does mention that
some legitimate mail will be sacrificed in the interests of
keeping out more general spam. For example, at the time of
this writing (200101), American Express's Internet Travel
Network, itn.net, is blacklisted by the RSS, but you don't
want to miss any important mail (smtp sender addresses of
pnr-notificationitn.net). Taking matters into your own
hands, you explicitly OK that address in your sender-access
map, and you try:

client: { reject_maps_rbl }
helo: { }
sender: { hash:sender-access }
recipient: { permit_mynetworks, check_relay_domains }

But wait! That doesn't work, because reject_maps_rbl, which
runs before we even know what the sender address is, returns
REJECT, and that's enough for smtpd_client_restrictions to
deny the client.

What you really want is

client: { }
helo: { }
sender: { hash:sender-access, reject_maps_rbl }
recipient: { permit_mynetworks, check_relay_domains }

so that the sender-access gets first crack at explicitly
approving the message before reject_maps_rbl denies it due
to RSS.1

>>> Then what's the point of the first three lists? If
>>> recipient_restrictions are a superset of all the others
>>> and if the order of evaluation is linear and fully
>>> specifiable, why not put everything in the
>>> recipient_restrictions list?

Maybe you want to express complex logic. You might want to
set up a series of explicit permits followed by possible
denies:

    listA: { permit1; deny1; }
    listB: { ... deny2; }
    listC: { ... deny3; }

if permit1 returns OK, and you still want to test against
deny2 and deny3, then you need more than one list ---
otherwise, permit1 would short-circuit you past deny2 and
deny3.

    listA: { permit1; deny1; deny2; deny3; } # not what you want

>>> Gotcha --- but how complex can you get with only four
>>> restriction lists?

Ah, but you aren't restricted to only four lists. The
smtpd_restriction_classes option lets you make up new
restriction lists.

As the smtpd man page says, smtpd_restriction_classes
declares the name of zero or more parameters that contain a
list of UCE restrictions. The names of these parameters can
then be used instead of the restriction lists that they
represent.

    smtpd_restriction_classes = mylist1,
                                mylist2
    mylist1 = check_sender_access foo:bar,
              reject_non_fqdn_recipient,
              reject_unknown_sender_domain,
              check_recipient_access maptype:mapname
              reject_non_fqdn_sender,
              etc etc etc.
    mylist2 = permit_auth_destination,
              permit_mx_backup,
              reject_unauth_destination,
              reject_unknown_recipient_domain

>>> Nice. But how do these lists get called?

The smtpd_*_restriction lists get called naturally as the
SMTP transaction proceeds. But restrictions lists that you
make up ... well, where's the entry point to those?

It turns out that access maps support a little-known
feature: on the right hand side of the access map you can
define OK, REJECT, and 500 and 400 error codes; this you
already knew.

But did you know that instead of saying OK or REJECT, you
can hop back up a level and define a list of UCE
restrictions?

Read uce.html carefully:
    Reject the request if the result is REJECT or "[45]XX
    text". Permit the request if the result is OK or RELAY
    or all-numerical. Otherwise, treat the result as another
    list of UCE restrictions.

That list is allowed to contain restrictions,
eg. reject_unknown_sender_domain, reject_non_fqdn_sender,
reject_unauth_destination, etc, etc --- but it is NOT
allowed to contain another access map.

That's when you bring out the smtpd_restriction_classes that
you made up. You can set up my_restrictions to check
another access map:

    smtpd_restriction_classes = my_restrictions
    my_restrictions = reject_non_fqdn_sender,
                      check_sender_access regexp:other_sender_access

You could call my_restrictions directly from a standard
smtpd_*_restrictions list, or you could call from inside an
access map.

    smtpd_recipient_restrictions = check_recipient_access regexp:my_recipient_regexp

    my_recipient_regexp:
        /localuser1/ OK
        /localuser2/ REJECT
        /localuser3/ reject_unknown_sender_domain, reject_non_fqdn_sender
        /localuser4/ reject_unknown_sender_domain, my_restrictions

Mail to localuser4localdomain gets tested against the
following restriction functions:

   1) check_recipient_access regexp:my_recipient_regexp
   2) reject_unknown_sender_domain
   3) my_restrictions:
   4) reject_non_fqdn_sender
   5) check_sender_access regexp:other_sender_access
   6) ... and so on ...

(my_recipient_regexp could have been a hash. I know.)

                     --- questions ---

>>> Shouldn't reject_unauth_pipelining be a yes/no parameter
>>> rather than a generic restriction?

Good question.

>>> Why does ITN.net, presumably a well-funded website with deep
>>> pockets backed by American Express itself still run as an
>>> open relay?

Good question.

>>> Is it worth hurting your users a lot in the short run
>>> just to teach third parties not to hurt other peoples' users
>>> a little in the long run? (a little = receiving spam that
>>> you didn't want. a lot = not receiving mail that you did.)

Perhaps it would be better to add a header line to the
message and passing it on for the LDA to shunt to a
low-priority mailbox, or for the MUA to flag as spam.

See http://www.kfki.hu/~kadlec/sw/postfix_patch.html for an
ingenious patch that allows users to decide whether they
want to use each particular MAPS (rbl, rss, dul) or not.

           ----- appendix: man smtpd_check -----

If you look in src/man, you'll see that the man pages are
generated from the source code using a srctoman | nroff -man
pipeline. Only some of the .c files, such as smtpd.c, are
converted into man pages; others, such as smtpd_check.c, are
not.

20001221-01:58:05 mengwongdumbo:/usr/local/src/postfix-19991231-pl09% mantools/srctoman smtpd/smtpd_check.c | nroff -man | less

SMTPD_CHECK(3) SMTPD_CHECK(3)

NAME
       smtpd_check - SMTP client request filtering

SYNOPSIS
       #include "smtpd.h"
       #include "smtpd_check.h"

       void smtpd_check_init()

       char *smtpd_check_client(state)
       SMTPD_STATE *state;

       char *smtpd_check_helo(state, helohost)
       SMTPD_STATE *state;
       char *helohost;

       char *smtpd_check_mail(state, sender)
       SMTPD_STATE *state;
       char *sender;

       char *smtpd_check_rcpt(state, recipient)
       SMTPD_STATE *state;
       char *recipient;

       char *smtpd_check_etrn(state, recipient)
       SMTPD_STATE *state;
       char *recipient;

DESCRIPTION
       This module implements additional checks on SMTP client
       requests. A client request is validated in the context of
       the session state. The result is either an error response
       (including the numerical code) or the result is a null
       pointer in case of success.

       smtpd_check_init() initializes. This function should be
       called once during the process life time.

       Each of the following routines scrutinizes the argument
       passed to an SMTP command such as HELO, MAIL FROM, RCPT
       TO, or scrutinizes the initial client connection request.
       The administrator can specify what restrictions apply.

       Restrictions are specified via configuration parameters
       named smtpd_{client,helo,sender,recipient}_restrictions.
       Each configuration parameter specifies a list of zero or
       more restrictions that are applied in the order as speci-
       fied.

       Restrictions that can appear in some or all restriction
       lists:

       reject

       permit Reject or permit the request unconditionally. This
              is to be used at the end of a restriction list in
              order to make the default action explicit.

       reject_unknown_client
              Reject the request when the client hostname could
              not be found. The unknown_client_reject_code con-
              figuration parameter specifies the reject status
              code (default: 450).

       permit_mynetworks
              Allow the request when the client address matches
              the mynetworks configuration parameter.

       maptype:mapname
              Meaning depends on context: client name/address,
              helo name, sender or recipient address. Perform a
              lookup in the specified access table. Reject the
              request when the lookup result is REJECT or when
              the result begins with a 4xx or 5xx status code.
              Other numerical status codes are not permitted.
              Allow the request otherwise. The
              access_map_reject_code configuration parameter
              specifies the reject status code (default: 554).

       check_client_access maptype:mapname
              Look up the client host name or any of its parent
              domains, or the client address or any network
              obtained by stripping octets from the address.

       check_helo_access maptype:mapname
              Look up the HELO/EHLO hostname or any of its parent
              domains.

       check_sender_access maptype:mapname
              Look up the resolved sender address, any parent
              domains of the resolved sender address domain, or
              the localpart.

       check_recipient_access maptype:mapname
              Look up the resolved recipient address in the named
              access table, any parent domains of the recipient
              domain, and the localpart.

       reject_maps_rbl
              Look up the client network address in the real-time
              blackhole DNS zones below the domains listed in the
              "maps_rbl_domains" configuration parameter. The
              maps_rbl_reject_code configuration parameter speci-
              fies the reject status code (default: 554).

       permit_naked_ip_address
              Permit the use of a naked IP address (without
              enclosing []) in HELO/EHLO commands. This violates
              the RFC. You must enable this for some popular PC
              mail clients.

       reject_non_fqdn_hostname

       reject_non_fqdn_sender

       reject_non_fqdn_recipient
              Require that the HELO, MAIL FROM or RCPT TO com-
              mands specify a fully-qualified domain name. The
              non_fqdn_reject_code parameter specifies the error
              code (default: 504).

       reject_invalid_hostname
              Reject the request when the HELO/EHLO hostname does
              not satisfy RFC requirements. The underscore is
              considered a legal hostname character, and so is a
              trailing dot. The invalid_hostname_reject_code
              configuration parameter specifies the reject status
              code (default:501).

       reject_unknown_hostname
              Reject the request when the HELO/EHLO hostname has
              no A or MX record. The unknown_host-
              name_reject_code configuration parameter specifies
              the reject status code (default: 450).

       reject_unknown_sender_domain
              Reject the request when the resolved sender address
              has no DNS A or MX record. The
              unknown_address_reject_code configuration parameter
              specifies the reject status code (default: 450).

       reject_unknown_recipient_domain
              Reject the request when the resolved recipient
              address has no DNS A or MX record. The
              unknown_address_reject_code configuration parameter
              specifies the reject status code (default: 450).

       check_relay_domains
              Allow the request when either the client hostname
              or the resolved recipient domain matches the
              relay_domains configuration parameter or a subdo-
              main therereof, or when the destination somehow
              resolves locally ($inet_interfaces, $mydestination
              or $virtual_maps). Reject the request otherwise.
              The relay_domains_reject_code configuration parame-
              ter specifies the reject status code (default:
              554).

       permit_auth_destination
              Permit the request when the resolved recipient
              domain matches the relay_domains configuration
              parameter or a subdomain therereof, or when the
              destination somehow resolves locally ($inet_inter-
              faces, $mydestination or $virtual_maps).

       reject_unauth_destination
              Reject the request when the resolved recipient
              domain does not match the relay_domains configura-
              tion parameter or a subdomain therereof, and when
              the destination does not somehow resolve locally
              ($inet_interfaces, $mydestination or $vir-
              tual_maps). Same error code as
              check_relay_domains.

       reject_unauth_pipelining
              Reject the request when the client has already sent
              the next request without being told that the server
              implements SMTP command pipelining.

       permit_mx_backup
              Allow the request when the local mail system is
              mail exchanger for the recipient domain (this
              includes the case where the local system is the
              final destination).

       restriction_classes
              Defines a list of parameter names, each parameter
              being a list of restrictions that can be used any-
              where a restriction is legal.

       smtpd_check_client() validates the client host name or
       address. Relevant configuration parameters:

           smtpd_client_restrictions
                  Restrictions on the names or addresses of clients
                  that may connect to this SMTP server.

       smtpd_check_helo() validates the hostname provided with
       the HELO/EHLO commands. Relevant configuration parameters:

           smtpd_helo_restrictions
                  Restrictions on the hostname that is sent with the
                  HELO/EHLO command.

       smtpd_check_mail() validates the sender address provided
       with a MAIL FROM request. Relevant configuration parame-
       ters:

           smtpd_sender_restrictions
                  Restrictions on the sender address that is sent
                  with the MAIL FROM command.

       smtpd_check_rcpt() validates the recipient address pro-
       vided with an RCPT TO request. Relevant configuration
       parameters:

           smtpd_recipient_restrictions
                  Restrictions on the recipient address that is sent
                  with the RCPT TO command.

       smtpd_check_etrn() validates the domain name provided with
       the ETRN command, and other client-provided information.
       Relevant configuration parameters:

           smtpd_etrn_restrictions
                  Restrictions on the hostname that is sent with the
                  HELO/EHLO command.
                  
       smtpd_check_size() checks if a message with the given size
       can be received (zero means that the message size is
       unknown). The message is rejected when:

       o The message size exceeds the non-zero bound speci-
              fied with the message_size_limit configuration
              parameter. This is a permanent error.

       o The message would cause the available queue file
              system space to drop below the bound specified with
              the min_queue_free configuration parameter. This is
              a temporary error.

       o The message would use up more than half the avail-
              able queue file system space. This is a temporary
              error.

       Arguments:

       name The client hostname, or unknown.

       addr The client address.

       helohost
              The hostname given with the HELO command.

       sender The sender address given with the MAIL FROM com-
              mand.

       recipient
              The recipient address given with the RCPT TO or
              VRFY command.

       size The message size given with the MAIL FROM command
              (zero if unknown).

BUGS
       Policies like these should not be hard-coded in C, but
       should be user-programmable instead.

SEE ALSO
       namadr_list(3) host access control
       domain_list(3) domain access control
       fsspace(3) free file system space

LICENSE
       The Secure Mailer license must be distributed with this
       software.

AUTHOR(S)
       Wietse Venema
       IBM T.J. Watson Research
       P.O. Box 704
       Yorktown Heights, NY 10598, USA

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]