FWIW, here's (again) my take at this topic.

On 5 August 2002, Michael Tokarev <mjttls.msk.ru> wrote:
> Having in mind various checks a modern MTA performs, I come to
> conclusion that keeping several different entities (e.g. recipients,
> client's IP/hostnames, helo arguments etc) in ONE single file makes
> life a bit easier and makes postfix more featureful.
>
> Take a look at how one may block mails with e.g. from=hotmail.com but
> that comes from non-hotmail server. There are other examples of such
> situations that in "vanilla" postfix requires multiple maps.
>
> What I'm thinking about is - how about adding an ability for a *map*
> to have composed KEY? I.e., for example:
>
> check_client_access hash:access:client,
> check_sender_access hash:access:sender,
> check_helo_access hash:access:helo,
> ...
>
> where access file contains:
>
> client:foo.bar.baz OK
> helo:hotmail.com 554 Fake helo
> sender:friendpublic.com 554 Die, spammer, die
> ...etc...
>
> I.e., before looking up a key in a file, prepend specified text before
> a key and lookup the whole resulting string.

    Minor point, mostly unrelated to what follows: the tags "client:",
"helo:" etc. that you prepend to the keys should be user-defined,
not fixed --- otherwise you would no longer be able to use the same
table in more than one place. The "scope" of the map would still be
identified uniquely by the (possibly implied) command preceding it:
check_client_access f.i. would be used to match client addresses, no
matter what the names of the key tags are.

> This way, we have an ability to have ONE map for all the things and
> construct unlimited number of smtpd_restriction_classes without
> requiriment to list (and USE) a huge number of databases.

[ More arguments for being able to merge several logical maps into fewer
"physical" database, deleted. ]

    IMO, this makes sense as a short-term optimization / hack, even
though it seems to be more or less specific to BD and DBM databases
(it doesn't really make sense to apply this logic to regexp or PCRE
maps, and even less to MySQL or LDAP). Despite the obvious limitations,
this is a first step towards addressing the problem of multi-attribute
queries. Wietse however extended the discussion to a more abstract
context:

On 5 August 2002, Wietse Venema <wietseporcupine.org> wrote:
[...]
> I have a dream, and that dream is to support queries and replies with
> multiple named attributes. As of November 2001, this is already how
> Postfix daemons communicate internally.
>
> The TCP (or any IPC) based map would be one the first map types to
> benefit from multi-attribute queries: one sends a list of attributes
> and receives multiple attributes as a result.
>
> Perhaps your idea could help to design multi-attribute queries for
> tables.

    In what follows I'll comment on this latter topic.

    IMO, in order to define an unified setup for maps that support
both multi-attribute queries and multi-attribute results, we have to
distinguish between several components:

(1) A mechanism to specify the back-end data source.
(2) A mechanism to specify the query, for a given data source.
(3) A mechanism to specify the "interesting" attributes of the results,
    for a given query.
(4) A mechanism to do various user-defined operations on the results.
(5) A mechanism to cache the queries.

    Of course, this goes beyond syntax:

    (1) should define the database type (hash, regexp, mysql etc.), and
whatever it takes to identify uniquely the data source for the given
database type (file name for DB, host / username / password / table
for MySQL, host / port for LDAP etc.). The syntax for this might be
(inspired by ODBC): a string of "flag" and "key=value" items, separated
by ":"; the existing syntax for MySQL and LDAP is unnecessarily verbose,
and should be abandoned. The mechanism would take such a string on
input, and yield a "database handle" on output.

    (2) would consist of an SQL query template for MySQL (something like
prepare() in Perl's DBI), a query filter for LDAP, a list of flags (case
sensitiveness etc.) for regexp and PCRE, and perhaps something like
the mechanism suggested above by Michael for DB and DBM. The syntax
can be a string with placeholders ("%s" as in DBI for MySQL, "%s" for
LDAP etc.). The mechanism would take a database handle and a string on
input, and yield a "query handle". Each query to the database would
take a query handle on input, and yield a "result handle" on output.

    (3) would consist of a template for munging results, like the "$N"
substitution for DB, DBM, regexp and PCRE, and accessing a given field
in the result of a MySQL query. The mechanism would act on result
handles, and yield the "interesting" part of the result in one of the
Postfix internal formats (more on this below). The exact syntax would
depend on the internal details of (2). For simplicity, (3) could be
merged into (2) (at least in a first draft implementation), but that
might also impose a performance penalty for multi-attribute results
(such as DNS queries).

    (4) would be merely syntactic sugar for the various lookups. Unlike
(1)-(3), a data-independent abstraction can be defined for its output,
and that makes it the natural candidate for interfacing (1)-(3) with the
rest of Postfix. It's also the only component that requires a "small
language", but exactly how far do we want to go on that path is not
really relevant to the big picture.

    Anyway, for the sake of creating a nice, fairy-tale dream, a
possible setup for it might look like this:

* Data types: the "language" would operate on variables (provided by (3)
    above) of one of the following data types:
    - boolean (true / false)
    - access type (OK / REJECT / DUNNO / SMTP reply string / etc.)
    - IP chunks
    - general strings
    - lists of the above objects (think "DNS queries")

* Operations: the "language" would apply the following operations to
    variables:
    - logical operations for boolean variables (including parenthesis)
    - concatenation, case folding and comparisons for strings
    - concatenation, iterators, and "member of" operators for lists
    - range matching for IP chunks
    - type promotions (IP chunks can get promoted to strings, strings
      and boolean can get promoted to access types)

* Work flow:
  - (3) above would provide results in the form of variables one of the
    above types
  - the user would provide description of "expressions" (strings of
    operations as above)
  - at the run time, the variables would get instantiated by queries,
    the expressions would be evaluated, and the result converted to the
    appropriate form.

    This would provide an unified interface to both smtpd restrictions,
DNS queries, and list-type maps (maps returning found / not found, not a
specific value).

    In order to minimize the impact this "language" might have on
performance, maps could be made persistent by a "mapping daemon". As
I pointed out a few times in the past, such a daemon might serve other
purposes as well --- among which (5) above would be one of the most
important.

    Regards,

    Liviu Daia

-- 
Dr. Liviu Daia               e-mail:   Liviu.Daiaimar.ro
Institute of Mathematics     web page: http://www.imar.ro/~daia
of the Romanian Academy      PGP key:  http://www.imar.ro/~daia/daia.asc
-
To unsubscribe, send mail to majordomopostfix.org with content
(not subject): unsubscribe postfix-users

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]