From: Victor Duchovni (Victor.DuchovniMorganStanley.com)
Date: Tue Jun 06 2006 - 09:32:30 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Tue, Jun 06, 2006 at 06:30:20AM -0700, Arya, Manish Kumar wrote:

> I am migrating from qmail to postfix, I will have a
> number of entries (abt 2000) in
> /etc/postfix/mynetworks file. inclduing some xx.xx/16
> networks
> because /etc/postfix/mynetworks is a plain txt file
> and there is no indexing, so can postfix perform good

Use a cidr: table (really still a plain text file, but loaded into memory
more efficiently). If you put the most used (traffic volume) netblocks
at the top of the list, performance should be acceptable. The list is
held in memory, with lookups requiring a couple of memory accesses per
entry (the comparison operations are fast enough to not matter). With
memory bandwidth around 1GB/s, matching against your list should take
well under a millisecond (likely a few microseconds).

On a 400 MHz Ultrasparc with a 65536 entry CIDR map:

        perl -e '
        for ($i = 0; $i < 256; ++$i) {
        for ($j = 0; $j < 256; ++$j) { printf qq(10.%d.%d.0/24 OK\n), $i, $j } }
        ' > /tmp/big.cidr

512 lookups:
        perl -e '
        for ($i = 0; $i < 256; ++$i) {
        for ($j = 0; $j < 256; $j+=128) { printf qq(10.%d.%d.0\n), $i, $j } }
        ' > /tmp/ip
        time postmap -q - cidr:/tmp/big.cidr < /tmp/ip | wc -l
        512
        real 0m4.326s
        user 0m4.211s
        sys 0m0.033s

1024 lookups:
        for ($i = 0; $i < 256; ++$i) {
        for ($j = 0; $j < 256; $j+=64) { printf qq(10.%d.%d.0\n), $i, $j } }
        ' > /tmp/ip
        time postmap -q - cidr:/tmp/big.cidr < /tmp/ip | wc -l
        1024
        real 0m7.886s
        user 0m7.611s
        sys 0m0.133s

This is an extra 512 lookups against an average of ~32K table entries
in 3.5 seconds. This is a much larger table, and a slow CPU and yet
per-lookup cost is about 7ms. Take a 10 times faster CPU and a
20 times smaller table, and there is nothing to worry about.

> or is there some better way to store mynetworks?

Use a cidr: table. The in-memory addresses and masks are pre-parsed.

If you want to try PgSQL:

http://groups.google.com/group/list.postfix.users/browse_frm/thread/d83db77ca0fd21f

Other databases may have suitable non-SQL extensions for matching IP
addresses against containing networks, but with only 2000 entries
in mynetworks, lookups should be fast enough with a cidr: map.

--
        Viktor.

P.S. Morgan Stanley is looking for a New York City based, Senior Unix
     system/email administrator to architect and sustain the Unix email
     environment. If you are interested, please drop me a note.

Disclaimer: off-list followups get on-list replies or get ignored.
Please do not ignore the "Reply-To" header.

To unsubscribe from the postfix-users list, visit
http://www.postfix.org/lists.html or click the link below:
<mailto:majordomopostfix.org?body=unsubscribe%20postfix-users>

If my response solves your problem, the best way to thank me is to not
send an "it worked, thanks" follow-up. If you must respond, please put
"It worked, thanks" in the "Subject" so I can delete these quickly.

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]