OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: high capacity configuration nightmares
From: Pere Camps (pereulivatar.com)
Date: Tue Oct 24 2000 - 16:33:20 CDT


Hi!

        I'm quite new to postfix and I've found myself in a position of
having to setup a postfix box which has to handle the around 20
emails/second and I simply can not configure it correctly. I hope you can
help me.

        The specs:

        The mail server is a Compaq w/ dual Pentium III 733 Mhz, 512 MB
RAM, RAID 5 HD (spool directory is a 40 GB partition of the array).

        The OS is Debian Linux 2.2 which comes with postfix
19991231pl05-2. The kernel is compiled with Solar Designer's Openwall
extensions and only has exactly what it needs to work.

        The kernel has been modified so the maximum number of open files
is 32k (`echo 32768 > /proc/sys/fs/file-max`)

        The OS is running (apart from postfix), dnscache from D. J.
Bernstein (a fast caching DNS server which postfix uses and works OK),
openssh, and an snmp daemon from SystemEdge.

        Postfix details:

        The function of the postfix is to accept incoming emails (is
configured as a relay) from about 10 very-big-sun boxes. Any emails it
receives from them is then relayed to the internet where appropiate. No
other machines are allowed to connect to the mail server. Only 3 local
users and no mail is sent to the local accounts.

        As the site is not yet live (it will be very soon), in order to
test the site we're using a fake virtual domain (testmail.com) which
simply forwards all the email to a local account on the mail machine which
is in turn an alias to /dev/null.

        The non-default config is:

myhostname = mail-machine
myorigin = mail-machine.domain.dom
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
default_process_limit = 10000
mynetworks = 10.0.106.0/24, 127.0.0.0/8
smtpd_banner = $myhostname ESMTP - Hijackers will be prosecuted

        Syslog only logs mail.notice in order to avoid a bottleneck with
it (at least within testing).
        
        Now, the problem.

        Without any load at all, I can connect to mail server and the
emails get accepted and send in no time at all (you can barely notice it).

        However when the load starts to build up (6000 emails in 5
minutes), the following symtons appear:

        o Around 400 smtpd processes build up.
        o cannot fork() messages start to show up in interactive
shells.
        o there's a 15 second delay (aprox) when connecting to port
25. Then there's another delay for the smtp banner. No problem when I
enter the helo. However, for the mail from it takes around a minute or so.
        o Because postfix is slow in handling the mail from, data, rcpt
to, etc... around 700 concurrent connections build up.

        When I load becomes low, everything gets back to normal with no
problems at all.

        So, can anybody think of anything apart from putting a lower
process_limit to something like 300 which has just occurred to me?

        Thanks a lot!

ps: some free beer in the London (UK) area if you solve this! ;)

-- p.