From: Mark Martinec (Mark.Martinec+postfixijs.si)
Date: Wed Sep 22 2004 - 19:22:54 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Victor Duchovni,
> On Wed, Sep 22, 2004 at 02:00:53PM -0400, Vivek Khera wrote:
> > On Sep 22, 2004, at 11:37 AM, Victor Duchovni wrote:
> > > and optionally, but recommended:
> > > max_use=10
> > >Why is the default max_use=100 not recommended? This is a
> > >global setting, that increases fork/exec overhead for all services.
> > >I rather suspect this is poor advice...
> >
> > I think it is midguided advice. The amavis configs default to child
> > death after processing 10 connections. I think the way postfix holds
> > open the LMTP connection to amavis, it doesn't get to die soon enough,
> > or something like that. I don't see any problems with leaving out the
> > max_use setting in postfix, though.
> Indeed there are good reasons to not set max_use this low. If anyone
> knows the author of the document well, please encourage him/her to
> drop this advice.

This is from the amavisd-new documentation (evolving). Please read
and advise. It seems the ultimate solution is to provide max_use
parameter to be applicable to individual Postfix services.

  Mark

$max_servers, $max_requests and max_use

   Amavisd-new runs under process control of Net::Server. This is a
   pre-forked environment where $max_servers child processes are
   constantly kept alive and ready to accept new tasks (mail messages
   to be checked). Each amavisd child process is able to handle several
   tasks in a row, which helps to reduce startup (fork) costs. In case of
   SMTP or LMTP protocol, each SMTP session may consist of several SMTP
   transactions. Each SMTP/LMTP transaction is counted a one task,
   regardless of whether it came in from the same SMTP/LMTP client in
   a multi-transaction session, or from different SMTP/LMTP clients.

   A configuration variable $max_requests (default value 10) controls the
   approximate number of tasks each child process is willing to handle.
   After that the child process terminates and Net::Server provides a new
   child process to take its place.

   The exact value of $max_requests is not critical. There are two
   opposing needs, and some in-between value should be chosen.

   On the low side, the number should not be too small in order for the
   startup cost to be averaged out / diluted through the entire child
   lifetime. A value above 5 or 10 meets this goal in most amavisd-new
   configurations.

   On the high side, the value depends on the amavisd-new configuration.
   The amavisd daemon itself is conservative in its use of dynamic memory
   and does not slurp in a mail into memory, but keeps mail being
   processed and its components on files. Similarly, most of the called
   external virus scanners and decoders are rational in their use of
   memory. Unfortunately this is not true for Perl module
   Mail::SpamAssassin, which expect to have an entire mail in memory in
   order to be able to run its large set of rules on it in reasonable
   time. This is a design decision made by SpamAssassin creators, and
   we have to live with it.

   When amavisd-new is not configured to use SpamAssassin, the value
   of $max_requests can be quite high without any known or expected
   problems. For general sanity reasons, an upper limit could be a
   100 for example, although anything above 20 or so would not bring
   measurable benefit to the maximum sustained mail throughput.

   When amavisd-new _is_ configured to use SpamAssassin however, the
   slurping of entire mail in memory may have implications, depending on
   the maximum mail size allowed at the MTA (e.g. Postfix setting for
   message_size_limit). Even though the allocated memory is reclaimed by
   Perl after mail processing, and is reused for subsequent processing,
   the process virtual memory footprint never shrinks, it can only expand
   as needed.

   With a default value of message_size_limit near 10 MB this is not a
   serious problem, and $max_requests can be fairly large, although since
   the additional performance gain is negligible for values beyond 20 or
   so, there is no good reason to choose much larger value than that.

   Some sites however chose not to limit mail size, or increase the
   maximum mail size limit substantially. If a large mail arrives on
   such site, the virtual memory of amavisd child processes is extended
   to accommodate the message. For the rest of its lifetime the child
   process that processed such large mail stays at its high virtual
   memory size. If this happens frequently, host resources may become
   scarce. Limiting the number of tasks each child is supposed to
   process is very much desirable on such systems.

   The default value of 10 for $max_servers was chosen as a reasonable
   compromise between averaging-out the startup costs and not wasting too
   much resources on hosts with high message size limit and SpamAssassin
   enabled.

   In the setup with Postfix where its LMTP client is chosen to feed
   amavisd-new, this client tries to keep LMTP session open and submit
   several mail messages in multiple transactions. With recent Postfix
   versions its SMTP client is capable and willing of using
   multiple-transaction sessions as well, although it seems to be less
   persistent than the LMTP client.

   According to SMTP and LMTP protocol specifications, dropping the
   session on the server side is considered rude and should be used only
   as a last resort. In order to respect the $max_requests setting (which
   is not strictly enforced by amavisd, and is considered an advisory
   value), the client side should be configured with a comparable limit.
   In case of Postfix, such a limit is max_use, which as far as I know
   can only be applied globally. A recommended value of max_use is the
   same or similar in value as $max_requests, 10 by default.
   Unfortunately max_use affects other Postfix services as well, not just
   the one feeding amavisd, so depending on your configuration, choose
   a sensible compromise.

(from: http://www.ijs.si/software/amavisd/amavisd-new-docs.html )

Mark

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]