OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: performance issues
From: Adam Levin (alevinaudible.com)
Date: Wed May 17 2000 - 14:21:11 CDT

On Wed, 17 May 2000, Brad Knowles wrote:
> You haven't shown us your postfix configuration. In particular,
> you haven't shown us the values of qmgr_message_active_limit,
> qmgr_site_hog_factor, hash_queue_depth, hash_queue_names, etc....
> Let us see the output of `postconf -n` for your system.

Is qmgr_site_hog_factor a new option? I don't see mention of it in the
samples for 19990906-pl9.

Here's my conf:
command_directory = /usr/local/postfix/sbin
daemon_directory = /usr/local/postfix/libexec
debug_peer_level = 2
default_destination_concurrency_limit = 10
default_privs = nobody
default_transport = smtp
header_checks = pcre:/etc/postfix/header_checks
local_destination_concurrency_limit = 2
mail_owner = postfix
mail_spool_directory = /var/mail
message_size_limit = 5242880
mydestination = /etc/postfix/mydestinations
mydomain = audible.com
myhostname = mailbox.audible.com
mynetworks = 207.159.120.0/24, 207.159.89.128/25, 192.215.168.96/27,
192.215.168.128/27, 192.215.168.160/30, 208.47.215.0/24,
208.45.131.174/32, 209.3.125.64/27, 24.11.77.81/32
myorigin = $mydomain
program_directory = /usr/local/postfix/
qmgr_message_active_limit = 10000
queue_directory = /var/spool/postfix
relay_domains = $mydestination, dbm:/etc/postfix/popclients
smtp_connect_timeout = 30
smtp_data_done_timeout = 30
smtp_data_init_timeout = 30
smtp_data_xfer_timeout = 30
smtp_helo_timeout = 30
smtp_mail_timeout = 30
smtp_quit_timeout = 30
smtp_rcpt_timeout = 30
smtpd_banner = $myhostname ESMTP $mail_name
smtpd_recipient_limit = 10000
transport_maps = dbm:/etc/postfix/transport

The timeouts were added, and I upped the active queue to 10000 (I haven't
seen any memory problems thus far, but I'm just trying to clear out the
queue). At this point, most of it is clear, but I'm not seeing the active
queue get up to near 10000 at this point -- it's stuck around 2600, and
the incoming queue is up over 2000.

> You really want to hash your incoming and deferred directories,
> if you ever expect that there is an outside chance that you might
> have more than ~1000 messages in your queue at any one time -- any
> more than that, and you start seeing significant effects of directory
> access times, etc.... A hash_queue_depth of two will let you handle
> ~1000 messages per directory times 16 * 16 directories, for a total
> of 256,000 messages.

Ok, done. I'll definitely want to do this on the machine handling the
huge outbound mailings, so this is good info anyway.

> If you boost your qmgr_message_active_limit beyond 1000, you may
> also want to hash the active directory, although sadly it will also
> have to share a level of hashed directories with all the other hashed
> directories, and this will probably slow it down somewhat (but not as
> much as not hashing it at all).

Well, I didn't really want to boost the limit to 10,000, but I was seeing
that 1000 wasn't enough -- the queue was getting stuck with 1000 messages
that were being delayed for some reason. When I upped it, there was
enough space in the queue for more immediate delivery of many more
messages.

> Also, you want to make sure that you're doing at least UFS
> Logging on the mqueue partition that postfix runs on. Ideally, you'd
> run Veritas VxFS or something even more intelligent with regards to
> the writing of synchronous meta-data, but even just simple UFS
> Logging should help somewhat.

It's /var/spool/postfix, and yes. Thanks,
-Adam

Adam Levin, Senior Unix Systems Administrator | http://www.audible.com/
Audible, Inc.
Wayne, NJ, 07470 There are two kinds of people in this world,
973-890-4070 x297 and I'm one of them.