From: Leandro Santi (lesantiuolsinectis.com.ar)
Date: Wed Jun 11 2003 - 13:19:32 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Wed, Jun 11, 2003, Victor.Duchovnimorganstanley.com wrote:
> On Wed, 11 Jun 2003, Leandro Santi wrote:
>
> > o timeout detection, so that the Postfix system doesn't hang while
> > querying or connecting to broken MySQL server (new).
> >
>
> One interesting consideration with timeouts for IPC based maps including
> LDAP, MySQL, ... is a possible correspondence between connection timeouts
> and maximum system throughput.
>
> Postfix only forks one service of each type ("cleanup", "smtpd", ...) at a
> time. The forked daemon initializes itself and reports (to the master
> daemon) that it is ready for service. Only when the last forked daemon is
> ready are new daemons spawned.

I'm not sure whether they report "ready for service". I think that single
threaded daemons like cleanup and smtpd will only send a MASTER_STAT_TAKEN
message as soon as a new connection is accept()ed (ie, the first message
is MASTER_STAT_TAKEN aka "server busy").

> Part of daemon initialization consists of opening the various maps the
> daemon is going to use. Each map may involve an additional TCP or unix
> domain socket connection. If these connections are slow (e.g. timeout and
> try another host), service initialization may take a long time.
>
> In the case of the "smtpd" and "cleanup" daemons each daemon processes 100
> (default value of the max_use configuration parameter) messages and exits.
> This means that for each hundred messages delivered master needs to spawn
> 100 cleanup daemons and 100 smtpd daemons.

Thats the worst case behavior (ie when those messages are sent in a very
short time interval).

> ...
> For LDAP there are two different solutions:
>
> - The optimal solution: set ldapsource_bind = no. This creates the
> LDAP map handle, but delays the actual LDAP connection until the first
> query is made. All daemons start quickly, but may experience latency
> during the first query. The throughput impact is lower by a factor equal
> to the process limit for the service.
>
> - The workaround solution: increase max_use, thereby reducing the demand
> for new processes.
>
> It may be useful (when supporting multiple MySQL backends) to implement
> similar delayed connection logic, so that the connection is completed in
> the lookup code, rather than in the map open code.

Thats how dict_mysql works: the connection is delayed until the first
lookup is requested.

> of chroot issues with unix domain sockets, so if the connection is not TCP
> based, it should not be delayed, presumably folks who run multiple
> backends connect over TCP.

Good catch. I didn't have this in mind. Maybe its better to split the
timeout option into "connect_timeout" and "query_timeout" and to enforce
"connect_timeout" for inet connections only (or, better yet, to have split
inet_connect_timeout and unix_connect_timeout parameters).

Thanks!
Leandro

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]