OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: SOLUTION: Too many open files error when sending to large list
From: Lutz Jaenicke (Lutz.Jaenickeaet.TU-Cottbus.DE)
Date: Sat Aug 26 2000 - 13:05:32 CDT

On Sat, Aug 26, 2000 at 09:42:57PM +0400, Michael Ju. Tokarev wrote:
> []
> > > In this case, it would help to increase the value duplicate_filter_limit
> > > from the default (1000) to a much higher value. I know, I am just shooting
> > > into the dark. It might however be worth a try :-)
>
> This (increasing duplicate_filter_limit) solves the problem.
> But it is only a workaround, as I can classify it.

Yes, it is only a workaround (I also made some experiments in the last
minutes to verify, what Wietse already mentioned on that list).

Whenever an alias for which an owner- partner exists is used to distribute
mails, the mail is requeued and re-sent by qmgr. While resending, the list
is split, so that the problem of the overflowing table is avoided.

Say my aliases for this test are:
owner-duck: jaenicke
duck: dagobert, donald, daisy

Now, when I send an email to "duck", the mail is requeued and then send again,
with different "local" processes. Here you see a log example:
Aug 26 19:46:32 lutzpc postfix/pickup[795]: 1ED58F3A: uid=11019 from=<jaenicke>
Aug 26 19:46:32 lutzpc postfix/cleanup[802]: 1ED58F3A: message-id=<20000826194632.A798lutzpc.t-online.de>
Aug 26 19:46:32 lutzpc postfix/qmgr[796]: 1ED58F3A: from=<Lutz.Jaenickeaet.TU-Cottbus.DE>, size=488 (queue active)
Aug 26 19:46:32 lutzpc postfix/cleanup[802]: 53AB1F35: message-id=<20000826194632.A798lutzpc.t-online.de>
Aug 26 19:46:32 lutzpc postfix/qmgr[796]: 53AB1F35: from=<owner-ducklutzpc.at.home>, size=616 (queue active)
Aug 26 19:46:32 lutzpc postfix/local[804]: 1ED58F3A: to=<ducklutzpc.at.home>, relay=local, delay=0, status=sent (forwarded as 53AB1F35)
Aug 26 19:46:32 lutzpc postfix/local[804]: 53AB1F35: to=<dagobertlutzpc.at.home>, relay=local, delay=0, status=sent (mailbox)
Aug 26 19:46:32 lutzpc postfix/local[804]: warning: biff_notify: Connection refused
Aug 26 19:46:32 lutzpc postfix/local[804]: 53AB1F35: to=<donaldlutzpc.at.home>, relay=local, delay=0, status=sent (mailbox)
Aug 26 19:46:32 lutzpc postfix/local[805]: 53AB1F35: to=<daisylutzpc.at.home>, relay=local, delay=0, status=sent (mailbox)

(postfix tends to re-use processes, so we only see 804 and 805 here, but you
already see the split.)

Having this said and rethinking Walter's statements, he sends mail to "all"
for which an "owner-all" alias exists. "all" is however only distributed to
sub-lists, for which there is no "owner-" alias available and that are very
large themselves.
So Walter's clean solution would be to also add "owner-" aliases for the
sub-lists.

I rechecked this by experiment myself, since I did not find this owner- split
explicitely mentioned in the html/ and the conf/ directory (or in the "local"
source).
Wietse mentioned it on the list, so I could also just have believed it :-)

Walter's problem (too many open files) would not have appeared, if "local"
would have cried out loud when hitting the table limit instead of silently
ignoring additional entries. On the other hand, probably many people with
long aliases lists already exceeded the limit and never noted, they might
have been surprised, why local suddenly stops working.
Due to the possibility of .forward files with an uncontroled number of
entries, there cannot be a fixed limit that will make everybody happy.
So, it is a design decision, a compromise to be made.

Probably, the problem could be avoided by re-queuing every alias expansion,
but then we could not detect alias or .forward loops, which would state
a far more significant problem...

Best regards,
        Lutz 

-- 
Lutz Jaenicke                             Lutz.Jaenickeaet.TU-Cottbus.DE
BTU Cottbus               http://www.aet.TU-Cottbus.DE/personen/jaenicke/
Lehrstuhl Allgemeine Elektrotechnik                  Tel. +49 355 69-4129
Universitaetsplatz 3-4, D-03044 Cottbus              Fax. +49 355 69-4153