OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: SOLUTION: Too many open files error when sending to large list
From: Michael Ju. Tokarev (mjttls.msk.ru)
Date: Sat Aug 26 2000 - 12:42:57 CDT

[]
> > In this case, it would help to increase the value duplicate_filter_limit
> > from the default (1000) to a much higher value. I know, I am just shooting
> > into the dark. It might however be worth a try :-)

This (increasing duplicate_filter_limit) solves the problem.
But it is only a workaround, as I can classify it.

Ok, let's explain.

Walter sent me his aliases. Nothing special/unusual, just some
number of groups of peoples, then groups of groups, etc, and after
that "all". Total users = 333. This is _very_ small list.
But note that almost every user has .forward file that looks like
  usersome.where.else, \user
i.e. mail should be duplicated to two addresses.

Thus, been_here called for:

  been_here: recipient 5 irmiamavis.tls.msk.ru: 0
  been_here: forward /home/irmi/.forward: 0
  been_here: indirect irmitls.msk.ru: 0
  been_here: recipient 9 irmiamavis.tls.msk.ru: 0
  been_here: forward /home/irmi/.forward: 1
  been_here_check: forward-done /home/irmi/.forward: 0
  been_here: mailbox irmi: 0
  been_here: forward-done /home/irmi/.forward: 0

The 'irmi' is a first address from list (I just created one
list with all of Walter's users), amavis.tls.msk.ru is my home
machine. This is just a grep for been_here. Voila, there are
7 (seven!) times for each "dotforwarded" recipient when
been_here called! So, for this list, duplicate table should
have 7*333 = 2331 entries, not including each sublist!
Ohh-ha! :(

Now it is time to serious walk throuth the source and find how
to deal with that. Note that on my home machine (it is nice one:
P-133 with 32 megs of memory and old IDE disk, linux inside)
delivering to that list tooks some significant time (when I
increased duplicate_filter_limit to 10000), and local process
was a bit huge for that memory, so I will not produce other
measurements. Look to source, Luke!

Ok, Walter's problem solved now.
But it is a bit wider -- been_here should not be so fat.

Probably I'll take another closer look to that -- been_here
should be rechecked (at least it's usage in .forward processing) --
but not now -- unfortunately -- just have no time to that.

Thanks, Lutz!

Regards,
 Michael.