From: ram (ramnetcore.co.in)
Date: Tue Dec 25 2007 - 23:21:11 CST

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Tue, 2007-12-25 at 09:51 -0500, Wietse Venema wrote:
> Ramprasad:
> > Wietse Venema wrote:
> > > ram:
> > >
> > >> I have a postfix2.3.4 with a custom milter ( basically doing personal
> > >> whitelisting/blacklisting )
> > >>
> > >> I occassionally see errors like these ( very infrequently though )
> > >>
> > >> ..
> > >> Dec 24 18:44:20 mta1 postfix/smtpd[26461]: connect from
> > >> acbt21.neoplus.adsl.tpnet.pl[83.9.117.21]
> > >> Dec 24 18:44:20 mta1 postfix/smtpd[18616]: connect from
> > >> unknown[78.174.27.44]
> > >> Dec 24 18:44:20 mta1 postfix/smtpd[18581]: warning: milter
> > >> unix:/var/run/f1.sock: can't read SMFIC_OPTNEG reply packet header:
> > >> Connection reset by peer
> > >>
> > >
> > > Typically, a Milter application is ONE multi-threaded program that
> > > handles all the connections from all Postfix processes.
> > >
> > > That's one smtpd_milters connection from an smtpd process for every
> > > SMTP connection to your server.
> > >
> > > In addition, the Milter gets one non_smtpd_milters connection from
> > > a cleanup process for each local submission and for each bounce
> > > message that Postfix generates (forwarded mail is not filtered
> > > again).
> > >
> > > This can potentially be a very large number of connections.
> > >
> > > If this number of connections gets too large the Milter will run
> > > into trouble and drop connections. It is not nice that it doesn't
> > > log a warning, though.
> > >
> > > What is the Milter program in question, and what are your smtpd
> > > process limits, and what are the process limits for local and smtp
> > > deliveries?
> > >
> > > Wietse
> > >
> > yes, my milter runs on a very high traffix server. receiving more than
> > 120k - 150k connections per hour ( the server is a dual Xeon 4GB RAM )
> >
> > I have set the smtpd process limit to 500 in my master.cf and typically
> > see 250 to 300 processes running. The milter is quiet a simple one. It
> > reads a set of personal whitelists and blacklists into memory and using
> > bsearch at run time determines wether to reject a mail or accept a mail
> > according to the recipients preferences.
> > If no match found accept mail, if matches whitelist insert a header for
> > SA , if matches blacklist return error code to drop connection I can
> > share the code if that is necessary .
>
> By default, libmilter uses select() instead of poll(). The SENDMAIL
> libmilter LOGS NO WARNING and closes the connection to the MTA when
> the process has more than FD_SETSIZE file descriptors. On Linux,
> FD_SETSIZE is fixed at 1024 and can't be changed at compile time.
>
> With an smtpd process limit of 500 plus some other activity (bounces
> while delivering single-recipient mailing list messages) you may
> get close enough to this limit that it matters.
>
> Fortunately, libmilter can be built to use poll() instead of select()
> to avoid this limitation:
>
> +----------------------+
> | CONFIGURATION MACROS |
> +----------------------+
>
> Libmilter uses a set of C preprocessor macros to specify platform
> specific features of the C compiler and standard C libraries.
>
> SM_CONF_POLL
> Set to 1 if poll(2) should be used instead of select(2).
>
> Perhaps rebuilding libmilter is an option.
>
> Wietse

Thanks for the reply ( To be honest .. I will have to quiet some reading
for now to be able to understand and implement that :-) )

Besides what is the drawback of using poll() instead of select(). There
surely nust be some else libmilter would have already used poll() IMHO.

Is poll() recommended for a high traffic server.

So what I assume is , my milter has hit a software bottleneck. I cannot
increase its performance by simply having better hardware. Am I right in
assuming that ?

Thanks
Ram

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]