OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Relaying Performance
From: Greg Stark (gsstarkmit.edu)
Date: Wed Feb 09 2000 - 17:02:53 CST

wietseporcupine.org (Wietse Venema) writes:

> Greg Stark:
> > > > Our initial experiments seemed to indicate that it couldn't deliver faster
> > > > than about 5/s. Is this just a simple tuning issue or are the numbers I'm
> > > > talking about simply infeasible with postfix?
> > >
> > > Postfix has a weak spot here where a lot of inbound mail can swamp
> > > all the I/O resources so that delivery rates drop. You can quite
> > > likely get more mail through Postfix by reducing the number of SMTP
> > > server processes.
> >
> > But my feeder only opens a single connection and sends all the mail over that.
> > (Actually that's not strictly true, it reopens the connection after any error
> > and after 1000 successful deliveries.)
>
> Thanks for prividing a little more detail on your measurement.
>
> For a more precise reply from my end you will also have to specify
> how recipients are distributed over messages (one mail with many
> recipients or one-to-one mail) and how mail is distributed over
> destinations (all mail having the same destination on the same LAN,
> or all mail going to widely different remote locations).
>
> Its little details like these that make a huge difference.

Sure, this is for a web site sending mail to its registered users. So there
are very few identical e-mail addresses, but there are some domains that have
large numbers of users at them (Yahoo, AOL, WebTV, etc). Some of these domains
are very slow to respond or are intermittently available, and once available
ideally it's a good idea to batch as many messages over that smtp stream as
possible.

Each e-mail is individually generated, with a single recipient. The generating
script just runs through the database, generates the appropriate e-mail and
sends it down the smtp connection. The script keeps using the same SMTP pipe
until it gets to 1000 messages or receives an SMTP error, in which case it
closes and reopens the connection. Each message is to a single recipient.

My targets is to receive and deliver about 100 messages per second.

One possibility I've considered is implementing a special purpose smtp
deliverator which would handle hundreds of messages in a single process using
an old fashioned state machine and non-blocking i/o. I think that would take a
lot less resources and be a lot more efficient than having many identical
processes context switching. I'm not sure how well that fits into the postfix
model however. 

-- 
greg