OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: large postfix installations?
From: Brad Knowles (blkskynet.be)
Date: Tue Apr 04 2000 - 08:13:56 CDT

At 11:37 PM -0500 2000/4/3, Michael Schwager wrote:

> Above that, and the machine starts hurting for resources (usually too many
> sendmail's pile up). I wonder which of your suggestions you think is the
> most apropos?

        They're listed in the "Things You Can Do" section, starting at
<http://www.shub-internet.org/brad/papers/sendmail-tuning/sld021.html>
in the order that I think you should look at them.

        However, note that moving queue files around with Paul Pomes'
program re-mqueue doesn't really buy you nearly as much as I used to
think it did, so I would avoid trying that. Also, I have since
learned that fork()/exec() overhead is not really an issue on any
modern systems around, so don't pay too much attention to that,
either.

> We do have named running on the same host. ...Is it really sharing
> sendmail's resources?

        They're both fighting for RAM, paging, etc..., and by having
separate nameservers running on each machine, you don't get to make
use of a larger centralized and shared pool.

> Our systems have lots of RAM, and CPU cycles to
> spare. And I seem to recall that, because of the DNS load, Eric Allman
> recommended in the Bat Book to put DNS on the same host as the server.

        I spoke with Eric and a number of other people at great length
while writing this paper. It is my belief that his recommendation is
for smaller sites, and is not appropriate for the really huge ones.

> - running a seperate sendmail on each machine, such that
> host pairs exchanging mail across our DMZ do so at a different
> port than 25.

        I don't see how this would help very much. You can run multiple
sendmail daemons on the same machine, each listening to a different
IP address, and using a different subdirectory under
/var/spool/mqueue, and this will keep your directory search time
down, etc....

> - getting Veritas' VxFS. (How much of a win is that? ...Especially
> with a SSD.)

        Veritas VxFS is a log-structured journaling extent-based
filesystem, and is typically a *huge* win performance-wise over
regular UFS or FFS. However, it is expensive as hell -- we pay
$100k/machine, so we make sure to use it only in those places where
we really, really need it.

> - Switching the filesystem to UFS logging... can you tell me
> more about that? (wrt Solaris especially, if you know...)

        UFS logging is for those systems that can't afford to run VxFS.
It's an implementation of log-structured writing on top of UFS, and
it does help to eliminate some of the synchronous meta-data
operations that would otherwise occur. However, if you can get it,
VxFS is much better at this.

        The advantage that UFS logging has is that it is shipped free
with every Solaris 7 server, as a standard part of the Solstice
DiskSuite tools.

        Of course, if you want really huge performance improvements, you
have to junk Solaris and go with *BSD, since they have the option of
using softupdates which is a much bigger performance win than using
even VxFS.

        Finally, I would be remiss if I didn't point out that running
postfix on the same sort of highly-tuned configuration would result
in considerably more speed than even sendmail could provide under the
same circumstances.

> Basically, I think what I'm asking is "If you had some Sun Ultra 450's
> with dual processors and 1 Gig RAM, and a caching SSD/RAID array, how
> would you fill in this blank: 'I'd make sure to _______________.'" I'd
> appreciate your insights.

        See above. If you really want absolute maximum throughput, you
junk them and replace them with FreeBSD boxes with as much RAM as you
can cram into them, software striped/mirrored /var/spool/mqueue
directories carefully tuned for minimum latency and with large enough
stripe sizes so as to virtually guarantee that all I/O for a single
file is handled by one particular drive mechanism and then you
achieve maximum parallelism in your filesystem writes, etc.... 

--
   These are my opinions -- not to be taken as official Skynet policy
======================================================================
Brad Knowles, <blkskynet.be>                || Belgacom Skynet SA/NV
Systems Architect, Mail/News/FTP/Proxy Admin || Rue Colonel Bourg, 124
Phone/Fax: +32-2-706.13.11/12.49             || B-1140 Brussels
http://www.skynet.be                         || Belgium