OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: SCSI, U2W in particular?, and poor performance...
From: Wietse Venema (wietseporcupine.org)
Date: Sun Mar 05 2000 - 12:56:29 CST

Justin Robertson:
>
> Well, I'm glad to see that you've all assumed I'm a retard. Seeing as both
> boxes have an identical install image, what you're suggesting still makes
> no sense. It's not that one box has had certain features disabled or
> enabled, and I do love the approach you take to your users questions.
> Prehaps you'd like to tell me why sendmail outperformed postfix on the
> SCSI machine then? hmm?

We're all pleading ignorance, Judge :-)

The Linux ext2fs file system invalidates some assumptions that are
true with UNIX file systems. With Linux, directory updates are
asynchronous, so even when a file's content is fsync(2)-ed to disk,
there is no guarantee about the status of the file's directory
entry (*). For example, open()-ing a new file returns before the
file system has been updated. Until very recently, when a file was
renamed, the directory updates could be done such that the old file
name was removed before the new file name was put in place. If the
system crashed in the middle of rename, it was possible lose files.

UNIX file systems, esp those based on FFS, take great pains to
ensure that directories are updated in a safe manner. For example,
when a file is renamed, the new name is put in place before the
old name is removed. And open()-ing a new file returns *after* the
directory entry is put in place.

Because of this difference, Postfix on Linux by default enforces
synchronous writes on the queue directory tree, so that Postfix
will not lose mail should the machine crash. This makes Postfix
on Linux slower than Postfix on comparable UNIX systems. That's
too bad, but simply I can't recommend configurations that lose mail
in a system crash.

It is possible that Sendmail uses the default asynchronous extfs
directory updates, which trades speed for loss in reliability. If
the machine crashes in the middle of receiving a burst of mail,
then you can expect to find some mail in the lost+found directory.

The measurement with the low-cost IDE disk was flawed. 1024 Messages
in 2 seconds means that hardly any data was written to disk. That
figures with Postfix or whatever on an async ext2fs file system.
The measurement gives no useful data about mail system performance.
If it acceptable to lose mail I can speed up Postfix tremendously.

        Wietse

(*) for that, Linus recommends that the application open()s the
directory and fsync()s it. Right. When pigs fly. Fortunately we
still have a choice of operating systems.

> Justin Robertson
> <zululinux.com>
>
> On Sun, 5 Mar 2000, Wietse Venema wrote:
>
> > I understand that I was assuming too much knowledge on the poster's
> > side when I explained why today's IDE disks can't deliver the
> > performance necessary for Postfix to do 500 email messages a second.
> >
> > I was assuming that someone who operates advanced hardware such as
> > mentioned in the subject has a general understanding of operating
> > systems, of file systems and of network protocols.
> >
> > That's not right. I apologize.
> >
> > Wietse
> >
>
>
>