From: Hamish Marson (hamishtravellingkiwi.com)
Date: Mon Jun 02 2003 - 04:56:37 CDT

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Roy S. Rapoport wrote:

>On Wed, May 28, 2003 at 02:04:10PM +0200, Ralf Hildebrandt wrote:
>
>
>>>I have a program that's constantly tail -f'ing the postfix logfile
>>>(/var/adm/maillog in my case); since it's just looking at the last
>>>line[s], its I/O operations are remarkably limited.
>>>
>>>
>>Good point
>>
>>
>
>One of my design goals for the program I wrote was for me to be able to
>run it on the logfiles of Apache on, say, a top-20 website (which,
>obviously, wouldn't be running on just one web server). I've got
>informal access to such a website through a friend and we're planning
>to, soon, see what happens when you put my stuff under stress. I don't
>think it's going to lose any data, but I'm curious to see what happens
>to disk I/O as the database synchronizes to disk (though with built-in
>exponential backoff, it shouldn't be too bad).
>
>
I did that with some home grown stuff (Perl of course :) for webseal
(The reverse proxy part of Tivolis Policy Director (Now called tivoli
access manager)) and I use AFS for the .db file... The AFS volume with
the .db on it is kept locally for each of the web servers and a separate
machine reads that twice a minute to generate the RRD's. In that way the
CPU usage on the web server itself is kept to a minimum, and all the
configs are available locally to work out which part of webseal was hit
(i.e. which junction). The RRD updates can get quite disk intensive with
large RRD's and especially when you have several hundred of them so
having them on a separate machine also makes sense. Using AFS means I
have no firewall issues (I won't let NFS through the firewall because of
it's (Current) lack of security, AFS acl's are way better here and
having caching is an advantage as well.

[The biggest problem I had was with File::Tail thinking the logfile had
restarted & it would close & re-open the file from scratch all the time... ]

I do the same thing with postfix as well... Except with postfix I can
syslog the logfiles out & don't need the AFS for the .db's, they can all
be done completely remotely. (The script for reading the postfix logs is
based on a modified mailgraph).

--

I don't suffer from Insanity... | Linux User #16396
        I enjoy every minute of it... |
                                        |
http://www.travellingkiwi.com/ |

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]