OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Pflogsumm Memory Consumption (was: Re: Interesting observations...)
From: Jim Seymour (jseymourLinxNet.com)
Date: Tue Aug 29 2000 - 17:46:59 CDT

Brad Knowles <blkskynet.be> wrote:
> Message-Id: <v04220808b5d024c60a86[195.238.1.121]>
>
[snip]
>
> Secondly, pflogsumm.pl can chew up a *huge* amount of RAM. ...
[snip]

I was wondering how long it would take before somebody noted this as an
issue. Tho I seem to recall somebody mentioning it once, early-on, in
private email.

>
> I recommend running it only late at night, ...
[snip]

Personally, I run it somewhat after midnight for daily runs and just
after the logs are rotated for weekly runs.

I *think* the person who mentioned the memory issue to me before solved
the problem by moving the logs to a separate machine and crunching them
there?

>
> It would be really nice if there were an option to run it with
> hashes tied to dbm format databases, ...
[snip]

That *probably* wouldn't be all that hard to do. Or at least once-
upon-a-time it wouldn't have been all that hard to do. I do know that
since I last considered that, I've greatly expanded what pflogsumm
reports. There are now hashes of hashes, arrays of hashes, hashes of
arrays of hashes, ... You get the idea.

As you might imagine: moving the hashes to disk will really whack
performance. But better slow performance than crashing due to out-of-
memory, I suppose :-).

No promises--except that I'll take it under advisement. I haven't
worked on pflogsumm in a while due to other things catching my
interest. And I've got two or three patches submitted for
consideration. I suppose it's about time I did some work on it. And
maybe do something about *documenting* that Date::Calc requirement I
keep getting questions about!

Regards,
Jim 

-- 
Jim Seymour                  | PGP Public Key available at:
jseymourLinxNet.com         | http://www.cam.ac.uk.pgp.net/pgpnet/wwwkeys.html
http://home.msen.com/~jimsun | http://www.trustcenter.de/cgi-bin/SearchCert.cgi