OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: CDB tables ?
From: Petru Paler (ppetruppetru.net)
Date: Sun Sep 10 2000 - 16:42:28 CDT

On Sun, Sep 10, 2000 at 08:23:14PM +0200, sthaugnethelp.no wrote:
> The size of the memory cache used can have a dramatic effect on speed
> when creating a hash database file. In FreeBSD, the pwd_mkdb command
> which is used to build the password database has option (-s) to specify
> the size of the memory cache - and it should be simple enough to extend
> postmap the same way.

It seems that it did have a dramatic effect. I increased the cache size to
50M and switched to btree tables:

$ time postmap btree:virtual
2.87user 0.39system 0:09.94elapsed 32%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (175major+2646minor)pagefaults 0swaps

Thanks!

> In any case, if your system uses 10 minutes for a 5M input file, something
> is seriously wrong. I tried it here with a 7M input file, and postmap took
> all of 11.5 seconds. A modified postmap with 10 MByte cache size instead
> of 1 Mbyte took all of 2 seconds. This is on a PIII-600, FreeBSD 4.1. The
> patch to use a 10 Mbyte cache is extremely quick and dirty, but included
> below.
>
> Steinar Haug, Nethelp consulting, sthaugnethelp.no
> ----------------------------------------------------------------------
> *** util/dict_db.c.orig Wed Mar 8 20:30:38 2000
> --- util/dict_db.c Sun Sep 10 20:08:22 2000
> ***************
> *** 77,83 ****
> char *path; /* pathname */
> } DICT_DB;
>
> ! #define DICT_DB_CACHE_SIZE (1024 * 1024)
> #define DICT_DB_NELM 4096
>
> /* dict_db_lookup - find database entry */
> --- 77,83 ----
> char *path; /* pathname */
> } DICT_DB;
>
> ! #define DICT_DB_CACHE_SIZE (10 * 1024 * 1024)
> #define DICT_DB_NELM 4096
>
> /* dict_db_lookup - find database entry */ 

--
Petru Paler, mailto:ppetruppetru.net
http://www.ppetru.net - ICQ: 41817235