OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Validating IDS Rulesets
From: Keith Pachulski (Keith.PachulskiCORP.PTD.NET)
Date: Wed Oct 18 2000 - 11:04:39 CDT

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

After you install a firewall do you test that firewall? I would hope
you do..

As with every other security product, the IDS products are tested for
workability.

It also comes down to internal security procedure. No one should have
access to security devices except the security persons designated to
manage those devices. Access to those devices should be restricted
and audited on a daily basis. What is your internal security policy
for non-security individuals accessing security devices? Do you have
a policy? Is the policy enforced?

Every new IDS signature is tested and retested in our network as is
every security device we implement into our network. Not only are
those devices tested before and after installation, but they are
tested monthly to confirm they are still operating as expected.

Older IDS signatures remain in the database, no signature is cycled
out or removed unless the signature has been proven to be faulty.
Those faulty signatures are either reworked to correct the fault or
removed after complete documentation as to why they were removed from
the IDS database. Signatures which have been reworked also require
extended documentation explaining why they were reworked and if the
rework of the signature has been successful.

This is not a complete answer, just touching the surface..

- -Keith

- -----Original Message-----
From: Joshua Krage [mailto:jkrageBUSER.NET]
Sent: Tuesday, October 17, 2000 11:09 PM
To: FOCUS-IDSSECURITYFOCUS.COM
Subject: Validating IDS Rulesets

After experiencing a notable IDS vendor's lack of QA on a released
update,
I started thinking... (which is generally dangerous for any
bystanders. :)

How do you (as IDS vendor or IDS tester/architect) validate your
chosen
IDSs' rulesets? Do you simply assume your vendor is infallible? I
do
believe that this is the default answer for most organizations.

Lets say a vendor releases a new improved version of their software.
And
lets also say that their data-entry clerk transposes a couple of
digits
in multiple rulesets and the signatures will no longer function as
intended. How do you find out?

I'm not talking about theoretical methods here... I'm talking about
actual applied methods. For theoretical, we can say that we'll go
download all the goodies from packetstorm and run them past the IDS.
But
when was the last time you ran, say, the Sendmail 'wiz' exploit past
your
sensors? :)

Has anyone put together a few hundred megabytes of network traces
that
can be tcpreplay'd past your IDS sensors and trigger every known
alert?
Is this even worth it? :)

To kick a discussion off, my methods of the moment include:
  - lots of research (like this list)
  - review of the software (esp. signature files and such)
  - spot testing (run exploits past the sensor)
  - comparing reported events with another system (other IDS,
tcpdump)

Thoughts?

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 6.5.8 for non-commercial use <http://www.pgp.com>

iQA/AwUBOe3JZeGTq6qVSXTQEQLC2ACfUPzVVrk6duPTcc4+MUXABOuNYU0An00z
AY2NEDV74CbUiK3NB9HPDvWI
=Jnri
-----END PGP SIGNATURE-----