OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Validating IDS Rulesets
From: Brian Bartholomew (brian.bartholomewAVERSTAR.COM)
Date: Thu Oct 19 2000 - 11:15:01 CDT

Lemme guess...You're talking about Axent(Symantec...whatever, it sux!). My
pals and I have recently spent the metric equivalent of a buttload of money
on Axent and have had nothing but problems. The main problem was that
NetProwler runs as an application and not as a service. This means the
sensor must be logged in at all times for it to be "sniffing". That blows!
Anywho...after discovering this...along with the fact that there were a
$h1tload of false positives and even more false negatives...we decided to go
with the lesser of the two evils...ISS. Anyone else with this problem?? Im
sure there are!
-----Original Message-----
From: Focus on Intrusion Detection Systems
[mailto:FOCUS-IDSSECURITYFOCUS.COM]On Behalf Of Hervé Debar
Sent: Thursday, October 19, 2000 9:50 AM
To: FOCUS-IDSSECURITYFOCUS.COM
Subject: Re: Validating IDS Rulesets

Joshua Krage wrote:
>
> After experiencing a notable IDS vendor's lack of QA on a released update,
> I started thinking... (which is generally dangerous for any bystanders.
:)
>
> How do you (as IDS vendor or IDS tester/architect) validate your chosen
> IDSs' rulesets? Do you simply assume your vendor is infallible? I do
> believe that this is the default answer for most organizations.

I have been thinking along the same lines:

- essentially (there are a couple of exceptions), when you buy an IDS
  product you have to trust the vendor for the signatures. Experience
  has shown me that there are a number of bad signatures in some
  products leading to false alarms or non-detection. If you can't look
  at the signature, you're blind (did someone say security by obscurity
...)

- Even if the signatures were good, when you define a security policy,
  there is no information on the interaction between signatures and the
  IDS. If you want to test that you are actually getting the alarms for
  the attacks you want, I do not currently know of any other method than
  using another tool and actually trying the attack.

- I would imagine that activating certain signatures has a performance
  impact (relative to other signatures, relative to protocol decodes,
etc),
  this is usually not mentioned in products documentation but is
important
  for deployment.

- What configuration information is needed for each signature? What is
the
  consequence of having it wrong ?

My .2 euro :-)

Hervé 

--
Hervé Debar               | Tel. : +33.2.31.75.92.61
France Telecom R&D        | Fax  : +33.2.31.75.93.13
42 Rue des Coutures       | Email: herve.debarfrancetelecom.fr
14000 Caen - France       |