OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Validating IDS Rulesets
From: Chuck Marchman (Charles.MarchmanPREDICTIVE.COM)
Date: Wed Oct 18 2000 - 13:22:28 CDT

On 10/17/00 10:08 PM Joshua Krage wrote:

>After experiencing a notable IDS vendor's lack of QA on a released
update,
>I started thinking... (which is generally dangerous for any bystanders.
:)

>How do you (as IDS vendor or IDS tester/architect) validate your chosen
>IDSs' rulesets? Do you simply assume your vendor is infallible? I do
>believe that this is the default answer for most organizations.

I can't speak for the vendors, but in my experiences with the NetRanger
sensors and the
Air Force's Automated Security Incident Measurement (ASIM) system, it was
more of a combination
of conducting vulnerability testing, incident response, subscribing to
sites such as
Security Focus, and a keen and analytical eye on the IDS monitors. If
anything new appeared in
the analysis of the sensor logs, or an alert was sent out to the security
populace, or in
investigating an incident provided information of a different type hack,
it was up to us to;

        - Isolate the problem
        - Verify the intrusion
        - Compile filters to identify/block such intrusions
        - Inform the commands/groups/bases of what was discovered and what
we were doing about it
        - And inform the vendor/contractors involved with updating the
programs

>Lets say a vendor releases a new improved version of their software. And
>lets also say that their data-entry clerk transposes a couple of digits
>in multiple rulesets and the signatures will no longer function as
>intended. How do you find out?

This could probably be answered as to how confident you are in your
vendor. As with anything
that is programmed the acronym of GIGO comes to mind. If the vendor has
been around for some
time, then it is safe to say that they are using good QA testing to ensure
the inadvertent
'data-entry clerk' mistake doesn't happen.

>I'm not talking about theoretical methods here... I'm talking about
>actual applied methods. For theoretical, we can say that we'll go
>download all the goodies from packetstorm and run them past the IDS. But
>when was the last time you ran, say, the Sendmail 'wiz' exploit past your
>sensors? :)

As a 'best practice', it is always a good idea to periodically run a test
against your
systems to verify that not only the IDS is functioning properly, but also
that your users
are aware of security threats and capable of recognizing/reporting
incidents that occur
on their systems.

Chuck Marchman
Information Security Consultant
Predictive Systems, Inc.
770-993-5010 ext. 766 (Voicemail)
678-923-4061 (Cell)
877-474-7375 (Pager)

charles.marchmanpredictive.com