On 30 Jul 2001, Yoann Vandoorselaere wrote:

> In practice, with a set of 1080 rules, Snort do not do less than
> 170 tests (minimal one by head), when Prelude can eliminate a packet
> in 2 or 3 tests. In normal usage, Snort do an average of 350 tests.
> Prelude have an average of 55 tests by packet with the same ruleset.
>
> More, CPU usage tests were done with a "normal" traffic on the UAB
> network. This one have a throughoutput between 300KBits/s and 9MBits/s.
>
> When the CPU usage were almost identical for the two product with
> low throughoutput, Snort was using more than 33% CPU more than Prelude
> when the throughoutput approached 1MBits/s and more than 300% for peeks
> of 9MBits/s.
>
> Snort reporting capability where put to the minimum for the tests.

I've been lurking lately while the heavy-weights hashed out some of the
OS/Linux/sniffing/snort debates, but I have a couple of notes on this
thread that people may, or may not, find useful:

- IDS testing. This is a huge topic in itself, but I continue to see all
sorts of testing results being posted - results that are complete and
utter crap, IMNSHO. I've quoted the above not to pick on Yoann (or the
dude that did the testing), but rather as an example of incomplete
information that IMHO you can't draw any useful conclusions from. I can
toss 40Mbps of traffic at an IDS that won't phase it, and another 40Mbps
that will knock it on its ass. Traffic rate? Traffic type? Packet size?
Session lengths? % of fragmentation? # of sessions? These will all make
a BIG difference.

(I rambled on about this in greater detail here, if anyone is interested:
http://archives.neohapsis.com/archives/sf/ids/2001-q1/0268.html)

Since most of the folks on this list are pretty switched on, I'd encourage
you to think twice about swallowing test results without looking hard at
them. For example, say all of your testing was based on traffic using TCP
port 0 *cough* Mier *cough* *cough*, some IDSes are going to react
differently but the traffic is still crap. (unless you use a lot of, er,
TCP port 0 traffic in your production environment). Supporting,
publishing, and quoting this type of garbage helps confuse people, and
IMHO, only degrades from the entire movement of third party validation.

- The great GIG debate. Ok, this one cracks me up. Given, Gbps
requirements are definitely coming (and I'm sure carriers need it now) and
I see this thread creep up all the time, no one ever seems to mention the
information overload and data management issues that come along with this.
Sure, if you deploy one Gbps IDS you are probably going to be swimming in
alerts. But if you are deploying multiple NIDS devices, crikey, even at
medium traffic rates there are so many alerts its going to be like trying
to swallow the Niagara falls. Some of the deployments we've done have
single NIDS devices pumping 20k+ alerts back to the console daily (before
tuning) - and that's moderate. I can't even imagine the alerts a
multi-sensor, carrier, Gbps-class NIDS deployment is going to fling at
you. The managing of this data, and the ability to sort through it
without loosing your mind is IMHO, a HUGE deal.

- Comparison points. Without context, arguing about the best NIDS is
about as useful as arguing about the best OS. In the review we
(Neohapsis) did for NWC (due out in about a week), we took the stance of a
mid-size company and reviewed products based on how they helped THAT
environment. We stayed away from doing the sensor-centric review as,
well, that's only part of the big IDS picture. I believe its important to
note that an org that is deploying a few devices locally to watch over a
class-C is going to have FAR different requirements than that of a
multinational org deploying hundreds of devices.

- Linux vs. BSD packet sniffing. Elliot (Turner) and Jeff Nathan covered
this one, but I'd like to stomp this one completely into the ground: most
of the "testing results" relating to this debate are ridiculously old.
If I get any spare cycles soon (doubtful, as in this market I'm being
forced to be Mr. Sales Guy) I'll try to run some tests on this, but before
posting, please check the dates of the tests. SStover is right - this
stuff changes quickly. ESPECIALLY open-source platforms.

- Not naming vendors. Ok, on a less serious note, is this another British
thing I'm missing, being a typical pig-headed American? :) I see posts
all the time asking "Hey, what do people think of <product X>," but then
when someone gets some juicy screw-up from a vendor, they strip the name
out. So why is it that its okay to ask about a product, but its not okay
to post on particular experiences? I don't want to promote vendor
bashing, but when people say "Yeah, and some IDSes (vendor removed to
protect the innocent) will steal your girlfriend," darn it, I want to know
which product that is. (might come in handy). Names damn it, names! :)

Being a verbose, bratty American,

-Greg

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]