OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: Gigabit IDS solutions
From: Jon Gary (jgaryCLICKTOSECURE.COM)
Date: Wed Nov 15 2000 - 17:22:35 CST

You make some very good points. One thing, which you alluded to, but did
not go into much detail about, is that whether or not bps or pps numbers are
a very worthwhile measure of an IDS's speed, the content of the packets sent
can greatly affect the speed at which the packets can be examined. For
example, an IDS that can monitor 100mb of normal traffic effectively would
almost certainly experience a performance hit if many of those packets were
obfuscated to some degree by any of several methods, including fragmentation
and similar well-known tricks. A malicious individual could mask his actual
attack by flooding the IDS's CPU while it tries to examine a massive amount
of decoy traffic, making it possible that the IDS would miss the actual
attack.

However, I don't personally believe that this is a huge fear. This is for a
few reasons. First, a large amount of strange traffic as described above
should set off an alarm in the IDS anyway, so even though you may not be
able to tell exactly what was attacked, an attack is detected. Second, the
attacker would be taking a huge gamble here. Even in the pathological case,
the IDS is still going to capture a good percentage of the data on the wire,
and the attacker is betting that it will miss the packets that contain his
attack. This is hardly the level of assurance against detection that this
sort of a sophisticated attacker would be looking for.

Despite the reasons that I don't see this sort of behavior as a huge threat,
I would be very interested to see numbers comparing the abilities of IDS's
to reconstruct complicated packet sequences in a variety of scenarios,
especially fragment reconstruction numbers on IDS's that reconstruct
fragments. You are very right, there is no standard way to test IDS
systems, and much of their behavior is limited by the underlying system on
which they exist.

Also, I have been pondering the Gigabit IDS problem for a bit here, and one
solution that I would be trying if I were an IDS vendor is a hierarchical
approach. We've seen systems that spread the load among multiple machines
for examination, but wouldn't a gradual filtering approach be more
effecient? Perhaps there is something that does this that I haven't heard
about, but I would envision a system in which there is one main IDS machine.
This machine does a cursory examination of all traffic, and does not do any
serious examination. All packets that don't look the least bit interesting
are thrown out, and the remaining packets are distributed to a few signature
checking systems. These systems are not inundated with little packets that
are not the least bit interesting, and the main system is using little
enough resources that it can keep up with the load. I'd be interested to
know if anyone has developed anything like that.

Jon Gary
http://www.clicktosecure.com/

-----Original Message-----
From: Focus on Intrusion Detection Systems
[mailto:FOCUS-IDSSECURITYFOCUS.COM]On Behalf Of Dragos Ruiu
Sent: Wednesday, November 15, 2000 9:52 AM
To: FOCUS-IDSSECURITYFOCUS.COM
Subject: Re: Gigabit IDS solutions

Whoa.... I really would rather stay out of the 3-4 way
quasi-mudslinging going on with this thread but I have
been biting my tongue for so long now it's starting to hurt :-).
(That said you'll all wonder at the stupidity of my next
statements. :-)

My current project is a vendor independent evaluation
of IDSes (like the ones the fine folks at neohapsis do - whom
I also hope to compare notes with). All of the gentlemen
you see lobbing posts at each other here are participating.
(And as a snort kinda guy I'll certainly be including it too, but
I'm not going to play favorites to anyone there, if anything
I'll likely be tougher on Snort because I know all the good
ways to blow it up... :-) I will probably have several papers,
test software releases and articles resulting from this work
as it is rather a massive project that has been underway
for about a month now. I'm gonna shut up about this following
this post until I have the real results.... because I don't want
to steal my own thunder, but I wanted to note:

The performance of an IDS can be judged in many ways,
numbers on raw data rates (fairly meaningless IMHO), packet
rates (though I would argue that the raw packet counts you
get by firing semi-stupid smart-bits packets are at best only
a rough measure of real world performance), signature coverage,
packet rates on varying real world traffic mixes, clarity and detail
of reporting, multi-probe integration and scalability of reporting
(I agree with Ron, tough to measure), frequency of updates
(impossible to judge without long-term mileage), customization,
applicability in different usage scenarios (home/small business,
small-midsize enterprise, large enterprise... all with different
needs), remote management and security of such management
architectures, price, effectiveness of coverage and many, many,
other factors. For my tests the final test lineup is not even
complete. But as far as gigabit goes, I feel the entire IDS industry
has a long ways to go... particularly because the main bottlenecks
there still lie in the underlying computing architecture as opposed
to the IDS software itself. I believe, and will outline in detail
shortly that pathological synthetic traffic sequences exist that the
current systems can't even deal with at a 100Mbps rate, nevermind
gigabit... Though I am not trying to discourage anyone from deploying
IDSes on Gigabit links, because IMHO, by distributing traffic to
multiple systems in various ways I _do_ feel it can be feasible to
monitor a Gig link for security with 3-4+ boxes dedicated to the
function through some front-end switching.

And even if your particulalr IDS can claim that it handles 320 Mbps
per second (which I believe is the fastest boast I've heard from
vendors until now), or 50K pps, there is the little niggling detail of
exactly how accurate you are at that rate and exactly how much work
and checking is done at that rate. The dillemma with IDS is that you
can always make it run faster by checking less....

Overall, since this is such a complicated measurement, that even the
vendors
can't agree amongst themselves as to how to run the "drag-race," they will
likely have a hard time convincing their users of these numbers and
resolving
the ambiguity in the performance numbers.... (that's something that
independent
evaluators _can_ address)

The bottom line for me after looking at this problem very closely is
that the maximum speed of an IDS is a damn hard thing to judge.
I took on a project where I have to do this... and my punt will be
to expose my testing methodology, publish the raw numbers and
let all the potential buyers of these IDS systems judge which factors
are most important to them in picking which system is right for them.
Because some of these IDSes are better suited to particular applications
and scenarios than others... and the knowledge level of the users and
the amount of dedicated security staff or hours available for monitoring
are also very important factors that have to be weighed in... Mostly
because some tools are better suited to novice users and others more
appropriate for experts or those with dedicated security staff.

In the near future, I will be providing some of my _opinions_ and my cut at
rankings and rate "drag-races" between products from these fine gentlemen
who are having heated data-sheet wars in this forum right now, but after
some initial looks at all their products, I must say that they each have
some
merit in their own specific areas. It's still a very tough decision for
those
purchasing such solutions, and requires some careful analysis of and
confidence in what you are trying to acheive in the first place..... As
far
as Gigabit IDS goes.... I also have to ask.... what are you hoping to
achieve
by monitoring your GigE backbone rather than the careful monitoring of the
choke-points in your network. (And if you _do_ have a Gigabit WAN
uplink....
gee, can I co-lo some servers in your farm ? ;-)

And now that I've gotten that off my chest I go go back to biting my tongue,
watching the mud-slinging, and crafting test sequences... :-)

cheers,
--dr