OSEC

Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email hr@neohapsis.com
 
Subject: Re: IDS: IDS Comparison
From: Martin Roesch (roeschhiverworld.com)
Date: Sun Mar 05 2000 - 23:55:34 CST

FAQ: See http://www.ticm.com/kb/faq/idsfaq.html
IDS: See http://www-rnks.informatik.tu-cottbus.de/~sobirey/ids.html
HELP: Having problems... email questions to ids-owneruow.edu.au
NOTE: Remove this section from reply msgs otherwise the msg will bounce.
SPAM: DO NOT send unsolicted mail to this list.
USUBSCRIBE: email "unsubscribe ids" to majordomouow.edu.au
--------------------------------------------------------------------------- 

---
"Marcus J. Ranum" wrote:
> 
> John S Flowers <jflowershiverworld.com> writes:
> >If this is true (which it is), then we'd have to modify NFR to only
> 
> Oh, now I remember who you are! You're the guys who are working
> on that IDS appliance product of your own. No wonder you've got
> nothing good to say about anyone else! ;)  How's development coming?
> Got anything more substantial than vaporware, yet?
> 
> mjr.

I guess that question falls to me.  I'm the lead designer and developer
of Hiverworld's NIDS offering, called ARMOR.  Let me answer in
excruciating technical detail why we feel we're developing the next
generation in IDS technology.

First off, a little about myself.  Some of you may know me as the author
of the Snort lightweight IDS, which kind of stands by itself as a pretty
good implementation of an IDS, especially considering the fact that it's
completely free and has performance and capabilities that exceed many
commercial NIDS.  Additionally, I've developed or contributed to several
other ID technologies that are in use around the world at major
corporations and by our own federal government.  The reason I'm spending
time outlining all of this is because I want to make it clear that I
know ID technology, and prior to joining Hiverworld I had begun to
despair over the state of the technology that we were selling (or even
giving away for free) to people.

The current model of network intrusion detection has a number of
problems:  

1) It has a hard time keeping up with network bandwidth because of the
complex computational requirements of doing signature analysis, running
neural networks, simulating IP stack(s), etc.  

2) False alarming.  Today's IDS' have *real* trouble with false alarming
constantly without hazardously trimming your signature base to ignore
"normal" traffic.  

3) Reporting across all systems is a mess, with most systems throwing a
blizzard of irrelevent/cryptic messages at the user and expecting them
to qualify the alert as real or false based on direct forensic analysis
of the alarm.  Prioritization of alerts is *really* bad, with some
products not prioritizing at all (Dragon, NetRanger, et al), or
ineffectively (e.g. ISS with hi-med-lo).  The data that *is* reported
requires either a black belt in network traffic analysis (Dragon, NFR,
etc.) or a blind "faith-based" approach that the product got it right
(ISS, NetRanger, BlackICE, etc).

4) Rules load adversely effects performance: the more rules (or n-code)
you load, the slower (and in some cases, less stable) you get.  

5) Writing rules/signatures/n-code is tough for someone who isn't a
"packet head" like myself or most of the other people on this list.  

6) Analyzing output is much the same, requiring someone who has intimate
knowledge of IP networking to perform intelligent analysis of the data
produced by the sensors.  

7) Distributing sensors and consolidating their output is a major hassle
with most commercial NIDS and, unless I'm mistaken, there are no
products available that can determine what we at Hiverworld call "depth
of access" of an attack.  "Depth of access" is the concept determining
how far into a network an attack/attacker has gotten (or can get) based
on what sensors detected the attack in which order.  For example, that
attack against your intranet's web server doesn't matter if it doesn't
get past your firewall (or whatever).

8) Ptacek and Newsham.  Your IDS performs fragmentation reassembly,
great.  Does it perform it the same way as the target under attack?  If
it doesn't, someone may fragroute their way right past you.  Ditto for
TCP stream reassembly.

9) NIDS sensors can be the victim's of DoS attacks.  I can think of
several ways to DoS every commercial system out there, as I'm sure some
attackers can as well.  

10) Updates.  How often does your ruleset get updated?  How about the
actual sensor/console software?  What's the update policy of the vendor
it was purchased from?  What if that policy is "you can't update the
signatures yourself, and we release updates every 6-10 months" as it is
with some vendors?  Is that an effective IDS solution?  I'd have to say
no, but that's what some (major) vendors are selling, notably ISS, NAI,
and Cisco.

The list goes on.  So, how does ARMOR intend to answer these problems
(and how can I do it without making this message into something our PR
department would churn out)?  :)  

Let me start off by saying that ARMOR is not vaporware.  I think people
who know me (and some of them are on this list) from my Snort
development work are pretty aware that I don't oversell my work or tend
to hype things.  Those of you who don't know me will have to take my
word for it....  ARMOR is at this point a pre-beta product (about 3-5
weeks  from beta stage).  Our beta cycle is going to take place with
real-world testing, and we already have several large institutions lined
up who have decided that the IDS market as it stands is not ready for
prime time and that they want to start using our product at the earliest
opportunity.  So it isn't quite vaporware, but you can't see it yet. ;)

Point by point:

1) ARMOR (and all Hiverworld products) is sold as a combined hardware,
software, and subscription service solution.  When you buy ARMOR, you
subscribe a network or domain to a given sensor, and that is *all* that
particular sensor watches.  In that way, we limit the amount of
bandwidth that a sensor will look at to a set point, guaranteeing a set
level of performance.  The development target for a single appliance
(called a Hivermute) is a single 100Mbps broadcast domain.  I'm
confident that the system that we ship will meet this performance goal,
*but if it doesn't* Hiverworld isn't going to sell a solution that is
insufficient to the task.  We will appropriately divide the
traffic/sensor load to make sure that the proper level of analysis power
is available to every packet on the monitored domains.

2) False alarming is one of those big problems that no one has really
thought of a good solution for yet.  The accepted method for dealing
with false alarms is to deactivate the rule/sig/n-code that detects the
anomaly and call that traffic "normal" on a particular network.  This is
obviously not the right way to do this, because if an attack comes in
that *is* valid and matches that signature, the IDS will let it pass
right by.

ARMOR integrates the IDS and the vulnerability scanner (ARMS) to
mitigate false alarms.  ARMS and ARMOR are tightly integrated (so
tightly, in fact, that you can't run ARMOR without having an ARMS
subscription as well).  This allows us to develop a "vulnerability
picture" of the network and hand that picture off to the IDS.  The IDS
then analyzes traffic in terms of that vulnerability picture, only
alerting on attacks that are valid for the set of known vulnerabilities
and logging everything else of "interest".  The alerts themselves are
prioritized based on a proprietary scoring system that we use in our
products, so that you don't have the CIO's beeper going off at 3:00 AM
because someone just tried a PHF attack on your IIS webserver (for
example).

This is all done without any "clever signature writing" too....

3) All of Hiverworld's products use the same web based reporting
interface.  Reports can be viewed/printed as HTML or (Adobe Acrobat)
PDF's, or exported to an Excel spreadsheet.  All alerts that come out of
ARMOR are scored for relevance and prioritized accordingly.  Historical
reporting is available across ARMS and ARMOR, so you can see what
servers are most vulnerable on a given day (ARMS scans the network for
new vulnerabilities continuously) and which ones are under heaviest
attack, as well as seeing improvement (through historical reporting) in
a manner that is easily quantifiable (through the scoring system) and
and communicable (through our shiny PDF's) to management.  ARMOR also
provides solutions to alerts through the reporting system.  When ARMOR
generates an alert, you know where the attack came from, where it was
going, whether it was effective, how effective it was, how to fix the
problem, and where to get patches so that it doesn't happen again.

4) ARMOR uses an advanced "ontological system" to analyze traffic and
recognize attacks.  This system is extremely scalable, because it
provides a set overhead for every analysis which doesn't grow at
anything approaching the rate of addition of new rules/signatures.  It's
hard to explain without explaining ontologies, but the performance of
the ARMOR detection engine loaded with 10000 rules is about the same
(within 10%) of the performance of the engine with 100 rules.  For all
intents and purposes, rules load doesn't matter to the system as a
performance limiter.

5) ARMS and ARMOR use a proprietary language called "SEARL" to describe
vulnerabilities and attacks.  People who are familiar with the syntax of
SQL will be very comfortable writing SEARL rules, they are far less
technical than even the Snort rules that many of you here are familiar
with.  No coding required. 

At the same time, SEARL allows for a high level of granularity.  Just
because it's simple doesn't mean it's not flexible.  SEARL incorporates
an extremely high level of functionality in a concise and pointed
language.  Just because it's easy doesn't mean it's lightweight.  SEARL
is CASL without the hassle. ;)

John can answer questions about SEARL better than I (since he developed
the language) but it will suffice to say that everyone at the Hiverworld
offices (business/sales/technical) can write SEARL.  It's ass simple. :)

6) Returning to the reporting interface, ARMOR starts off with a
"faith-based" system because the false alarm rejection and dynamic
prioritization functionality allow it to send out alerts with a measure
of confidence that nothing else out there can approach.  If a user is
suspicious of the alert, the actual traffic which caused it can be
pulled up via the reporting interface and directly analyzed by a "packet
head".  This keeps it simple for the average user, yet the flexibility
to dig down into the traffic that caused the alert is there as well.

7) All Hiverworld products are designed to be distributed across capital
(class B+/Fortune 500) networks.  They talk to each other and to our HQ,
reporting, gathering rules and software updates, collating data, etc. 
At the IDS level, ARMOR is fully capable of determining the depth of
access of any attack, as well as the depth of access *required for an
attack to be successful*.  If that TTL is going to decrement to 1 before
the attack hits the target segment, you aren't going to see an alert
(it'll get logged under the "Daily activity report").

8) This is where the integration of the vulnerability scanner with the
IDS really pays off.  Given that we detect every device on the network
and can identify the operating system/IP stack that it's running (which
ARMS does for over 650 platform combinations) we can do something really
cool: accurately model the behavior of the IP defragmentation/TCP stream
reassembly of *any* target host on the network in a computationally
inexpensive manner.  We don't have to maintain multiple
decode->reassemble paths, or write IP stack simulators, we merely use
the knowledge that we've developed from the vulnerability scanner and
select the proper analysis scheme for a given host.  Using this
methodology, we completely defeat the IDS evasion schemes outlined in
the Ptacek and Newsham papers, as well as any fragrouter attacks. 
Integration of IDS + Scanner = robust risk management (which is what
Hiverworld's all about).  <-- blatant advertising!

9) How do you avoid DoS attacks as an IDS designer?  Know what's
possible and know your limitations.  ARMOR is designed with this concept
in mind, and is built to avoid attacks that would bring other IDS' to
their knees.  Every IDS has a "drop dead" point where state tables fill
up, memory runs out, etc.  Setting intelligent "flush points" where the
IDS "saves itself" in the face of a looming DoS condition is more of an
art than a science, but I'll say that ARMOR is designed with the same
design principle that I developed Snort under: thou shalt not die.

Without going into the gory details of good methods to DoS specific IDS
sensors (I don't think any of the vendors on this list would appreciate
that), suffice it to say that stability should be designed in from the
beginning and that a DoS-able ID sensor *will* be DoS'd when it really
counts.

10) Finally (hmm, is it a whitepaper yet? ;) we come to updating. 
Hiverworld's product line is designed to take advantage of the fact that
it's connected to a network.   Hiverworld's appliances are network aware
and self-updating.  What this means is that ARMS and ARMOR get
vulnerability/rule updates from Hiverworld HQ every day, and update
themselves with the latest version of which ever software system (ARMS
or ARMOR) they're running *the same day it's released*.  We have staff
specifically dedicated to keeping the databases of rules and
vulnerabilities updated every day and making those new pieces of SEARL
code available to *every* ARMS and ARMOR system at *every* customer,
automatically, every day.  

Contrast this with RealSecure (semi-yearly), Dragon (grab 'em yourself
and update manually), NFR (download the n-code and integrate it into the
system, which is supposedly "simple"), NetRanger (do they *ever* update
that thing?), BlackICE (download/update manually, although I thought I
heard that their system had some sort of auto-updater), etc.  When a new
version of the software for all of these products comes out, things can
get much more hairy.  New version of NFR come out?  Great, pop in the
CD, then reload all of your custom modules, and all the ones you got off
the net.....

Ok, that ought to about cover it.  Hopefully I've outlined why ARMOR
isn't vaporware, but a startlingly real system that was designed by
thinking through the problem of performing intrusion detection and
implemented in an innovative and far more capable manner than the
current crop of NIDS.  The model of ID that is out there right now
(including Snort) is a dying concept that isn't scalable, robust,
efficient, or even particularly effective.  We aim to change that.  

In a bit of irony, let me end with an (oft quoted) quote from Marcus in
last November's issue of the CSI Security Journal.  The reason that I
say it's ironic is because it outlines pretty clearly what we've
accomplished and yet is from a source that had no idea that when he
called it the "ultimate IDS", the system to do just that had already
been designed and was largely implemented:

"The ultimate IDS would not only identify an attack, it would: 1) assess
the target's vulnerability; 2) if the target is vulnerable, it would
notify the administrator; 3) if the vulnerability has a known fix, it
would include directions for applying the fix.  This would require huge,
detailed knowledge." 
 -Marcus Ranum, CSI: Computer Security Journal; Vol XIV #4 

This is outlines *exactly* what we have designed and implemented, and
it's going to be shipping to market late this spring.

Speaking of shipping, ARMS, the management console, SEARL and the
ontology have been shipping products since July '99, and have been in
use at some very large organizations for over two years now. 
Vaporware?  Not quite.

If anyone is intrigued by the concepts and technologies outlined in this
novella, there is going to be a far more comprehensive technology
whitepaper released by Hiverworld next month.  It will outline the basis
for and reasoning behind our integrated product strategy, as well as the
ontological systems that form the foundation of our product line.  If
you're interested in taking a look at it when it's ready, feel free to
drop me or John an e-mail.

-- 
Martin Roesch                      <roeschhiverworld.com>
Director of Forensic Systems     http://www.hiverworld.com
Hiverworld, Inc.               Enterprise Network Security
Network Forensics, Intrusion Detection and Risk Assessment