Neohapsis is currently accepting applications for employment. For more information, please visit our website www.neohapsis.com or email firstname.lastname@example.org
RE: Network hardware IPS
From: Dave Killion (Dkillionnetscreen.com)
Date: Mon Oct 06 2003 - 15:03:26 CDT
I hate Marketing spin as much as the next engineer, but with respect, I
disagree here entirely.
False Positive reduction has nothing to do with Detection Rate.
Reducing False Positives has everything to do with accuracy and context.
A hostile attack looks like "LeetAttack 1.0" - this is the actual, valid
attack string. But say this string is only hostile if sent as the User
Agent in an HTTP connection. Maybe it's a backdoor coded by the
webserver author, etc whatever.
IDS System A has a signature to detect this attack. They look for "1.0"
anywhere in an HTTP stream. Do they detect the attack? Yes. How many
false positives - that is, triggers on this signature that are not valid
attacks - you think they'll get? I'd say quite a bit. So, Detection =
100%, FP ~ 60-99%.
IDS System B also has a signature to detect this attack. They look for
"1.0", but they are advanced and have a context matching system that
allows them to look only at certain fields within the HTTP stream, one
of which is the User Agent. So they put "1.0" in the User Agent
context. Do they detect the attack? Yes. How's their false positive
rate? Lower than System A, I'd wager, but there's still some there. Do
they detect the attack any less than System A? No = both systems would
always detect every attack. Detection = 100%, FP ~ 30-50% - No decrease
in detection, but half the FP's.
IDS System C also has a signature to detect this attack. They have the
User Agent context as well, and they put "LeetAttack 1.0" as the
detection string. Do they detect the attack? You bet - 100%. Do they
have False Positives? No - unless someone was stupid enough to make a
valid web browser with that string as the User Agent. And you'd have to
wonder at their motivations if they did. Detection = 100%, FP = 0% - No
decrease in detection, but infinitely less FP's.
Obviously, the real world isn't as cut and dry as this example, but the
principles are the same - find something unique to the attack, go for
root cause, and get the context as specific as possible. You will
maximize detection while minimizing false positives.
I hope this information is helpful,
Senior Security Engineer
Security Group, NetScreen Technologies, Inc.
This email contains material that is confidential. The content of this
email is for the sole use of the intended recipient(s). Any review or
distribution by persons other than the intended recipient(s) without the
express permission of NetScreen Technologies, Inc. is strictly
prohibited. If you are not the intended recipient, please contact the
sender and delete/destroy all copies of this email and any related
attachments. NetScreen does not guarantee the accuracy or completeness
of third party materials or information.
From: Stefano Zanero [mailto:zaneroelet.polimi.it]
Sent: Friday, October 03, 2003 3:15 AM
Subject: Re: Network hardware IPS
> They claim a "92% reduction in false positives".
Sometimes this kind of bragging makes me wonder: these people actually
they are speaking to clueless folks ? Or the average audience is
inclined to hear "92%" and then run to buy a copy of whatever they are
False Positive rate and Detection Rate are inversely proportional, in
detection system. It is true for radars, it is true for medical
systems, it is true for anything. Check out the ROC, receiver operating
I cannot draw in an e-mail, but you can pick up a sheet of paper
Draw a graph: on x-axis, put FP rate. On y-axis, put DR.
Now think of a clueless, totally clueless, intrusion detection system.
generates totally random answers. You CAN obtain a 100% detection rate
it - if you accept a 100% false positive rate. The diagram on your chart
a line, bisecting the quadrant. If you want a 50% detection rate, you
to accept a 50% false positive rate, and so on.
Better intrusion detection systems would have a different graph, which
stands "above" the diagonal line. Draw it - it's just any curve you may
think of, which (hopefully !) is monotonically increasing, starting from
(0,0) and ending up in (100,100).
Do you notice something ? You _CAN_ reduce by any factor (92%, 95%,
99.9999%) the FP rate - but you WILL, always, without doubt, pay a price
detection rate terms. You can do it for the "idiot" IDS described above,
can do it for the best IDS you may think of: but it has always got a
The curve gives you a suggestion: the best "working point" is the one
the rate of FP increase vs. DR increase is at its top. Of course,
determining it in reality is not as simple as on our simple equation !
this model explains clearly (even clear enough for a salesperson maybe)
"decrease in false positive" or "increase in detection rate" mean
all, by themselves.
Captus Networks IPS 4000
Intrusion Prevention and Traffic Shaping Technology to:
- Instantly Stop DoS/DDoS Attacks, Worms & Port Scans
- Automatically Control P2P, IM and Spam Traffic
- Precisely Define and Implement Network Security & Performance
FREE Vulnerability Assessment Toolkit - WhitePapers - Live Demo
- application/x-pkcs7-signature attachment: smime.p7s