On Wed, 17 Jan 2001, Mark Teicher wrote:

> I think you are answering a different question. Duke inquired about IDS
> certification (i.e. ICSA Firewall Certification program) for vendors. As of
> this moment, I think there are talks about ICSA will be conducting the IDS
> certification process. I don't know if any vendors have ponied up the dough
> to ICSA in order to start the certification process for their particular
> product. Since the products change drastically from release to release, it
> is going to be almost impossible for ICSA to certify an IDS application. In
> the last quarter alone, ISS issued almost 4 updates to the Real Secure 5.0
> version

They are different questions, absolutely. Possibly taking this thread for
a nose-dive, it has been my experience that in the hard-core security
circles the ICSA Firewall cert doesn't hold a lot of weight. While this
is a huge can of worms to even think about opening (*preparing to dodge
flying shoes*), I've got to ask: what, exactly, are they certifying?

From the ICSA labs web page, it is stated:
(http://www.icsalabs.com/index.shtml)

"The goal of ICSA Certification is to improve significantly commercial
computer trust and security by improving the implementation, sales and use
of appropriate security products, services, policies, techniques and
procedures....Recognizing that perfect computer security is unattainable,
ICSA's Certification Program provides assurance to the user community that
ICSA Certified products reduce security risks consistent with a set of
publicly vetted and industry-accepted criteria. "

Ok, that's great, but how many FW vulnerabilities came out in 1999? In
2000? And how many of those firewalls were "ICSA Certified?" My point
isn't that the cert doesn't do any good - I'm sure it does, but let's be
honest here, what does that cert REALLY get you? A tested product? A
more secure product? The fact that a vendor paid six-figures for a
branded "stamp of approval?"

Don't get me wrong, I think there is a huge need for 3rd-party
involvement, and dare I say it, "certification." IMHO, there are some
fronts to this that are REALLY important on. For example, I've heard that
the ICSA team is working on IPSEC *compliance* and interoperability
testing. Ok, that's huge, as anyone who has worked with multi-vendor VPN
deployments knows that the VPN space is a mess on that front.

The problem is, I question whether or not people are being mislead, and
how much good some of these certifications (like the firewall one) really
do. Ultimately, does this type of "branding" help provide for a false
sense of security?

One other point and then I'll shutup (and please don't read into this as
some sort of twisted promotion, I use this example solely to prove the
point that there is a problem):

Months ago a large client of ours approached us to "review" (privately) a
firewall unit they were thinking of deploying in bulk (read: hundreds).
After digging into this thing, one of our guys (Jeff) found a myriad of
older code-bases, an IP stack vulnerable to known DoS attacks, and the my
personal favorite, a warez copy of Macromedia's Director residing on the
filesystem (complete with serialz).

We of course contacted this vendor, and I'm hoping they got the warez copy
of Director out of their images, but this is the kind of crap that
continues to go on these days. IMHO, it's disgusting. And for those that
are curious, yes, that vendor is on that ICSA firewall list. (Although I
don't know if ICSA got a warez copy of Director with their unit *grin*)

Ah...but I digress.

Anyway, if the community did want IDS certification, my question would be,
what would that certification mean? What would it cover? And who would
define the criteria?

Those I think, are the bigger issues.

I'll shutup now,

-Greg

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]