The underlying question here is -- how do you find the most serious
holes for the least money?

There are certain problems (concurrency, Easter eggs, design flaws) that
are extremely difficult to find with penetration testing. Likewise,
there are many problems that are invisible when sifting through a
mountain of code.

I believe there is a strong argument that the most cost-effective
approach is to do BOTH. Doesn't that cost twice as much? No -- we've
found that reviews that include both penetration testing and code
review:

    - take about the same amount of time
    - provide a much better completeness argument
    - find more serious problems
    - provide better information to developers about how to fix it

The problem is building a team that is skilled in both security and web
app development. To be effective, they need to be able to read and
understand the code quickly. I wouldn't want a building inspector who
couldn't read the blueprints.

So, in my opinion, penetration testing alone is not going to provide the
best bang for your buck. Code review is way too easy and productive to
leave out of your balanced security breakfast.

--Jeff

Jeff Williams
Aspect Security, Inc.
www.aspectsecurity.com

----- Original Message -----
From: Kevin Spett
To: danidsec.com ; glyn.geoghegancorsaire.com ;
securityarchitecthush.com
Cc: webappsecsecurityfocus.com
Sent: Tuesday, December 03, 2002 5:27 PM
Subject: Re: WebAppSec Training Courses in UK

I've got a couple of quick reactions here... and this isn't a rebuttal
or a
disagreement with what Security Architect wrote, it's some contextual
information that should be considered along with it.

White box auditing is very, very, very expensive. Normal IT support
guys
often charge $50 or more an hour these days. A qualified security
source
code auditor can charge four times that. Plus expenses.

But that's not all. There's more than just source code. You've got to
check the web server for misconfiguration issues. And the web
application
server. And how about the database server?

Having a professional go through all of these steps is a remarkably
expensive procedure. Regardless of whether companies *should* budget
for
that kind of top-to-bottom thorough inspection, most (and by most I mean
nearly every last one of them) don't. So let's say you've got a $20k
budget
to make a large web application infastructure as secure as possible.

For that money, a skilled pen-test team can probably do more good than a
source code auditor. Two and a half work weeks (using $200/hour and
$20k
budget) isn't a whole lot to go through a large codebase, not to mention
securing multiple server configurations. An experienced pen-test team
with
good automated black box testing tools will probably be able to find
most of
the serious issues that most hackers would go after in your regular 40
hour
pen test. (Yes, if all they do is run ISS Scanner or Nessus and give
you a
report warning about parameter tampering, you get screwed.)

Of course, the best solution is to set up solid security policies and
requirements for coding, configuration, administration, user management,
etc. in the beginning, but most people don't have that luxury. So
you've
got to compromise. If you can pay for it, a complete security-conscious
overhaul in policy and implementation is a great idea, as is a complete
manual source code and configuration audit. But in a more practical
situation where you've already got something built, maybe even deployed,
and
all of a sudden a manager says 'Hey! Make sure it's secure!' , you may
be
able to get more bang for your buck with a quality pen-test team.

Kevin Spett
SPI Labs
http://www.spidynamics.com/

----- Original Message -----
From: <securityarchitecthush.com>
To: <danidsec.com>; <glyn.geoghegancorsaire.com>
Cc: <webappsecsecurityfocus.com>
Sent: Tuesday, December 03, 2002 2:08 PM
Subject: RE: WebAppSec Training Courses in UK

>
> With respect I think your description of security assessment training
is
woefully inadequate in todays world. Penetration testing is a snapshot
at
best and a time trial at worst. Having ran some teams for some well
known
consulting companies in the past I know all to well the business model
and
why its pushed so hard by them. Now working in corporate America I also
see
why we the clients (yeah we as in my company and others at like minded
user
groups who surprisingly do talk) are getting very frustrated with some
security consulting companies and training companies.
>
> <rant>
> Firstly there is little accountability. Its perceived as an art and
not a
science and therefore you really have little confidence that all of the
things that should have been tested were. Secondly with 78% of attacks
being
from insiders (see FBI reports) , looking at the hard crunchy outside is
of
little value. Too many companies reports read “High Vulnerability –
Parameter tampering”. After the sticker shock you read between the lines
and
find out you can change the page color and they have made an incredible
leap
of faith from that to saying you “may” be able to login in with another
users username. An indicator of parameter tampering in one place can
lead to
it in another. It’s the consulting fluff syndrome. You’ve all heard it
before I am sure. “These sessionID’s don’t look random”. Well test the
randomness if you have a math degree! If not look for the source of
randomness and if /urandom is used then call it out.
> </rant>
>
> Someone once used a great analogy. If you’re testing for cancer would
you
take someone’s temperature? Would you look at their eyeballs? Hell No!
Get
them on the cat scan machine. Even if the eyeballs are dilated and you
can
tell theyre ill, you still need to locate the problem (offending code)
to
treat it.
>
> One of the things I liked when I spoke to the OWASP testing people was
how
they are going to cover what I think should be included in a web
application
security testing methodology. In a structured meaningful test you need
to
firstly sit down and understand the security requirements. How can you
ever
say there is a problem unless you know the requirements and how it
should
be? Secondly you need to understand the application architecture. That’s
an
assessment in itself! How are people using JNDI, LDAP JMS <insert
architecture component of choice here>. People are finally realizing
that
XSS is easily cured with a proper architecture;-) You don’t fix it
tactically, you fix it strategically.
>
> Then there is a technical assessment which is where most people think
the
pen test comes in. But think of this. My requirements have shown that
sessions timeout after 20 mins and my architecture review shows I use
the
servlet container config (server.xml) to do it and the controller
servlet to
enforce it. I can sit there with a perl script and make a request every
21
mins to each url (dumb in my opinion) or I can parse web.xml and
server.xml
for the config. Ones a much more effective way to technically test the
requirements have been implemented IMHO. A pen test may have a place in
ensuring that stuffs functioning as it should be that’s where it belongs
again IMHO, flamesOff(security, architect).
>
> And then there’s a security source code review, a web application
security
management review (what happens when it goes down, who reviews logs,
what
policy exists to manage the security of the application).
>
> Web application security assessment is far more than a pen test. They
are
prevalent because consulting companies can pull the wool of clients eyes
with buzz words and hacker speak, not to mention the business model that
works well for the consulting companies. If you pay 40K for a hit and
run
that’s good business. But if you fix the first hole and have to pay $40K
for
the next then its not economical and the client will soon feel ripped
of.
>
> And why does this relate to training? Well people IMHO need to be
trained
that web application security assessment consists of many things not
just
how to own a web server in 20 mins or how to test for XSS from the
outside.
Assess strategically not tactically. Asses how security is baked into
the
development process and not just in a deployment scenario.
>
>
> On Tue, 03 Dec 2002 01:54:14 -0800 Glyn Geoghegan
<glyn.geoghegancorsaire.com> wrote:
> >You also need to determine whether the training you want is
> >1/ Architecting secure applications
> >2/ Building secure applications
> >3/ Application Security Assessments (pentesting)
> >
> >Each has a very different target audience, and its own set of
concerns.
> >
> >
> >Secure application architecture can involve broad concepts (e.g.
> >using
> >proper input validation, building a tiered structure of least
privilege)
> >or
> >specifics (e.g. secure .Net design).
> >
> >Building secure apps could start with pseudo code examples of
important
> >programming concepts and drill down into specific languages with
> >their pros
> >and cons.
> >
> >Application Security Assessments could take an application slant
> >on more
> >typical ethical hacking type courses.
> >
> >I believe Stake, ISS and Defcom provide Application courses in
> >the UK.
> >http://www.atstake.com/services/education/courses.html
> >
> >Glyn.
> >
> >> -----Original Message-----
> >> From: Dan Cuthbert [mailto:danidsec.com]
> >> Sent: 02 December 2002 21:57
> >> To: phuc4hushmail.com
> >> Cc: webappsecsecurityfocus.com
> >> Subject: Re: WebAppSec Training Courses in UK
> >>
> >>
> >> i think the problem is finding a trainer that understands the
> >
> >> problems associated with web applications and security. also
> >> the trainer that is providing the training would need to have
> >
> >> one helluvah understanding of security\building applications
> >> and the whole process
> >>
> >> its a lovely idea... hmmm yeah i can see a owasp opportunity here
> >>
> >>
> >>
> >> * phuc4hushmail.com (phuc4hushmail.com) wrote:
> >> >
> >> > I have unsuccessfully been looking for any decent WebAppSec
> >
> >> training
> >> > courses in the UK.
> >> >
> >> > It seems that courses are more on the networking side of things
> >or
> >> > when restricted to either specific technologies like J2EE
> >> or .Net but
> >> > I have yet to find a useful technology independent course
> >> that takes
> >> > in the wider picture as well as the grimey details.
> >> >
> >> > Any ideas?
> >> >
> >> > Maybe OWASP could start doing training courses?
> >> >
> >> >
> >> >
> >> >
> >> > Concerned about your privacy? Follow this link to get
> >> > FREE encrypted email: https://www.hushmail.com/?l=2
> >> >
> >> > Big $$$ to be made with the HushMail Affiliate Program:
> >> > https://www.hushmail.com/about.php?subloc=affiliate&l=427
> >>
> >>
> >
> >
> >----------------------------------------------------------------
> >------
> >CONFIDENTIALITY: This e-mail and any files transmitted with it
> >are
> >confidential and intended solely for the use of the recipient(s)
> >only.
> >Any review, retransmission, dissemination or other use of, or taking
> >any action in reliance upon this information by persons or entities
> >other than the intended recipient(s) is prohibited. If you have
> >received this e-mail in error please notify the sender immediately
> >and destroy the material whether stored on a computer or otherwise.
> >----------------------------------------------------------------
> >------
> >DISCLAIMER: Any views or opinions presented within this e-mail
> >are
> >solely those of the author and do not necessarily represent those
> >of Corsaire Limited, unless otherwise specifically stated.
> >----------------------------------------------------------------
> >------
> >
> >Corsaire Limited, 3 Tannery House, Tannery Lane, Send, Surrey, GU23
> >7EF
> >Telephone: +44(0)1483-226000 Email:infocorsaire.com
> >
> >
> >
>
>
>
> Concerned about your privacy? Follow this link to get
> FREE encrypted email: https://www.hushmail.com/?l=2
>
> Big $$$ to be made with the HushMail Affiliate Program:
> https://www.hushmail.com/about.php?subloc=affiliate&l=427
>

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]