Agree totally on the white box vs black box thing.
Maybe a testing framework would be better served as
"With full knowledge and permission on the system",
"With no knowledge or permission of the system" and a
few levels in between (ie a basic non-priviledged
account).

That said since when has black box testing ever been a
time trial ? Thats just not the case. If it is then
you probably do whitebox testing in the same way. As
you say you can get a time based confidence like that
but if its as good as a bad tester for a week, you are
still onoy good from 10 secs of someone whose good.
Any testing framework should be irrelevant of testers
skills and down to science of what and how things get
tested.

I also think your example of cross site scripting is a
GOOD example of where black box testing is clearly
quicker and more efficient that trawling though code.
There are only so may ways you can do it, but stacks
of payloads. Its easy to test the ways to do it (I
agree Sleuth only does a few of them (try Unicoding
and URL encoding them etc) but harder to backtrack the
inout filter from a code analysis.

--- Nathan Catlow <nscqsf.demon.co.uk> wrote:
>
> All,
>
> I seem to be intrigued by the argument, as far as I
> can see these are mutually
> inclusive. I have never agreed with the
> white/black/grey/pink thing anyway,
> what is the purpose of a test in the web application
> sense?
>
> Test: To Identify security weaknesses in the coding
> of a web application.
>
> If you want to fix your car blindfolded, go for
> black box testing, has the
> customer got the money to wait for websleuth to
> iterate through all parameters
> trying to bruteforce a form? Probably not. Just as I
> wouldn't pay a
> blindfolded mechanic to feel his way round my car
> all day just to find the
> fuel pump or indeed pay two mechanics in different
> garages to identify the
> same problem (regardless of the colour of their head
> attire).
>
> Testers should identify a problem as efficiently as
> possible and consult the
> code if necessary to identify if this is a problem,
> consult with the
> developers to identify a fix and you're there, this
> way developers learn and
> the site becomes more secure.
>
> A 'black box' test is nothing more than a time-trial
> to see how many
> vulnerabilities can be found until the contract
> expires. You just have to see
> the cross site scripting plugin for webslueth for
> the reason why black box
> testing is inefficient, if you as a 'black box'
> tester never get to see the
> results back from the application unless you can put
> your 'white hat' on (god
> I hate that terminology) and monitor the database
> then you will not find that
> problem quickly.
>
> Some customers like this, probably for the following
> reasons:
>
> 1. It gives you a risk indicator, if a pentest team
> takes a week and doesn't
> break the site then you a protected against a
> similar skilled attack for more
> than a week.
> 2. If they are given the all clear, managers and
> developers are given a pat on
> the back and they can go down the pub instead of
> re-coding the obscure
> security problem all night.
>
> IMHO the Framework project should identify 'what
> needs to be done to
> efficiently identify security problems'. I don't
> mind 'black box' testing as
> long as the customer has been made fully aware of
> the issues, if somebody
> wants to pay me to sit around brute forcing
> parameters then cool, I personally
> think it's a bit annoying when you can't quite put
> your finger on a problem
> but given more time or the source.....
> [Un|F]ortunately real hackers don't
> have time constricting contracts.
>
> regards,
>
> Nathan.
> --
> Computer Crime Consultants Ltd
> www.ccc-ltd.com
>
>

__________________________________________________
Do You Yahoo!?
Send FREE video emails in Yahoo! Mail!
http://promo.yahoo.com/videomail/

• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]