[Dailydave] WhiteHat Security report, or what use is SCA for web apps?

[Vitaly Osipov]

A while ago I've read an article absolutely not about security but
about how great it is to work in small friendly teams -
http://pragprog.com/magazines/2012-12/agile-in-the-small

It contains an awesome quote:

"...most best practices are just crutches for having a heterogeneous
skill mix in one’s team."

Please hold that quote in mind while I turn to the figures recently
released by WhiteHat Security
(http://www.slideshare.net/jeremiahgrossman/whitehat-security-website-statistics-full-report-2013).
They say that 39% of their clients use some sort of source code
analysis on their webapps. These customers experience (probably
meaning 'discover') more vulnerabilities, resolve them *slower* and
have a *worse* remediation rate.

Why is this? If you have ever been a customer to a SCA salesman then
you know. Their pitch goes like this:

"All you need to do is to run our magic tool with this 'best practice'
configuration and fix all results. The tool does not require a person
who understands app security to be involved. It's like a tester in a
box (plug: I wrote a post about "a fool with a tool" -
http://www.surrendercontrol.com/2013/03/scams-in-security-testing.html
a while ago). Even better, just use "OWASP top 20" (I call it "fake
silver bullet") configuration, this is what everyone else does."

Typical outcomes: the tool finds a large amount of rather unimportant
noise, rates the issues overly high just in case. Developers get tired
fixing these often nonsensical results. You'd be amazed how many
people run SCA (or web scanners) with the default config and then
forward results to developer teams, their own or third party's.
Eventually, the person running the magical scanner starts being
treated as the boy who cried wolf too often. And then Jeremiah
Grossman publishes his report ;)

Now, this is *not* a post against static analysis. Static analysis can
be an awesome tool for vulnerability research, especially for C/C++
code (although everyone seems to be 'fuzzing kernels' instead) and
maybe even in web apps. That is, if the tool you've got is capable of
being used as a research helper, not a checkbox filler.

Unfortunately the reaction of SCA salesmen to such a request (not of
all, but many) is usually "You want what? Write your own rules? And
drill down on results? And specify sanitisers and stuff? Crazy! Let me
find someone back at headquarters who knows what you're talking
about…"

Very often, a few simple scripts involving minimal lexing/parsing,
written for *your* specific web app (even without getting into ASTs,
solvers and data taint analysis) can be way more useful in finding old
and preventing new issues. Especially if they are run as commit hooks
in your git repo.

Back to the 'best practices' quote - if you are a software vendor and
you want to get real benefits from commercial SCA tools (I do not
count compliance among the benefits), do two things: hire someone with
a clue (about app sec, about development, about SCA) and get a tool
which has configurable rules.

Otherwise don't even bother. It will be about as effective as, and
much more expensive than, running an IDS or AV.


Regards,
Vitaly