[Dailydave] CVSS is the worst compression algorithm ever

Thu Jan 10 18:28:21 UTC 2019

CVSS' greatest attribute is that it lets assessors fudge the numbers to
make assessors happy and gives risk people some kind of industry standard
document/organization attesting to the risk. Everyone wins.

It's only when people start asking (valid) questions where things fall
apart.

There are two other scoring systems I haven't seen referenced much: NIST's
CMSS and Mitre's CWSS. May be worth checking out if you're some kind of
risk nerd.

Regards,
Eric

On Thu, Jan 10, 2019, 12:50 PM Dave Aitel <dave.aitel at gmail.com wrote:

> Ok, so half of FIRST or the CVSS team is angry at me for my tweets about
> the examples on FIRST.com being wrong. But here, in general, is a common
> issue I see with CVSS scores in our deliverables, that I try to correct,
> although admittedly I'm not an expert at CVSS itself.
>
> The issue is simplified to: If an SQLi exists, how does that rank for the
> CVSS Confidentiality, Integrity, and Availability sections. Like, here's an
> example: https://nvd.nist.gov/vuln/detail/CVE-2013-0375 . As you can see
> there is "low" impact on confidentiality and integrity, and NO impact on
> availability.
>
> But how can that be correct? The questions you start to ask as you make
> those decisions are: What user context am I running in on the SQL Server
> (i.e. sa?) and what does that user have access to in terms of tables, and
> what importance is that information? Also what clause is the injection
> running in the SQL statement itself? Does this database support sub-queries
> such that I can alter information? Are there functions that do things with
> side effects I can call? Answering these questions is complex and possibly
> dependent on configuration and the CVSS way is to assume the worst, which
> cannot POSSIBLY BE "LOW".
>
> And at a minimum, you would expect possible Availability issues to be
> high, because anyone who's played with an SQL injection tool knows that
> even doing SLEEP statements has a tendency to take down web applications.
> Imagine if your goal was to take down a web application with an SQLi...? I
> think Microsoft Research did a whole paper on doing SQL Injection timing
> attacks just with random function calls? I can't find it now though.
>
> Ok, so that brings us to XSS and "HTTPOnly" and the FIRST.org assessment:
> https://www.first.org/cvss/examples#1-phpMyAdmin-Reflected-Cross-site-Scripting-Vulnerability-CVE-2013-1937
>
> I've never run phpMyAdmin, and I've certainly never tried to use BeEF with
> a XSS in an attack against it. But you'd have to imagine that it would work
> fine to drive the interface, and that interface looks like it has a full "execute
> any SQL statement <https://www.youtube.com/watch?v=0jjnlpSGB1Y>" section
> in it. Also usually with this sort of program you have a whole "install
> add-on" interface, which if driven at the administrator level, is RCE. I
> don't consider that two bugs, because "installing an add-on" is the
> functionality admin users need to have and it's completely built-in.
>
> So the question is: Can phpMyAdmin be driven AS IF FROM THE ADMIN by this
> XSS (aka, is the proper CVSS score an 11?) I would guess yes. Or, am I
> completely wrong, and the impact is quite limited?
>
> -dave
>
>
>
>
>
>
>
>
>
> On Thu, Jan 10, 2019 at 9:59 AM toby <toby00 at gmail.com> wrote:
>
>> I'm going to nitpick this. Not because your complaints about CVSS are
>> bad, just that they are unsupported and insufficiently explained.
>>
>> On Tue, Jan 8, 2019 at 8:23 AM Dave Aitel <dave.aitel at cyxtera.com> wrote:
>>
>>> I wanted to take a few minutes and do a quick highlight of a paper from
>>> CMU-CERT which I think most people have missed out on:
>>> https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf
>>> Towards Improving CVSS - resources.sei.cmu.edu
>>> <https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf>
>>> resources.sei.cmu.edu
>>> SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY
>>> REV-03.18.2016.0 Distribution Statement A: Approved for Public Release;
>>> Distribution Is Unlimited TOWARDS IMPROVING CVSS
>>> It's almost as funny a read as their previous best work on how "clientless
>>> HTTPS VPNs are insanely dumb <https://www.kb.cert.org/vuls/id/261869/> what were
>>> you thinking omg?"
>>>
>>> They use a ton of big words in the paper to call CVSS out and give it a
>>> shellacking. Like most of you, we have extensive use of CVSS in our
>>> consulting practice and I've seen this stuff first hand. CVSS is of course
>>> just a buggy compression algorithm for taking complex qualitative data and
>>> then putting it on a number line. The paper has three angles here:
>>>
>>>    1. Qualitative mappings into quantitative numbers are a silly thing
>>>    to do, like people trying to do "social science" by using SurveyMonkey.
>>>
>>>
>>
>> A. I have been smacking people who try to pretend that qualitative
>> measurements are made better by wrapping them in numbers for 15 years. I
>> completely agree.
>>
>> Second. We use numbers to represent qualitative values to enable
>> reasoning. You can't multiply High * Medium * Low but you can multiply 5 *
>> 3 * 1. That's not turning qualitative data into quantitative data it is
>> just providing a short cut to think about qualitative data.
>>
>> Finally. Social sciences when done right are collecting quantitative data
>> about qualitative data so Survey Monkey is actually useful from that
>> perspective. We will set aside the problem of selection bias due to access
>> and interest in participation for the moment. The point stands; a single
>> person's assessment of a qualitative thing is not data. 10,000 people's
>> assessment of that qualitative thing is data.
>>
>>
>>
>>>
>>>    1. We're pretty sure that the compression algorithm is not, in fact,
>>>    putting higher risk items as bigger numbers, which is the whole point of
>>>    the thing.
>>>    2. Nobody is applying this in any sort of consistent way (which is
>>>    probably impossible) which is ALSO the whole point of the thing.
>>>
>>>
>>> It's fine to have a lossy compression algorithm that emphasizes certain
>>> aspects of the input signal over others, of course, but an additional
>>> CERT/CC critique is we have no reason to think CVSS does this in any useful
>>> way.
>>>
>>
>> 1. By definition every compression algorithm emphasizes certain aspects
>> of the signal over others. In this case you are complaining that the parts
>> that are emphasized are not the ones you think are important.
>> B. That's completely reasonable. Offer me an alternative. Seriously, I'm
>> not a fan of CVSS but I haven't seen a better alternative to a complete
>> memory dump and description of all the consequences beyond that. So give me
>> an alternative or grab me at the next conference and ply me with drinks and
>> conversation and we can debate it.
>>
>>
>>>
>>> There's definitely people in the CVSS process (who I will avoid calling
>>> out by name) who think ANY quantization is good. But read the paper and
>>> decide for yourself - because these are probably serious issues that are
>>> turning your entire risk org into a Garbage-In-Garbage-Out org...
>>>
>>
>> That I cannot agree more with.
>>
>>
>>>
>>> -dave
>>>
>>>
>>> _______________________________________________
>>> Dailydave mailing list
>>> Dailydave at lists.immunityinc.com
>>> https://lists.immunityinc.com/mailman/listinfo/dailydave
>>>
>> _______________________________________________
>> Dailydave mailing list
>> Dailydave at lists.immunityinc.com
>> https://lists.immunityinc.com/mailman/listinfo/dailydave
>>
> _______________________________________________
> Dailydave mailing list
> Dailydave at lists.immunityinc.com
> https://lists.immunityinc.com/mailman/listinfo/dailydave
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.immunityinc.com/pipermail/dailydave/attachments/20190110/2791492a/attachment-0001.html>