<div dir="ltr">Everywhere I've ever pentested, we've used a low/medium/high or low/medium/high/critical scale - this is my first encounter with DREAD. What you describe though - clients attempting to negotiate down the severity of vulns on the report - was common regardless of the scoring system used. I don't see DREAD being unique in that respect.<br><div><br></div><div>Reflecting, it's probably what pushed me towards the binary system I ended up using. No score - just exploitable or not. Then, if it's someone that just refuses to have a critical finding on their report, they'll argue over the likelihood of it being exploited. Or over whether the finding was in scope. Then I consider firing that client.</div><div><br></div><div>I try to explain - when you're hiring a pentester, you're paying for their opinion as they perform a point-in-time assessment. The client is free to debate the findings and they don't have to agree with the consultant, but the client can't change/rewrite the report without destroying the integrity of the report.</div><div><br></div><div>--Adrian</div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Jan 10, 2019 at 5:21 PM Adam Shostack <<a href="mailto:adam@shostack.org">adam@shostack.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Okay, I'll respond generally about DREAD. The issue comes up when<br>
people say "We'll treat a DREAD rating of >= 8 as critical." Then<br>
someone looks at your discoverability of 7, and says "hmm, if this<br>
were a 6, then DREAD would be 7.9...can we change it?" Lacking any<br>
guidance on the difference, it's hard to say no.<br>
<br>
Really, it's often "You're being unreasonable by making<br>
discoverability a 7 here. It's not that high!"<br>
<br>
Adam<br>
<br>
<br>
<br>
On Thu, Jan 10, 2019 at 04:38:30PM -0500, Adrian Sanabria wrote:<br>
> I probably shouldn't have brought it up - I'm not involved much on the<br>
> pentesting side. I know we've discussed replacing it, but finding little out<br>
> there to replace it with.<br>
> <br>
> In my own work, I find most of my pentesting results come down to a binary<br>
> value (hackable, not hackable) and some sense of likelihood of it getting<br>
> exploited by a malicious party. Highs/mediums/lows all seem pointless when<br>
> emulating the attacker perspective.<br>
> <br>
> Looking at DREAD, I honestly can't say I find anything fatally wrong with it.<br>
> Perhaps it's because I've never known pentesting to be terribly consistent<br>
> across tests or consultants in my career, so the bar is set pretty low in my<br>
> mind?<br>
> <br>
> --Adrian<br>
> <br>
> On Thu, Jan 10, 2019 at 2:12 PM Adam Shostack <<a href="mailto:adam@shostack.org" target="_blank">adam@shostack.org</a>> wrote:<br>
> <br>
> On Wed, Jan 09, 2019 at 08:18:48AM -0500, Adrian Sanabria wrote:<br>
> > Our pentesters use DREAD, which I think most people have moved on from,<br>
> but at<br>
> > least the scoring is clear and consistent. <br>
> <br>
> I'm sorry, but I need to rant a little.<br>
> <br>
> A decade back, I wrote a "DREAD is DEAD, please stop" blog post for<br>
> Microsoft. If you are getting consistent scoring out of DREAD, you<br>
> are not using DREAD (as described in Writing Secure Code 1, which I<br>
> think is the first public description).<br>
> <br>
> You are using some derivitive that adds tools to provide for<br>
> that consistency. Those tools may be as simple as a set of examples<br>
> of each of the constiuents and what constitutes a 7 or a 3.<br>
> <br>
> I care about this because I one of the biggest things that I see<br>
> making threat modeling hard is everyone calls their specific<br>
> collection of techniques 'lightweight threat modeling' or 'agile<br>
> threat modeling' and people trying to learn get confused because<br>
> there's 6 contradictory descriptions that have been labeled "agile<br>
> tm". People writing down process so their engineers can do it<br>
> consistently get confused in the same way they'd get confused if we<br>
> all said "oh yeah. we're writing code, and you can assign variables<br>
> with either = or <=". We name our languages, we version them. We need<br>
> to start doing the same for threat modeling constructs.<br>
> <br>
> If you say "We're using DREADNOP 1.0" that's cool. Alternately, maybe<br>
> you're using DREAD 1.0 in its raw form, in which case, how are you<br>
> getting consistent scores?<br>
> <br>
> Adam<br>
> <br>
> <br>
> > In addition to CVE being wrong on critical details, I've found that most<br>
> of<br>
> > ExploitDB isn't exploits. Many are vulnerability checks and almost all<br>
> are<br>
> > incorrectly entered. PrivEsc will be labeled RCE and RCE will be labeled<br>
> DoS.<br>
> > It's all a mess. If I had the resources to burn it all down and start<br>
> from<br>
> > scratch, I'd do it. <br>
> ><br>
> > --Adrian<br>
> ><br>
> > On Tue, Jan 8, 2019, 17:47 Nathaniel Ferguson <<a href="mailto:jferguson@126.com" target="_blank">jferguson@126.com</a> wrote:<br>
> ><br>
> > > They use a ton of big words in the paper to call CVSS out and give<br>
> it a<br>
> > shellacking. Like most of you, we have extensive use of CVSS in our<br>
> > consulting practice and I've seen this stuff first hand. CVSS is of<br>
> course<br>
> > just a buggy compression algorithm for taking complex qualitative<br>
> data and<br>
> > then putting it on a number line.<br>
> ><br>
> ><br>
> > Over the years I've worked at a few different consultancies and at<br>
> least<br>
> > originally basically no one used any sort of standardized metric, the<br>
> > reports were generally humorous from a technical standpoint as the<br>
> numbers<br>
> > were basically just made up and didn't adhere to even basic<br>
> statistics<br>
> > methodologies-- we take the X and multiple it by Y and add the Z and<br>
> > there's your score! Some even plotted them along cartoon looking<br>
> graphs and<br>
> > plots and my suspicion was that they were really included to give<br>
> depth to<br>
> > the reports and break up the monotony of text on text on text. Oddly,<br>
> I've<br>
> > never even worked at a place that described the methodology as<br>
> outlined in<br>
> > their reports to their employees leaving some question as to how a<br>
> > methodology was to be implemented if only the client ever heard about<br>
> it.<br>
> ><br>
> > In that sense, CVSS et al make some amount of sense, if nothing else<br>
> it<br>
> > standardizes to a common metric and isn't what the sales guy or<br>
> operations<br>
> > manager made up. Additionally, what a strange word-- shellacking,<br>
> there is<br>
> > no 'k' in the word until its made into a present participle.<br>
> ><br>
> > > The paper has three angles here:<br>
> > > Qualitative mappings into quantitative numbers are a silly thing to<br>
> do,<br>
> > like people trying to do "social science" by using SurveyMonkey.<br>
> ><br>
> > Which is what most people are or were selling.<br>
> ><br>
> > > It's fine to have a lossy compression algorithm that emphasizes<br>
> certain<br>
> > aspects of the input signal over others, of course, but an additional<br>
> CERT/<br>
> > CC critique is we have no reason to think CVSS does this in any<br>
> useful way.<br>
> ><br>
> > Well there 's a missing line here, you can see it from the way that<br>
> > client-side attacks perverted the concept of remote and so they made<br>
> them<br>
> > remote also instead of adding the new line to the plot. Because of<br>
> stuff<br>
> > like this. everything is remote now which limits its usefulness. This<br>
> > doesn't even touch on the fact that most of the CVE database is<br>
> basically<br>
> > wrong from submissions including very limited data, id est "memory<br>
> > corruption results in a DoS".<br>
> ><br>
> > Nathaniel<br>
> ><br>
> > 在 2019-01-09 00:14:00,"Dave Aitel" <<a href="mailto:dave.aitel@cyxtera.com" target="_blank">dave.aitel@cyxtera.com</a>> 写道:<br>
> ><br>
> > <br>
> > I wanted to take a few minutes and do a quick highlight of a<br>
> paper from<br>
> > CMU-CERT which I think most people have missed out on: https://<br>
> > <a href="http://resources.sei.cmu.edu/asset_files/WhitePaper/" rel="noreferrer" target="_blank">resources.sei.cmu.edu/asset_files/WhitePaper/</a><br>
> 2018_019_001_538372.pdf<br>
> ><br>
> > Towards Improving CVSS - <a href="http://resources.sei.cmu.edu" rel="noreferrer" target="_blank">resources.sei.cmu.edu</a><br>
> > <a href="http://resources.sei.cmu.edu" rel="noreferrer" target="_blank">resources.sei.cmu.edu</a><br>
> > SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY<br>
> > REV-03.18.2016.0 Distribution Statement A: Approved for Public<br>
> Release;<br>
> > Distribution Is Unlimited TOWARDS IMPROVING CVSS<br>
> > It's almost as funny a read as their previous best work on how "<br>
> > clientless HTTPS VPNs are insanely dumb what were you thinking<br>
> omg?"<br>
> ><br>
> > They use a ton of big words in the paper to call CVSS out and<br>
> give it a<br>
> > shellacking. Like most of you, we have extensive use of CVSS in<br>
> our<br>
> > consulting practice and I've seen this stuff first hand. CVSS is<br>
> of<br>
> > course just a buggy compression algorithm for taking complex<br>
> > qualitative data and then putting it on a number line. The paper<br>
> has<br>
> > three angles here: <br>
> > 1. Qualitative mappings into quantitative numbers are a silly<br>
> thing to<br>
> > do, like people trying to do "social science" by using<br>
> > SurveyMonkey.<br>
> > 2. We're pretty sure that the compression algorithm is not, in<br>
> fact,<br>
> > putting higher risk items as bigger numbers, which is the<br>
> whole<br>
> > point of the thing. <br>
> > 3. Nobody is applying this in any sort of consistent way (which<br>
> is<br>
> > probably impossible) which is ALSO the whole point of the<br>
> thing.<br>
> ><br>
> > It's fine to have a lossy compression algorithm that emphasizes<br>
> certain<br>
> > aspects of the input signal over others, of course, but an<br>
> additional<br>
> > CERT/CC critique is we have no reason to think CVSS does this in<br>
> any<br>
> > useful way. <br>
> ><br>
> ><br>
> > There's definitely people in the CVSS process (who I will avoid<br>
> calling<br>
> > out by name) who think ANY quantization is good. But read the<br>
> paper and<br>
> > decide for yourself - because these are probably serious issues<br>
> that<br>
> > are turning your entire risk org into a Garbage-In-Garbage-Out<br>
> org...<br>
> ><br>
> ><br>
> > -dave<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > <br>
> ><br>
> > _______________________________________________<br>
> > Dailydave mailing list<br>
> > <a href="mailto:Dailydave@lists.immunityinc.com" target="_blank">Dailydave@lists.immunityinc.com</a><br>
> > <a href="https://lists.immunityinc.com/mailman/listinfo/dailydave" rel="noreferrer" target="_blank">https://lists.immunityinc.com/mailman/listinfo/dailydave</a><br>
> ><br>
> <br>
> > _______________________________________________<br>
> > Dailydave mailing list<br>
> > <a href="mailto:Dailydave@lists.immunityinc.com" target="_blank">Dailydave@lists.immunityinc.com</a><br>
> > <a href="https://lists.immunityinc.com/mailman/listinfo/dailydave" rel="noreferrer" target="_blank">https://lists.immunityinc.com/mailman/listinfo/dailydave</a><br>
> <br>
> <br>
> --<br>
> Adam Shostack<br>
> President, Shostack & Associates<br>
> <a href="https://associates.shostack.org" rel="noreferrer" target="_blank">https://associates.shostack.org</a> • +1 917 391 2168<br>
> <br>
> Join my very quiet annnoucement list: <a href="https://adam.shostack.org/newthing" rel="noreferrer" target="_blank">https://adam.shostack.org/newthing</a><br>
> <br>
> <br>
<br>
-- <br>
Adam Shostack<br>
President, Shostack & Associates<br>
<a href="https://associates.shostack.org" rel="noreferrer" target="_blank">https://associates.shostack.org</a> • +1 917 391 2168<br>
<br>
Join my very quiet annnoucement list: <a href="https://adam.shostack.org/newthing" rel="noreferrer" target="_blank">https://adam.shostack.org/newthing</a><br>
<br>
</blockquote></div>