<div dir="auto">Okay, we keep touching on this point, that CVSS isn't intended to score risk, just vulnerability severity. I'm having a hard time seeing what value there is in having a vulnerability score that doesn't reflect risk. What use does it have?<div dir="auto"><br></div><div dir="auto">Or is that exactly what we're saying? That since it doesn't reflect risk, it's essentially useless. If that's the conclusion, I'm on the same page.</div><div dir="auto"><br></div><div dir="auto">--Adrian </div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Jan 10, 2019, 9:56 AM Wim Remes <<a href="mailto:wremes@gmail.com">wremes@gmail.com</a> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space">Hi,<div><br></div><div>Bruce really hits the nail on the head here. CVSS != Risk. To broaden that discussion and not waste too many words, I’ll reference FAIR (Factor Analysis of Information Risk, <a href="https://www.fairinstitute.org/what-is-fair" target="_blank" rel="noreferrer">https://www.fairinstitute.org/what-is-fair</a>) to indicate where “Vulnerability” contributes to an eventual quantitative risk valuation. </div><div><br></div><div>I also always considered CVSS scoring to be qualitative instead of quantitative and the numbers to be ordinal. That makes them fine for ranking vulnerability, but horrible to perform math on (Jet Fuel x Peanut Butter = Shiny — hi Alex Hutton!). </div><div><br></div><div>That said, it all boils down to a point I’ve been rapping on about for a long long time now. Organizations should not expect third party penetration testers to make an accurate assessment of risk. The data provided by a third party penetration tester should feed into your risk management framework, that is also fed with internally acquired business data, to produce (or adjust) a risk valuation. It would be helpful if we, as consultants, wouldn’t pretend that we (a) can come up with any form of credible risk score during such assessments and (b) are delivering scoring that can help with prioritization in a business context without additional effort on the client side. On the other hand, clients that have a risk management framework that can actually take vulnerability scores and use them to generate risk scores should be clear in what they expect from us. If you are asked, whether in an RFP or an SoW, to produce a risk score for your findings at the very least you should be returning a question for asset valuation and threat community descriptions. </div><div><br></div><div>Cheers,</div><div>Wim</div><div><br></div><div><br><div><br><blockquote type="cite"><div>On 8 Jan 2019, at 18:33, Monroe, Bruce <<a href="mailto:bruce.monroe@intel.com" target="_blank" rel="noreferrer">bruce.monroe@intel.com</a>> wrote:</div><br class="m_773332623209258761Apple-interchange-newline"><div><div class="m_773332623209258761WordSection1" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">Hi Dave,</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)"> </span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">I participate on the CVSS SIG being ran out of FIRST that is working on improvements to CVSS.<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761GramE"><span class="m_773332623209258761grame">So</span></span><span class="m_773332623209258761Apple-converted-space"> </span>do a number of people out of CERT CC, NIST, MITRE along with a good representation of industry.<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761grame">A number of</span><span class="m_773332623209258761Apple-converted-space"> </span>us provided feedback on this paper. CVSS is for scoring the severity of a vulnerability. CVSS does not = Risk.</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)"> </span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">My understanding is there is<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761GramE"><span class="m_773332623209258761grame">a number of</span></span><span class="m_773332623209258761Apple-converted-space"> </span>government entities that believe CVSS does = Risk and are using it in a vacuum for that purpose. While the CVSS score is a single component - you also must look at how the vulnerable component is deployed, controls in place, value of asset, patching windows, likelihood of exploit,<span class="m_773332623209258761SpellE"><span class="m_773332623209258761spelle">ect</span></span>…there is a lot that goes into determining risk.</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)"> </span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">The fact that various USG entities is using CVSS wrong is an education issue<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761SpellE"><span class="m_773332623209258761spelle">imo</span></span>.<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761GramE"><span class="m_773332623209258761grame">Yes</span></span><span class="m_773332623209258761Apple-converted-space"> </span>CVSS has it’s issues with some of<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761SpellE"><span class="m_773332623209258761spelle">it’s</span></span><span class="m_773332623209258761Apple-converted-space"> </span>elements being subjective eye of the beholder type items but that isn’t the reason for this paper…they’ve got USG people using it in a vacuum when it’s only a single element of determining your orgs risk due to a vulnerability. That isn’t a CVSS problem that’s a vulnerability management 101 problem.</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)"> </span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">Regards,</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">Bruce</span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">Intel PSIRT<u></u><u></u></span></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)"> </span><u></u><u></u></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Book Antiqua",serif;color:rgb(31,73,125)">Opinions expressed are my own and may not reflect those of my employer.</span><u></u><u></u></div><div><div style="border-style:solid none none;border-top-width:1pt;border-top-color:rgb(225,225,225);padding:3pt 0in 0in"><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><a name="m_773332623209258761______replyseparator" rel="noreferrer"></a><b><span>From:</span></b><span><span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761SpellE">Dailydave</span><span class="m_773332623209258761Apple-converted-space"> </span><<a href="mailto:dailydave-bounces@lists.immunityinc.com" target="_blank" rel="noreferrer">dailydave-bounces@lists.immunityinc.com</a>><span class="m_773332623209258761Apple-converted-space"> </span><b>On Behalf<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761GramE">Of</span><span class="m_773332623209258761Apple-converted-space"> </span></b>Dave<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761SpellE">Aitel</span><br><b>Sent:</b><span class="m_773332623209258761Apple-converted-space"> </span>Tuesday, January 08, 2019 8:14 AM<br><b>To:</b><span class="m_773332623209258761Apple-converted-space"> </span><a href="mailto:dailydave@lists.immunityinc.com" target="_blank" rel="noreferrer">dailydave@lists.immunityinc.com</a><br><b>Subject:</b><span class="m_773332623209258761Apple-converted-space"> </span>[<span class="m_773332623209258761SpellE">Dailydave</span>] CVSS is the worst compression algorithm ever<u></u><u></u></span></div></div></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><u></u> <u></u></div><div id="m_773332623209258761divtagdefaultwrapper"><p><span style="font-size:12pt">I wanted to take a few minutes and do a quick highlight of a paper from CMU-CERT which I think most people have missed out on: <a href="https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf" style="color:purple;text-decoration:underline" target="_blank" rel="noreferrer">https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf</a><u></u><u></u></span></p><div id="m_773332623209258761LPBorder_GT_15469630031460.183622448101032" style="margin-bottom:15pt;overflow:auto"><table class="m_773332623209258761MsoNormalTable" border="1" cellspacing="0" cellpadding="0" width="90%" style="width:452.6875px;background-color:white;border-top-width:1pt;border-style:dotted none;border-top-color:rgb(200,200,200);border-bottom-width:1pt;border-bottom-color:rgb(200,200,200);background-position:initial initial;background-repeat:initial initial"><tbody><tr><td valign="top" style="border:none;padding:0in"><div id="m_773332623209258761LPTitle_15469630031430.5656392074504"><p class="MsoNormal" style="margin:15pt 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:16pt;font-family:"Segoe UI Light",sans-serif;color:rgb(105,9,139)"><a href="https://resources.sei.cmu.edu/asset_files/WhitePaper/2018_019_001_538372.pdf" style="color:purple;text-decoration:underline" target="_blank" rel="noreferrer"><span style="text-decoration:none">Towards Improving CVSS - resources.sei.cmu.edu</span></a><u></u><u></u></span></p></div><div id="m_773332623209258761LPMetadata_15469630031440.623748296566867" style="margin-top:7.5pt;margin-bottom:12pt"><p class="MsoNormal" style="margin:15pt 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;line-height:10.5pt"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:rgb(102,102,102)"><a href="http://resources.sei.cmu.edu" target="_blank" rel="noreferrer">resources.sei.cmu.edu</a><u></u><u></u></span></p></div><div id="m_773332623209258761LPDescription_15469630031450.5530823382524725"><p class="MsoNormal" style="margin:15pt 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;line-height:15pt"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:rgb(102,102,102)">SOFTWARE ENGINEERING INSTITUTE | CARNEGIE MELLON UNIVERSITY REV-03.18.2016.0 Distribution Statement A: Approved for Public Release; Distribution Is Unlimited TOWARDS IMPROVING CVSS<u></u><u></u></span></p></div></td></tr></tbody></table></div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">It's almost as funny a read as their previous best work on how "<a href="https://www.kb.cert.org/vuls/id/261869/" style="color:purple;text-decoration:underline" target="_blank" rel="noreferrer">clientless HTTPS VPNs are insanely dumb</a> what were you thinking omg?"<span class="m_773332623209258761Apple-converted-space"> </span><u></u><u></u></span></div><div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt"><u></u> <u></u></span></div></div><div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">They use a ton of big words in the paper to call CVSS out and give it a shellacking. Like most of you, we have extensive use of CVSS in our consulting practice and I've seen this stuff first hand. CVSS is of course just a buggy compression algorithm for taking complex qualitative data and then putting it on a number line. The paper has three angles here: <u></u><u></u></span></div></div><div><ol start="1" type="1" style="margin-bottom:0in"><li class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">Qualitative mappings into quantitative numbers are a silly thing to do, like people trying to do "social science" by using SurveyMonkey.<u></u><u></u></span></li><li class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">We're pretty sure that the compression algorithm is not, in fact, putting higher risk items as bigger numbers, which is the whole point of the thing.  <u></u><u></u></span></li><li class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">Nobody is applying this in any sort of consistent way (which is probably impossible) which is ALSO the whole point of the thing.<u></u><u></u></span></li></ol></div><div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt"><u></u> <u></u></span></div></div><div><div style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-size:12pt">It's fine to have a lossy compression algorithm that emphasizes certain aspects of the input signal over others, of course, but an additional CERT/CC critique is we have no reason to think CVSS does this in any useful way. <u></u><u></u></span></div><p><span style="font-size:12pt"><u></u> <u></u></span></p><p><span style="font-size:12pt">There's<span class="m_773332623209258761Apple-converted-space"> </span><span class="m_773332623209258761GramE">definitely people</span><span class="m_773332623209258761Apple-converted-space"> </span>in the CVSS process (who I will avoid calling out by name) who think ANY quantization is good. But read the paper and decide for yourself - because these are probably serious issues that are turning your entire risk org into a Garbage-In-Garbage-Out org...<u></u><u></u></span></p><p><span style="font-size:12pt"><u></u> <u></u></span></p><p><span style="font-size:12pt">-<span class="m_773332623209258761SpellE">dave</span><u></u><u></u></span></p><p><span style="font-size:12pt"><u></u> <u></u></span></p></div></div></div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">_______________________________________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important">Dailydave mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important"><a href="mailto:Dailydave@lists.immunityinc.com" target="_blank" rel="noreferrer">Dailydave@lists.immunityinc.com</a></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline!important"><a href="https://lists.immunityinc.com/mailman/listinfo/dailydave" target="_blank" rel="noreferrer">https://lists.immunityinc.com/mailman/listinfo/dailydave</a></span></div></blockquote></div><br></div></div>_______________________________________________<br>
Dailydave mailing list<br>
<a href="mailto:Dailydave@lists.immunityinc.com" target="_blank" rel="noreferrer">Dailydave@lists.immunityinc.com</a><br>
<a href="https://lists.immunityinc.com/mailman/listinfo/dailydave" rel="noreferrer noreferrer" target="_blank">https://lists.immunityinc.com/mailman/listinfo/dailydave</a><br>
</blockquote></div>