[Dailydave] Encrypted Malware Traffic Detection == hilarious?

[Robert Graham]

There are two kinds of AI/ML:1. the kind that recognizes what humans recognize (faces, cars, etc.)2. the kind that recognizes things humans can't see (stock market trends, etc.)
The first item is real, and is slowly changing the world. The second is bogus, snake oil, emperors without clothes.
As long as I've been in the field of network intrusion detection (more than 2 decades), there have been a stream of papers every year promising machines can see evil on the network that humans couldn't see. They've never worked in practice.
That's not to say good things don't exist. Arbor Networks, for example, does fine job at pointing out anomalies. But it's based on human ingenuity, not machine learning, and it requires human effort to use.




 

    On Wednesday, June 21, 2017 10:40 AM, dave aitel <dave at immunityinc.com> wrote:
 

   Let's talk about the giant pile of wrong that is this reporting on Cisco's new marketing campaign around detecting encrypted malware traffic. "This is a seminal moment in networking" is the quote from their CEO that CNBC decided to run. Let's revisit the basics of this "new" technology: do statistical analysis on encrypted data to find malware traffic. 
  People have literally decoded conversations from encrypted data using that same basic technique. Not even recently - that work is from 2008 and was not surprising even then.
  "The software, which will be offered as a subscription service, is currently in field trials with 75 customers, and according to Robbins, is 99 percent effective." 99% effective with the kind of traffic a normal network sees means you are FLOODED AND OVERWHELMED WITH FALSE POSITIVES. Although they don't specify what that number even means. Is it false positives? False negatives? Both? Let's just say this: 99.99% is useless when doing a network-based IDS. All that might get you is an indicator you can use to remotely load a more sophisticated remote tool onto an endpoint for further detailed analysis. You essentially, need BOTH if you have this level of network-based IDS, and the endpoint people will probably say you don't need the network sniffer anymore, because scaling good analysis at that level at anything near realtime is nearly impossible (c.f. Alex Stamos's talk) to the point where they still try to sell you stuff that has 1% false positive rates. :) I'm going to bug our big customers to see if any of them are in this 75 field trial and what they think in real life. And I'm going to be honest and say that if you are thinking of investing in this sort of thing, but you haven't tested it against Cobalt Strike and INNUENDO, then you are knowingly buying snake oil. A good percentage of our consulting business right now is literally just that because these anomaly detection products are so expensive and so hard to test. Anyways, maybe I am wrong! If you are one of the privileged 75 and you love this and it is amazing, let me/us know!
  -dave 
  
  _______________________________________________
Dailydave mailing list
Dailydave at lists.immunityinc.com
https://lists.immunityinc.com/mailman/listinfo/dailydave


   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.immunityinc.com/pipermail/dailydave/attachments/20170623/ee6b07a6/attachment.html>