From dave at immunityinc.com Mon Sep 18 16:09:30 2017 From: dave at immunityinc.com (dave aitel) Date: Mon, 18 Sep 2017 12:09:30 -0400 Subject: [Dailydave] So many bugs... Message-ID: <55265a45-1a6f-ece9-16ea-37b38b56c69a@immunityinc.com> Seriously - it is like the Cambrian explosion up in here. Every platform seems to have dissolved - be it Java, or Windows, or various forms of "Secure Computing" now protected by a combination of platitudes and useless aphorisms. For example, check out this news article from last week: Not to pick on anyone in particular, but there's a word for "bug that allows guests to execute code on hosts" and it's "Hypervisor escape", which sounds more appreciably scary and impactful except we like to pretend they don't exist. :) Look, I dunno what sort of policy arm or governmental agency is supposed to do the big picture stuff in cyber.  But it might be worth poking them if they sit next to you and pointing out that climate change may, or may not be, controlled by humans, but for whatever reason, it's getting pretty windy out if you consider Struts bugs to be air current, at least. The NTIA would call this "Market Failure" but perhaps it is more pretense failure? Recently, as a study in pretense, I spent some time looking at the Florida State educational assessments and by their own depressed standards, Florida fails 40% of its students in Math at pretty much every grade level . So even while the Miami Dade Educational Commissioner iscrowing about how no school is an "F" this year - I'm not sure if you can judge a school system full of 60%s anything other than an Fail overall. We spend all our time arguing about the details of our public school system but the pretense is that you even have a public school system, in other words. It's not hard to draw similar analogies to a lot of how we talk about the information security ecosystem. Or perhaps it's just that too much time in the rarefied gasses of the policy world have depressed me, and I need some time turning rage into clicky-clicky things that may or may not pop calculators.  Anyways, if you like writing up exploits, for the many many cool bugs that are now out there, please let me know because we are still hiring. You do have to out-hack me during the interview though. :) And if you want to know the one thing that does scare me, as an attacker[1], then I'll be going into it in depth in my keynote at T2 next month, although without revealing anything secret[2], which, as it turns out, is a super hard balancing act. So far reviews of the talk have gone from "That was awful, like eating glass but more painful" to "This is not great". -dave [1]. It is automated security response and apoptosis. [2]. This is the slide I'm having the most trouble with, because I don't want to call people out, but I think it is an important concept strategically. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ggeaakapkekjlnjn.png Type: image/png Size: 269054 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dhifpmhhkngbplnb.png Type: image/png Size: 487100 bytes Desc: not available URL: From dave at immunityinc.com Wed Sep 27 15:13:28 2017 From: dave at immunityinc.com (dave aitel) Date: Wed, 27 Sep 2017 11:13:28 -0400 Subject: [Dailydave] Equitablefax Message-ID: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> So I assume most people skim any news reports of big breaches in the same way these days. Was this predictable? Was it preventable? Do we know who did it? Did they do anything new to attack or defend? In Equifax's case, the reportable information clearly is the alleged trading anomalies, rather than the hack itself. But the third question is interesting to a point. I've been trying to write a keynote for T2 for the past few weeks, and while my muse is clearly on an extended vacation, there are some interesting generational changes afoot with regards to these questions. At some level, in a world where vulnerabilities are super rare, governments dominate the discussion of malicious actors. I think there's a lot of news chaff about every little 20-something hacker or aspiring malware businessman who gets caught. Filtering those out, there are relatively few reports of hacking groups with high skills levels. And because of our assumptions that "Governments" are behind everything now, I think we naturally err towards flinching at boogeymen who...wield SQLi and Phishing with .jar files. But when you look at the accomplishments of truly skilled hackers, they're amazing. And the environment we live in is not one where major vulnerabilities are rare. The environment is such that any specialized extremophile can penetrate and persist all of cyberspace. In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific. Likewise, learning everything about RMI is enough to be everywhere, or .Net serialization, or CCleaner. In cyber, where there's a way there's a will. It used to be we would be more afraid if it was China or Russia or Iran or whoever. But these days I like to annoy people by asking what if it's not? Also, does anyone know how often Equifax did their penetration testing? My new rule is that if you only do it in Q4 you are unlikely to have a mature security program. :) -dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve_smith1999 at yahoo.com Wed Sep 27 16:00:21 2017 From: steve_smith1999 at yahoo.com (Steve R. Smith) Date: Wed, 27 Sep 2017 16:00:21 +0000 (UTC) Subject: [Dailydave] Equitablefax In-Reply-To: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: <2029208485.11305182.1506528021594@mail.yahoo.com> Was this predictable: probably I would be surprised if the PCI assessors (and therefore leadership) didn't know about some of the control environment deficiencies. Typically you get - "that's not a priority", "it was designed that way", "we need to update to the next version first", or even "we don't have the budget to fix that". In some cases, if you think it's an issue - you have to rationalize, push, and play politics to get it addressed. Maybe even threaten to escalate the issue. I've had IT VPs that I worked with refuse to fix something because it was a revenue generating system and they didn't want to risk business objectives.     Was it preventable: unlikely I think based on historical trends and what we see in the wild, we can predict with confidence that many companies are and/or will be at risk for compromise. IT environments were complicated 18 years ago when I first got into security and they've become even more complicated with the evolution of technology.      Do we know who did it: maybe  Mandiant is very good at what they do but sometimes attribution just isn't possible because of all the hops the attackers may have taken to get to their final target. The other compromised systems sometimes live in countries that won't help us investigate cyber crimes.   Did they do anything to new to attack or defend: unlikely As you point out above, there are many vulnerabilities that go unpatched and unaddressed. Combine that with IT operational mistakes and you may have have a large environment susceptible to compromise. This could be a misconfiguration (TFTP with / access, world readable/writeable cron scripts owned by root), purposeful change that introduces a weakness (open NFS shares combined with availability of r-services, open X display), trust relationships, shared passwords across the environment- you name it.   My rule is if all you're doing are the bare minimums and/or you have leadership pushing back in the form of not providing executive level support, determining your strategy or tactics, or limiting your budget - you are unlikely to have an effective security program. By the way - I think you're right. We focus way too much on claiming these compromises are caused by nation states. It very well could be one person or a small team of opportunists.  No, I have no clue how or the frequency of their penetration testing. Considering that it's been reported that web portals with easily guessable usernames/passwords were used for data exfiltration, their competence is questionable.  Kind regards, ~steve  On Wednesday, September 27, 2017, 10:15:12 AM CDT, dave aitel wrote: So I assume most people skim any news reports of big breaches in the same way these days. Was this predictable? Was it preventable? Do we know who did it? Did they do anything new to attack or defend? In Equifax's case, the reportable information clearly is the alleged trading anomalies, rather than the hack itself. But the third question is interesting to a point. I've been trying to write a keynote for T2 for the past few weeks, and while my muse is clearly on an extended vacation, there are some interesting generational changes afoot with regards to these questions. At some level, in a world where vulnerabilities are super rare, governments dominate the discussion of malicious actors. I think there's a lot of news chaff about every little 20-something hacker or aspiring malware businessman who gets caught. Filtering those out, there are relatively few reports of hacking groups with high skills levels. And because of our assumptions that "Governments" are behind everything now, I think we naturally err towards flinching at boogeymen who...wield SQLi and Phishing with .jar files. But when you look at the accomplishments of truly skilled hackers, they're amazing. And the environment we live in is not one where major vulnerabilities are rare. The environment is such that any specialized extremophile can penetrate and persist all of cyberspace. In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific. Likewise, learning everything about RMI is enough to be everywhere, or .Net serialization, or CCleaner. In cyber, where there's a way there's a will. It used to be we would be more afraid if it was China or Russia or Iran or whoever. But these days I like to annoy people by asking what if it's not? Also, does anyone know how often Equifax did their penetration testing? My new rule is that if you only do it in Q4 you are unlikely to have a mature security program. :) -dave _______________________________________________ Dailydave mailing list Dailydave at lists.immunityinc.com https://lists.immunityinc.com/mailman/listinfo/dailydave -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian.hermansen at gmail.com Wed Sep 27 16:30:40 2017 From: kristian.hermansen at gmail.com (Kristian Erik Hermansen) Date: Wed, 27 Sep 2017 09:30:40 -0700 Subject: [Dailydave] Equitablefax In-Reply-To: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: If Equifax had a public bug bounty program, someone would have reported the Java RCE in March 2017 and picked up $10K or more for it. But no, Equifax did not have a public bug bounty program. Say what you will about the pros and cons of a bug bounty program, especially for financial institutions which "know better than the public how to protect themselves", but at least in this case a known issue would have been well documented much earlier. We should encourage other credit and financial companies to consider public or at the very least private bug bounty programs. It's a mess to operate them, but not patching a known critical web flaw ASAP that allows RCE is precisely the legal definition of negligence. Equifax should pay dearly for it. Perhaps it's time to consider federal Cyber Security Insurance laws for such companies which forces them to pay fees to operate on the Internet just like everyone that drives a car on the road? If you crash your car every time you get on the highway, or you damaged 140 million cars while driving, you would lose your license for some time. Why hasn't Equifax lost their license to operate on the internet for some time? How about a 2 year hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar can chime in on why Cyber Security Insurance regulation like that is OR is not the answer. He has been working on that lately... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmcauley at ixiacom.com Wed Sep 27 18:00:50 2017 From: cmcauley at ixiacom.com (Chuck McAuley) Date: Wed, 27 Sep 2017 18:00:50 +0000 Subject: [Dailydave] Equitablefax In-Reply-To: References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: <695C87E8-B60B-4B7D-9BC8-F4A2B04EB1BE@ixiacom.com> In the US, the roads are owned by someone (Private Individual, Town, State, Country). They can set the rules for driving on them as they see fit. Who owns the Internet? In the US, definitely not the government. I guess you could argue it would be ISPs. They could govern who peers. But why would they care? More noise should be made that the current credit scoring model cannot be trusted after this PII data has been leaked. I can't see a reliable means to protect 'your' score after this breach. -chuck From: Dailydave on behalf of Kristian Erik Hermansen Date: Wednesday, September 27, 2017 at 1:32 PM To: Dave Aitel Cc: dailydave Subject: Re: [Dailydave] Equitablefax If Equifax had a public bug bounty program, someone would have reported the Java RCE in March 2017 and picked up $10K or more for it. But no, Equifax did not have a public bug bounty program. Say what you will about the pros and cons of a bug bounty program, especially for financial institutions which "know better than the public how to protect themselves", but at least in this case a known issue would have been well documented much earlier. We should encourage other credit and financial companies to consider public or at the very least private bug bounty programs. It's a mess to operate them, but not patching a known critical web flaw ASAP that allows RCE is precisely the legal definition of negligence. Equifax should pay dearly for it. Perhaps it's time to consider federal Cyber Security Insurance laws for such companies which forces them to pay fees to operate on the Internet just like everyone that drives a car on the road? If you crash your car every time you get on the highway, or you damaged 140 million cars while driving, you would lose your license for some time. Why hasn't Equifax lost their license to operate on the internet for some time? How about a 2 year hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar can chime in on why Cyber Security Insurance regulation like that is OR is not the answer. He has been working on that lately... -------------- next part -------------- An HTML attachment was scrubbed... URL: From kristian.hermansen at gmail.com Wed Sep 27 18:06:39 2017 From: kristian.hermansen at gmail.com (Kristian Erik Hermansen) Date: Wed, 27 Sep 2017 11:06:39 -0700 Subject: [Dailydave] Equitablefax In-Reply-To: References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: But clearly Equifax didn't know ALL public facing attack surfaces controlled by Equifax which were affected by that vulnerability. A bug bounty likely would have surfaced those missing attack surfaces. Internal folks always make assumptions about their own network, which is biased and almost never reality. >From the Equifax blog post: - Based on the company's investigation, Equifax believes the unauthorized accesses to certain files containing personal information occurred from May 13 through July 30, 2017. - The particular vulnerability in Apache Struts was identified and disclosed by U.S. CERT in early March 2017. - Equifax's Security organization was aware of this vulnerability at that time, and took efforts to identify and to patch any vulnerable systems in the company's IT infrastructure. - While Equifax fully understands the intense focus on patching efforts, the company's review of the facts is still ongoing. The company will release additional information when available. There is also no mention of the other International systems that had "admin/admin" as the portal credentials to some customer data. Just like when Yahoo was affected by HeartBleed in 2014 and went on to write a blog post about "all systems being fully patched and heartbleed no longer being on the Yahoo network" (months later) I disclosed numerous additional systems that Yahoo operated that were still unpatched and leaking private data. It's hard to identify ALL attack surfaces. And even if Equifax thought they were well patched, maybe they forgot to reload the application / libraries or reboot the systems. Anyone that has run a full entity Internet facing penetration test knows that there is the list that you get from the client that they THINK is the attack surface...and that list is almost always incomplete. It's the duty of a pentester to fill in those gaps, validate if the list is complete, and suggest additional targets for inclusion if appropriate. External attackers don't have that internal organizational bias and that's why you should consult wide external expertise for something so important. I still stand by the claimed benefits of such a bug bounty system. It's clear that Equifax hadn't patched enough systems quickly enough...well into March and beyond. What if I told you Equifax still has at least one publicly facing system still vulnerable to that March Struts bug? Would that change your mind? -------------- next part -------------- An HTML attachment was scrubbed... URL: From konrads.smelkovs at gmail.com Wed Sep 27 18:17:04 2017 From: konrads.smelkovs at gmail.com (Konrads Smelkovs) Date: Wed, 27 Sep 2017 19:17:04 +0100 Subject: [Dailydave] Why people aren't stealing ADFS secrets? Message-ID: I was thinking about long term persistence and clearly, it would make a lot of sense to steal the private key of the ADFS certificate that is used to authenticate SAML claims. Anyone seen it done? -- Konrads Smelkovs Applied IT sorcery. -------------- next part -------------- An HTML attachment was scrubbed... URL: From k8ek8e at gmail.com Wed Sep 27 19:38:25 2017 From: k8ek8e at gmail.com (Katie M) Date: Wed, 27 Sep 2017 12:38:25 -0700 Subject: [Dailydave] Equitablefax In-Reply-To: References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: Having a bug bounty program wouldn't have helped Equifax. Only Equifax could have helped Equifax. The root cause of the problem wasn't that they didn't know about the bug, it was that they face the same patch prioritization risk vs resource balance that all orgs gamble with. They lost that gamble, which is what every breach represents: a lost bet on the tradeoffs. Simply knowing about a bug, via a bug bounty or otherwise, is just that. And knowing is at best half the battle. But to return to Dave's assertion about the bug bounty ecosystem itself and what it currently is good for and what it's used for - I have many thoughts. And even more songs. "In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific. " https://en.m.wikipedia.org/wiki/Mariana_Trench Agreed that the bug bounty market has evolved in this *particular stage of it's growth* through its own complex system, the dynamics of which are heavily influenced by factors like: 1. the types of organizations who have been adopting these incentives so far (mostly tech companies), 2. the typical targets (mostly web sites), and 3. the types of vulnerabilities they tend to use bug bounties to find (mostly low hanging fruit that could have been found using common free tools & techniques). Also a factor in this ecosystem is the geolocation and socioeconomic status of the script kiddie bug hunting masses, who, unlike the early professional penetration testers like us, don't have to adapt their techniques to find more interesting, higher quality bugs to continue to be paid relatively small amounts that are worth much more to them in their part of the world. That's good for those bug hunters who are in this category. That's actually bad for the evolution of the bug bounty ecosystem, and is accurate in Dave's characterization of what's happening *right now*. The upside effect though is that the bug hunter masses can now access a safe marketplace for their skills regardless of those facts of where they are and whether they could ever become a "security consultant". That's generally good, but the *dominant* "species" of bug hunter, as Dave accurately points out about them now will remain relatively unskilled if we don't act with higher-order outcomes in mind. It will be like an attempt at brewing beer that gets taken over and soured by undesirable flora before the brewers yeast kicks in and creates the desired effect. And I've been brewing the defensive market for vulnerabilities far too long to watch idly and let the batch sour. We ideally want to create an upward trend in bug hunter population skills, as well as move the bug hunter targets themselves, towards more sophisticated bugs. We are not raising the tide, and we are not causing all ships to rise with it. Just by slapping a bug bounty or vuln disclosure program on something, we are missing the point. One of the papers that we produced out of the MIT Sloan School visiting scholar systems modeling I worked on will come out sometime this fall (2017) as a chapter in an MIT Press book. That paper looks specifically at bug bounty participant data at a specific point in the development of this economy. Bet you're curious about that supply side snapshot of the bug bounty Mariana Trench. :) Look for that book with our research paper when it's out. Bug bounties as they have mostly manifested *right now, at this specific stage in that ecosystem's development,* are a cheap, shiny thing to do, with few exceptions. And no, the exceptional bug bounties are not the ones that pay the most more on that later. The presence of a bug bounty program is being currently used by organizations to virtue signal that they take security seriously by paying for web bugs, but often missing or ignoring aggregate threats, and ignoring their internal failing processes to fix bugs. It matters very much what's on the inside, versus the superficial, shiny, bug bounty exterior. Shiny (but still very insecure): https://www.youtube.com/watch?v=93lrosBEW-Q The alliterative buzz word "bug bounty", deceptively simple and so very misunderstood, needs to evolve as an accepted concept into the more accurate, more strategic "incentives". Straight cash as the only lever for bringing all the (good) bugs to the yard is short-sighted & pollutes the entire defensive reward ocean in this evolution of the vulnerability and exploit markets. Cash is only one lever in this system, and it isn't the most effective one if you're buying bugs for defense purposes, as I've been saying for several years. Perhaps if a strapping demigod of security would just repeat this for me, it would replace the econ 101 BS that has plagued the emerging bug bounty market. Of course, I'm sure they'd happily forget where they heard it first. https://www.youtube.com/watch?v=79DijItQXMM Just kidding, I'll speak for myself, as always: https://www.rsaconference.com/writable/presentations/file_ upload/ht-t08-the-wolves-of-vuln-street-the-1st-dynamic- systems-model-of-the-0day-market_final.pdf Better-than-a-bug-bounty incentives that are much more effective for improving defense may not be direct cash, may not be rewards at all. Instead they might be a much harder deep introspective process, to examine what drives the heart of an organization, what they are doing to defend what's important to them, and whether the security choices, tradeoffs, resources, and budgets are actually working for them. What incentives can they use to tease out real risk, rather than being lazy and trendy and calling it a success. No, a bug bounty would not have helped Equifax prevent what happened, and we need to seriously stop the VC-backed tsunami of propaganda that says that it would. That stupid marketing trick employed by at least one of the bug bounty platform vendors should be beneath the critically-thinking readers of this list to entertain in terms of its obvious oversimplicity of a non-trivial problem. I'm not even going to address the cyber insurance idea on this, and by now in this long operetta of a post, it should be obvious as to why. Bug bounties and cyber insurance are not a remedy for a fundamentally unscalable remediation model that most orgs and governments face today. That's precisely why 94% of the Forbes Global 2000 in 2015 didn't even have a front door to report a vulnerability, let alone a bug bounty, and it's not much better now. They struggle to fix the bugs they already know about, and the bottlenecks in that *internal* process are what need work. Putting up a front door to receive bug reports, even without a bug bounty, when there's nothing operationally sufficient inside that org to address what comes through the door, is not the chaos an org needs in the midst of drowning in technical debt. It's time to return the heart of the bug bounty ocean to stop the spread of this intellectual and ecosystem-poisoning darkness. I've been staring at the edge of the water, long as I can remember... https://www.youtube.com/watch?v=GeIHvhnQbbI - k8eM0ana https://www.youtube.com/watch?v=Lg_cweoJXyo 🏝️👩‍💻🐞🌋🌺 @k8em0 @lutasecurity @k8eM0ana 🏝️👩‍💻🐞🌋🌺 On Wed, Sep 27, 2017 at 9:30 AM, Kristian Erik Hermansen < kristian.hermansen at gmail.com> wrote: > If Equifax had a public bug bounty program, someone would have reported > the Java RCE in March 2017 and picked up $10K or more for it. But no, > Equifax did not have a public bug bounty program. Say what you will about > the pros and cons of a bug bounty program, especially for financial > institutions which "know better than the public how to protect themselves", > but at least in this case a known issue would have been well documented > much earlier. We should encourage other credit and financial companies to > consider public or at the very least private bug bounty programs. It's a > mess to operate them, but not patching a known critical web flaw ASAP that > allows RCE is precisely the legal definition of negligence. Equifax should > pay dearly for it. > > Perhaps it's time to consider federal Cyber Security Insurance laws for > such companies which forces them to pay fees to operate on the Internet > just like everyone that drives a car on the road? If you crash your car > every time you get on the highway, or you damaged 140 million cars while > driving, you would lose your license for some time. Why hasn't Equifax lost > their license to operate on the internet for some time? How about a 2 year > hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar > can chime in on why Cyber Security Insurance regulation like that is OR is > not the answer. He has been working on that lately... > > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kyle.creyts at gmail.com Wed Sep 27 20:14:27 2017 From: kyle.creyts at gmail.com (Kyle Creyts) Date: Wed, 27 Sep 2017 20:14:27 +0000 Subject: [Dailydave] Why people aren't stealing ADFS secrets? In-Reply-To: References: Message-ID: Or other SAML IDP private keys. ADFS is good, but stealing them from IDP vendors might be much more efficient, and open many more doors. One hopes that Google, OneLogin, Okta, and friends all do the needful to compartment and protect these private keys. On Wed, Sep 27, 2017 at 1:00 PM Konrads Smelkovs wrote: > > I was thinking about long term persistence and clearly, it would make a > lot of sense to steal the private key of the ADFS certificate that is used > to authenticate SAML claims. Anyone seen it done? > > > -- > Konrads Smelkovs > Applied IT sorcery. > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave > -- Kyle Creyts -------------- next part -------------- An HTML attachment was scrubbed... URL: From k8ek8e at gmail.com Wed Sep 27 20:07:20 2017 From: k8ek8e at gmail.com (Katie M) Date: Wed, 27 Sep 2017 13:07:20 -0700 Subject: [Dailydave] Equitablefax In-Reply-To: References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: I actually tried helping coordinate one of the new bugs that someone found and wanted to report to Equifax. Unfortunately, before they had time to even look up from their current conflagration, eyebrows still singed, a reporter published it. At this instant, even one bug report, while completely helpful in the micro-sense, is process-wise another tax on the resources they have working on the big breach. It still has to go into the queue of their existing technical debt in a long mission of what they are already clearly struggling with. Not to say don't report it - definitely do and I can help if that's the issue. But that is very different than recommending a bug bounty to them right now. But a homeowner currently putting out a fire on their house shouldn't be simultaneously setting up a bug bounty program to pay for folks to point out that each blade of dry grass on their lawn is also flammable and could cause another fire. -K8e On Wed, Sep 27, 2017 at 11:06 AM, Kristian Erik Hermansen < kristian.hermansen at gmail.com> wrote: > But clearly Equifax didn't know ALL public facing attack surfaces > controlled by Equifax which were affected by that vulnerability. A bug > bounty likely would have surfaced those missing attack surfaces. Internal > folks always make assumptions about their own network, which is biased and > almost never reality. > > From the Equifax blog post: > > - Based on the company's investigation, Equifax believes the > unauthorized accesses to certain files containing personal information > occurred from May 13 through July 30, 2017. > - The particular vulnerability in Apache Struts was identified and > disclosed by U.S. CERT in early March 2017. > - Equifax's Security organization was aware of this vulnerability at > that time, and took efforts to identify and to patch any vulnerable systems > in the company's IT infrastructure. > - While Equifax fully understands the intense focus on patching > efforts, the company's review of the facts is still ongoing. The company > will release additional information when available. > > There is also no mention of the other International systems that had > "admin/admin" as the portal credentials to some customer data. > > Just like when Yahoo was affected by HeartBleed in 2014 and went on to > write a blog post about "all systems being fully patched and heartbleed no > longer being on the Yahoo network" (months later) I disclosed numerous > additional systems that Yahoo operated that were still unpatched and > leaking private data. It's hard to identify ALL attack surfaces. And even > if Equifax thought they were well patched, maybe they forgot to reload the > application / libraries or reboot the systems. > > Anyone that has run a full entity Internet facing penetration test knows > that there is the list that you get from the client that they THINK is the > attack surface...and that list is almost always incomplete. It's the duty > of a pentester to fill in those gaps, validate if the list is complete, and > suggest additional targets for inclusion if appropriate. External attackers > don't have that internal organizational bias and that's why you should > consult wide external expertise for something so important. > > I still stand by the claimed benefits of such a bug bounty system. It's > clear that Equifax hadn't patched enough systems quickly enough...well into > March and beyond. What if I told you Equifax still has at least one > publicly facing system still vulnerable to that March Struts bug? Would > that change your mind? > > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jpleger at gmail.com Thu Sep 28 02:12:36 2017 From: jpleger at gmail.com (James Pleger) Date: Wed, 27 Sep 2017 19:12:36 -0700 Subject: [Dailydave] Why people aren't stealing ADFS secrets? In-Reply-To: References: Message-ID: <50EF1F3E-4607-4BE6-92E3-FDED8EABA28E@gmail.com> I'm not holding out much hope on the OneLogin side, the breach they had earlier this year sounded really bad. Maybe that event woke up the other identity providers though. http://www.zdnet.com/article/onelogin-security-chief-new-details-data-breach/ > On Sep 27, 2017, at 13:14, Kyle Creyts wrote: > > Or other SAML IDP private keys. ADFS is good, but stealing them from IDP vendors might be much more efficient, and open many more doors. One hopes that Google, OneLogin, Okta, and friends all do the needful to compartment and protect these private keys. > >> On Wed, Sep 27, 2017 at 1:00 PM Konrads Smelkovs wrote: >> >> I was thinking about long term persistence and clearly, it would make a lot of sense to steal the private key of the ADFS certificate that is used to authenticate SAML claims. Anyone seen it done? >> >> >> -- >> Konrads Smelkovs >> Applied IT sorcery. >> _______________________________________________ >> Dailydave mailing list >> Dailydave at lists.immunityinc.com >> https://lists.immunityinc.com/mailman/listinfo/dailydave > > -- > Kyle Creyts > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave -------------- next part -------------- An HTML attachment was scrubbed... URL: From thegrugq at gmail.com Thu Sep 28 23:12:34 2017 From: thegrugq at gmail.com (the grugq) Date: Fri, 29 Sep 2017 06:12:34 +0700 Subject: [Dailydave] Equitablefax In-Reply-To: References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> Message-ID: <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> I’m not going to address any of the points in the excellent post by Katie but rather put some facts together in a timeline so people can see the Equihax event better. The “if only bug bounty” claptrap is, as Katie points out (much more politely), complete bullshit. Timeline of events: 2017-03-06: Apache announces struts bug 2017-03-07: PoC exploit released to public 2017-03-10: Equihax compromised via struts exploit. Genius hackers use super elite hacker command “whoami” during their sophisticated hacking session. [0] 2017-03-13: Equihax genius elite hackers install 30 webshells to allow traversing all the different compromised hosts to pass data out of the company 2017-04-xx: Oracle releases quarterly bundle of patches, including the Struts patch. (They actually crow about this while blasting Equihax for being slow to apply the patch) [1] 2017-06-30: Equihax patches their struts installs, no longer vulnerable to the struts exploit. They patch the boxes that got popped and almost certainly had webshells installed but notice nothing. [2] 2017-07-29: Equihax discovers they have been compromised by super elite awesome hackers using one webshell for every day of the month (spares in Feb.) 2017-07-30: Equihax evicts the elite hackers and their 30 webshells from their systems 2017-08-01: Equihax CFO sells $1mm stock, US President of Information sells $600k stock, President of Workforce Solutions sells $250k stock [3] 2017-09-05: FireEye registers the Equihax domain name as part of a broader PR damage control move, which Equihax will do everything it can to sabotage 2017-09-07: Equihax mentions that maybe there might have been some sort of hack or something but definitely not a big deal unless you're an American adult with a credit record. 2017-09-08: Equihax offers an opportunity to sign away your right to sue Equihax in exchange for waiting a week and getting yet another year of free credit reporting. (If you don’t already have 3-5 years of free credit reporting by now, are you even using the Internet??) [4] 2017-09-11: FireEye (owner of Mandiant, who did the IR + PR for Equihax) quietly pulls the case study white paper about how FireEye 0day protection technology is keeping Equihax safe from unknown threats and "up to 29 webshells” What this looks like to me is a bunch of web app hackers who used a fresh PoC exploit to mass hack everything they could find. Then, while going through their hacked logs, they discover they have an interesting victim. They turn their attention to it and start working on getting deeper into the environment (this is around the 13th, so a couple days after they popped a shell). I’m guessing that what happened was they went on a bit of a rampage inside the DMZ area popping all the shells they could. Then assembled some Rube Goldberg webshell machine to exfil data from the various databases, including, apparently, legacy databases. I’m calling this mostly a problem with Equihax architecture. This isn’t about a struts bug, this is about a terrible network design that allows random kiddies to scrape the data store clean via a single shell (well, 30, but still). That Equihax was focussing on buying boxes to protect against 0day, and (from stories I’ve read circa 2015) working on ensuring employee phones are compartmented for BYOD. Well, they were clearly spending money out of the security budget. And it wasn’t trivial sums either, FireEye boxes aren’t exactly free. But from the looks of it, the problem wasn’t that they got compromised, the problem was that they couldn’t detect a compromise and prevent it from becoming a breach (seriously: 30 webshells exfiltrating data on 143 million people would have left some pretty hefty “access.log” files). This is not a “bug” issue, it is an architecture issue. You know, if they threw a canary.io tool into that DMZ and configured it to look like a database, they’d have known about the hack during that first week. If they monitored their logs for unusual activity, such as the installation of 30 webshells, and gigabytes of data going the wrong way. If they had an architecture that prevented a compromise of a web server enabling access to sensitive company data. If they had asset management and decommissioned legacy databases, rather than leaving them in the DMZ. There are a lot of things here which would have prevented this compromise from becoming a disastrous breach, but spending money on a bug bounty program or FireEye silver bullet boxes, or mobile device management systems — none of those would, or did, help. The important things are always simple. The simple things are always hard. The easy way is always mined. — Murphy’s Laws of Enterprise Information Security. —gq [0]: https://arstechnica.com/information-technology/2017/09/massive-equifax-hack-reportedly-started-4-months-before-it-was-detected/ [1]: https://threatpost.com/oracle-patches-apache-struts-reminds-users-to-update-equifax-bug/128151/ [2]: a cron job running `find` for new files, AIDE (or Tripwire), would trivially notice the modifications to the file system and alert. [3]: https://www.bloomberg.com/news/articles/2017-09-07/three-equifax-executives-sold-stock-before-revealing-cyber-hack [4]: https://www.cnbc.com/2017/09/08/were-you-affected-by-the-equifax-data-breach-one-click-could-cost-you-your-rights-in-court.html > On 28 Sep 2017, at 02:38, Katie M wrote: > > Having a bug bounty program wouldn't have helped Equifax. Only Equifax could have helped Equifax. The root cause of the problem wasn't that they didn't know about the bug, it was that they face the same patch prioritization risk vs resource balance that all orgs gamble with. They lost that gamble, which is what every breach represents: a lost bet on the tradeoffs. Simply knowing about a bug, via a bug bounty or otherwise, is just that. And knowing is at best half the battle. > > But to return to Dave's assertion about the bug bounty ecosystem itself and what it currently is good for and what it's used for - I have many thoughts. And even more songs. > > "In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific. " > > https://en.m.wikipedia.org/wiki/Mariana_Trench > > Agreed that the bug bounty market has evolved in this *particular stage of it's growth* through its own complex system, the dynamics of which are heavily influenced by factors like: > > 1. the types of organizations who have been adopting these incentives so far (mostly tech companies), > 2. the typical targets (mostly web sites), and > 3. the types of vulnerabilities they tend to use bug bounties to find (mostly low hanging fruit that could have been found using common free tools & techniques). > > Also a factor in this ecosystem is the geolocation and socioeconomic status of the script kiddie bug hunting masses, who, unlike the early professional penetration testers like us, don't have to adapt their techniques to find more interesting, higher quality bugs to continue to be paid relatively small amounts that are worth much more to them in their part of the world. > > That's good for those bug hunters who are in this category. That's actually bad for the evolution of the bug bounty ecosystem, and is accurate in Dave's characterization of what's happening *right now*. > > The upside effect though is that the bug hunter masses can now access a safe marketplace for their skills regardless of those facts of where they are and whether they could ever become a "security consultant". That's generally good, but the *dominant* "species" of bug hunter, as Dave accurately points out about them now will remain relatively unskilled if we don't act with higher-order outcomes in mind. > > It will be like an attempt at brewing beer that gets taken over and soured by undesirable flora before the brewers yeast kicks in and creates the desired effect. And I've been brewing the defensive market for vulnerabilities far too long to watch idly and let the batch sour. > > We ideally want to create an upward trend in bug hunter population skills, as well as move the bug hunter targets themselves, towards more sophisticated bugs. We are not raising the tide, and we are not causing all ships to rise with it. Just by slapping a bug bounty or vuln disclosure program on something, we are missing the point. > > One of the papers that we produced out of the MIT Sloan School visiting scholar systems modeling I worked on will come out sometime this fall (2017) as a chapter in an MIT Press book. That paper looks specifically at bug bounty participant data at a specific point in the development of this economy. Bet you're curious about that supply side snapshot of the bug bounty Mariana Trench. :) Look for that book with our research paper when it's out. > > Bug bounties as they have mostly manifested *right now, at this specific stage in that ecosystem's development,* are a cheap, shiny thing to do, with few exceptions. > > And no, the exceptional bug bounties are not the ones that pay the most more on that later. The presence of a bug bounty program is being currently used by organizations to virtue signal that they take security seriously by paying for web bugs, but often missing or ignoring aggregate threats, and ignoring their internal failing processes to fix bugs. > > It matters very much what's on the inside, versus the superficial, shiny, bug bounty exterior. > > Shiny (but still very insecure): > > https://www.youtube.com/watch?v=93lrosBEW-Q > > The alliterative buzz word "bug bounty", deceptively simple and so very misunderstood, needs to evolve as an accepted concept into the more accurate, more strategic "incentives". > > Straight cash as the only lever for bringing all the (good) bugs to the yard is short-sighted & pollutes the entire defensive reward ocean in this evolution of the vulnerability and exploit markets. Cash is only one lever in this system, and it isn't the most effective one if you're buying bugs for defense purposes, as I've been saying for several years. > > Perhaps if a strapping demigod of security would just repeat this for me, it would replace the econ 101 BS that has plagued the emerging bug bounty market. Of course, I'm sure they'd happily forget where they heard it first. > > https://www.youtube.com/watch?v=79DijItQXMM > > Just kidding, I'll speak for myself, as always: > > https://www.rsaconference.com/writable/presentations/file_upload/ht-t08-the-wolves-of-vuln-street-the-1st-dynamic-systems-model-of-the-0day-market_final.pdf > > Better-than-a-bug-bounty incentives that are much more effective for improving defense may not be direct cash, may not be rewards at all. Instead they might be a much harder deep introspective process, to examine what drives the heart of an organization, what they are doing to defend what's important to them, and whether the security choices, tradeoffs, resources, and budgets are actually working for them. What incentives can they use to tease out real risk, rather than being lazy and trendy and calling it a success. > > No, a bug bounty would not have helped Equifax prevent what happened, and we need to seriously stop the VC-backed tsunami of propaganda that says that it would. That stupid marketing trick employed by at least one of the bug bounty platform vendors should be beneath the critically-thinking readers of this list to entertain in terms of its obvious oversimplicity of a non-trivial problem. > > I'm not even going to address the cyber insurance idea on this, and by now in this long operetta of a post, it should be obvious as to why. > > Bug bounties and cyber insurance are not a remedy for a fundamentally unscalable remediation model that most orgs and governments face today. That's precisely why 94% of the Forbes Global 2000 in 2015 didn't even have a front door to report a vulnerability, let alone a bug bounty, and it's not much better now. > > They struggle to fix the bugs they already know about, and the bottlenecks in that *internal* process are what need work. Putting up a front door to receive bug reports, even without a bug bounty, when there's nothing operationally sufficient inside that org to address what comes through the door, is not the chaos an org needs in the midst of drowning in technical debt. > > It's time to return the heart of the bug bounty ocean to stop the spread of this intellectual and ecosystem-poisoning darkness. I've been staring at the edge of the water, long as I can remember... > > https://www.youtube.com/watch?v=GeIHvhnQbbI > > - k8eM0ana > > https://www.youtube.com/watch?v=Lg_cweoJXyo > > > 🏝️👩‍💻🐞🌋🌺 @k8em0 @lutasecurity @k8eM0ana 🏝️👩‍💻🐞🌋🌺 > > > > On Wed, Sep 27, 2017 at 9:30 AM, Kristian Erik Hermansen wrote: > If Equifax had a public bug bounty program, someone would have reported the Java RCE in March 2017 and picked up $10K or more for it. But no, Equifax did not have a public bug bounty program. Say what you will about the pros and cons of a bug bounty program, especially for financial institutions which "know better than the public how to protect themselves", but at least in this case a known issue would have been well documented much earlier. We should encourage other credit and financial companies to consider public or at the very least private bug bounty programs. It's a mess to operate them, but not patching a known critical web flaw ASAP that allows RCE is precisely the legal definition of negligence. Equifax should pay dearly for it. > > Perhaps it's time to consider federal Cyber Security Insurance laws for such companies which forces them to pay fees to operate on the Internet just like everyone that drives a car on the road? If you crash your car every time you get on the highway, or you damaged 140 million cars while driving, you would lose your license for some time. Why hasn't Equifax lost their license to operate on the internet for some time? How about a 2 year hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar can chime in on why Cyber Security Insurance regulation like that is OR is not the answer. He has been working on that lately... > > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave > > > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave From dave at immunityinc.com Fri Sep 29 15:20:24 2017 From: dave at immunityinc.com (dave aitel) Date: Fri, 29 Sep 2017 11:20:24 -0400 Subject: [Dailydave] Twitter Message-ID: Right now everyone is going on and on about how Russians spent 256K on ads on Twitter to influence the election. Much less understood is how great Twitter ads are for targeting phishing attacks! I wrote this whole article while back here on it. People are genuinely good at phishing now.  The "Fake RedTube subscription " gambit is great fun. And yet, Jar files still fill my inbox, because clearly those still work? Halvar talks in his current keynote about how important it is to understand the iterative nature of the hacking game, but I think willful ignorance is still the name of the game in SV. In any case, has anyone been paid to do a "Twitter Ad Phishing Attack" the way people get paid to do email campaigns? -dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From arrigo at alchemistowl.org Fri Sep 29 15:31:46 2017 From: arrigo at alchemistowl.org (Arrigo Triulzi) Date: Fri, 29 Sep 2017 17:31:46 +0200 Subject: [Dailydave] Equitablefax In-Reply-To: <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> Message-ID: <0C66C1B4-DCE4-48BF-964D-CF52F9C4DA19@alchemistowl.org> On 29 Sep 2017, at 01:12, the grugq wrote: > This is not a “bug” issue, it is an architecture issue. You know, if they threw a canary.io tool into that DMZ and configured it to look like a database, they’d have known about the hack during that first week. If they monitored their logs for unusual activity, such as the installation of 30 webshells, and gigabytes of data going the wrong way. If they had an architecture that prevented a compromise of a web server enabling access to sensitive company data. If they had asset management and decommissioned legacy databases, rather than leaving them in the DMZ. Just in passing: "Equifax is ISO/IEC 27001:2013 certified by a reputable independent third party.”[0]. Asset management is a core part of ISO27001:2013. Cheers, Arrigo [0] https://www.equifax.com/assets/WFS/the_work_number_best_practices_in_data_security.pdf (1st page) From spacerog at spacerogue.net Fri Sep 29 16:15:27 2017 From: spacerog at spacerogue.net (spacerog at spacerogue.net) Date: Fri, 29 Sep 2017 12:15:27 -0400 Subject: [Dailydave] Equitablefax In-Reply-To: <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> Message-ID: <59CE719F.8060903@spacerogue.net> Thank you for this timeline because honestly I haven't been paying that close attention. Based on this it looks like Equifax did actually patch, just not fast enough, and by the time they got around to it the bad guys where already inside. Based on this list the delta from patch release to install was <91 days. Am I reading this correctly? If so then the absolute shit ton of criticism heaped on Equifax for not patching is IMO unwarranted. While a 91 day patch cycle for a huge enterprise isn't great it's a hell of a lot better than most other orgs. Granted your points about bad network design and inability to detect a breach are well made but based on this list the communities criticism on Equifax's inability to patch is inappropriate. - SR the grugq wrote: > I’m not going to address any of the points in the excellent post by Katie but rather put some facts together in a timeline so people can see the Equihax event better. The “if only bug bounty” claptrap is, as Katie points out (much more politely), complete bullshit. > > > Timeline of events: > > > 2017-03-06: Apache announces struts bug > 2017-03-07: PoC exploit released to public > > 2017-03-10: Equihax compromised via struts exploit. Genius hackers use super elite hacker command “whoami” during their sophisticated hacking session. [0] > > 2017-03-13: Equihax genius elite hackers install 30 webshells to allow traversing all the different compromised hosts to pass data out of the company > > 2017-04-xx: Oracle releases quarterly bundle of patches, including the Struts patch. (They actually crow about this while blasting Equihax for being slow to apply the patch) [1] > > 2017-06-30: Equihax patches their struts installs, no longer vulnerable to the struts exploit. They patch the boxes that got popped and almost certainly had webshells installed but notice nothing. [2] > > 2017-07-29: Equihax discovers they have been compromised by super elite awesome hackers using one webshell for every day of the month (spares in Feb.) > 2017-07-30: Equihax evicts the elite hackers and their 30 webshells from their systems > > 2017-08-01: Equihax CFO sells $1mm stock, US President of Information sells $600k stock, President of Workforce Solutions sells $250k stock [3] > > 2017-09-05: FireEye registers the Equihax domain name as part of a broader PR damage control move, which Equihax will do everything it can to sabotage > > 2017-09-07: Equihax mentions that maybe there might have been some sort of hack or something but definitely not a big deal unless you're an American adult with a credit record. > > 2017-09-08: Equihax offers an opportunity to sign away your right to sue Equihax in exchange for waiting a week and getting yet another year of free credit reporting. (If you don’t already have 3-5 years of free credit reporting by now, are you even using the Internet??) [4] > > 2017-09-11: FireEye (owner of Mandiant, who did the IR + PR for Equihax) quietly pulls the case study white paper about how FireEye 0day protection technology is keeping Equihax safe from unknown threats and "up to 29 webshells” > > > What this looks like to me is a bunch of web app hackers who used a fresh PoC exploit to mass hack everything they could find. Then, while going through their hacked logs, they discover they have an interesting victim. They turn their attention to it and start working on getting deeper into the environment (this is around the 13th, so a couple days after they popped a shell). I’m guessing that what happened was they went on a bit of a rampage inside the DMZ area popping all the shells they could. Then assembled some Rube Goldberg webshell machine to exfil data from the various databases, including, apparently, legacy databases. > > I’m calling this mostly a problem with Equihax architecture. This isn’t about a struts bug, this is about a terrible network design that allows random kiddies to scrape the data store clean via a single shell (well, 30, but still). That Equihax was focussing on buying boxes to protect against 0day, and (from stories I’ve read circa 2015) working on ensuring employee phones are compartmented for BYOD. Well, they were clearly spending money out of the security budget. And it wasn’t trivial sums either, FireEye boxes aren’t exactly free. But from the looks of it, the problem wasn’t that they got compromised, the problem was that they couldn’t detect a compromise and prevent it from becoming a breach (seriously: 30 webshells exfiltrating data on 143 million people would have left some pretty hefty “access.log” files). > > This is not a “bug” issue, it is an architecture issue. You know, if they threw a canary.io tool into that DMZ and configured it to look like a database, they’d have known about the hack during that first week. If they monitored their logs for unusual activity, such as the installation of 30 webshells, and gigabytes of data going the wrong way. If they had an architecture that prevented a compromise of a web server enabling access to sensitive company data. If they had asset management and decommissioned legacy databases, rather than leaving them in the DMZ. > > There are a lot of things here which would have prevented this compromise from becoming a disastrous breach, but spending money on a bug bounty program or FireEye silver bullet boxes, or mobile device management systems — none of those would, or did, help. > > > The important things are always simple. The simple things are always hard. The easy way is always mined. > — Murphy’s Laws of Enterprise Information Security. > > > —gq > > [0]: https://arstechnica.com/information-technology/2017/09/massive-equifax-hack-reportedly-started-4-months-before-it-was-detected/ > [1]: https://threatpost.com/oracle-patches-apache-struts-reminds-users-to-update-equifax-bug/128151/ > [2]: a cron job running `find` for new files, AIDE (or Tripwire), would trivially notice the modifications to the file system and alert. > [3]: https://www.bloomberg.com/news/articles/2017-09-07/three-equifax-executives-sold-stock-before-revealing-cyber-hack > [4]: https://www.cnbc.com/2017/09/08/were-you-affected-by-the-equifax-data-breach-one-click-could-cost-you-your-rights-in-court.html > > > >> On 28 Sep 2017, at 02:38, Katie M wrote: >> >> Having a bug bounty program wouldn't have helped Equifax. Only Equifax could have helped Equifax. The root cause of the problem wasn't that they didn't know about the bug, it was that they face the same patch prioritization risk vs resource balance that all orgs gamble with. They lost that gamble, which is what every breach represents: a lost bet on the tradeoffs. Simply knowing about a bug, via a bug bounty or otherwise, is just that. And knowing is at best half the battle. >> >> But to return to Dave's assertion about the bug bounty ecosystem itself and what it currently is good for and what it's used for - I have many thoughts. And even more songs. >> >> "In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific." >> >> https://en.m.wikipedia.org/wiki/Mariana_Trench >> >> Agreed that the bug bounty market has evolved in this *particular stage of it's growth* through its own complex system, the dynamics of which are heavily influenced by factors like: >> >> 1. the types of organizations who have been adopting these incentives so far (mostly tech companies), >> 2. the typical targets (mostly web sites), and >> 3. the types of vulnerabilities they tend to use bug bounties to find (mostly low hanging fruit that could have been found using common free tools& techniques). >> >> Also a factor in this ecosystem is the geolocation and socioeconomic status of the script kiddie bug hunting masses, who, unlike the early professional penetration testers like us, don't have to adapt their techniques to find more interesting, higher quality bugs to continue to be paid relatively small amounts that are worth much more to them in their part of the world. >> >> That's good for those bug hunters who are in this category. That's actually bad for the evolution of the bug bounty ecosystem, and is accurate in Dave's characterization of what's happening *right now*. >> >> The upside effect though is that the bug hunter masses can now access a safe marketplace for their skills regardless of those facts of where they are and whether they could ever become a "security consultant". That's generally good, but the *dominant* "species" of bug hunter, as Dave accurately points out about them now will remain relatively unskilled if we don't act with higher-order outcomes in mind. >> >> It will be like an attempt at brewing beer that gets taken over and soured by undesirable flora before the brewers yeast kicks in and creates the desired effect. And I've been brewing the defensive market for vulnerabilities far too long to watch idly and let the batch sour. >> >> We ideally want to create an upward trend in bug hunter population skills, as well as move the bug hunter targets themselves, towards more sophisticated bugs. We are not raising the tide, and we are not causing all ships to rise with it. Just by slapping a bug bounty or vuln disclosure program on something, we are missing the point. >> >> One of the papers that we produced out of the MIT Sloan School visiting scholar systems modeling I worked on will come out sometime this fall (2017) as a chapter in an MIT Press book. That paper looks specifically at bug bounty participant data at a specific point in the development of this economy. Bet you're curious about that supply side snapshot of the bug bounty Mariana Trench. :) Look for that book with our research paper when it's out. >> >> Bug bounties as they have mostly manifested *right now, at this specific stage in that ecosystem's development,* are a cheap, shiny thing to do, with few exceptions. >> >> And no, the exceptional bug bounties are not the ones that pay the most more on that later. The presence of a bug bounty program is being currently used by organizations to virtue signal that they take security seriously by paying for web bugs, but often missing or ignoring aggregate threats, and ignoring their internal failing processes to fix bugs. >> >> It matters very much what's on the inside, versus the superficial, shiny, bug bounty exterior. >> >> Shiny (but still very insecure): >> >> https://www.youtube.com/watch?v=93lrosBEW-Q >> >> The alliterative buzz word "bug bounty", deceptively simple and so very misunderstood, needs to evolve as an accepted concept into the more accurate, more strategic "incentives". >> >> Straight cash as the only lever for bringing all the (good) bugs to the yard is short-sighted& pollutes the entire defensive reward ocean in this evolution of the vulnerability and exploit markets. Cash is only one lever in this system, and it isn't the most effective one if you're buying bugs for defense purposes, as I've been saying for several years. >> >> Perhaps if a strapping demigod of security would just repeat this for me, it would replace the econ 101 BS that has plagued the emerging bug bounty market. Of course, I'm sure they'd happily forget where they heard it first. >> >> https://www.youtube.com/watch?v=79DijItQXMM >> >> Just kidding, I'll speak for myself, as always: >> >> https://www.rsaconference.com/writable/presentations/file_upload/ht-t08-the-wolves-of-vuln-street-the-1st-dynamic-systems-model-of-the-0day-market_final.pdf >> >> Better-than-a-bug-bounty incentives that are much more effective for improving defense may not be direct cash, may not be rewards at all. Instead they might be a much harder deep introspective process, to examine what drives the heart of an organization, what they are doing to defend what's important to them, and whether the security choices, tradeoffs, resources, and budgets are actually working for them. What incentives can they use to tease out real risk, rather than being lazy and trendy and calling it a success. >> >> No, a bug bounty would not have helped Equifax prevent what happened, and we need to seriously stop the VC-backed tsunami of propaganda that says that it would. That stupid marketing trick employed by at least one of the bug bounty platform vendors should be beneath the critically-thinking readers of this list to entertain in terms of its obvious oversimplicity of a non-trivial problem. >> >> I'm not even going to address the cyber insurance idea on this, and by now in this long operetta of a post, it should be obvious as to why. >> >> Bug bounties and cyber insurance are not a remedy for a fundamentally unscalable remediation model that most orgs and governments face today. That's precisely why 94% of the Forbes Global 2000 in 2015 didn't even have a front door to report a vulnerability, let alone a bug bounty, and it's not much better now. >> >> They struggle to fix the bugs they already know about, and the bottlenecks in that *internal* process are what need work. Putting up a front door to receive bug reports, even without a bug bounty, when there's nothing operationally sufficient inside that org to address what comes through the door, is not the chaos an org needs in the midst of drowning in technical debt. >> >> It's time to return the heart of the bug bounty ocean to stop the spread of this intellectual and ecosystem-poisoning darkness. I've been staring at the edge of the water, long as I can remember... >> >> https://www.youtube.com/watch?v=GeIHvhnQbbI >> >> - k8eM0ana >> >> https://www.youtube.com/watch?v=Lg_cweoJXyo >> >> >> 🏝️👩‍💻🐞🌋🌺 @k8em0 @lutasecurity @k8eM0ana 🏝️👩‍💻🐞🌋🌺 >> >> >> >> On Wed, Sep 27, 2017 at 9:30 AM, Kristian Erik Hermansen wrote: >> If Equifax had a public bug bounty program, someone would have reported the Java RCE in March 2017 and picked up $10K or more for it. But no, Equifax did not have a public bug bounty program. Say what you will about the pros and cons of a bug bounty program, especially for financial institutions which "know better than the public how to protect themselves", but at least in this case a known issue would have been well documented much earlier. We should encourage other credit and financial companies to consider public or at the very least private bug bounty programs. It's a mess to operate them, but not patching a known critical web flaw ASAP that allows RCE is precisely the legal definition of negligence. Equifax should pay dearly for it. >> >> Perhaps it's time to consider federal Cyber Security Insurance laws for such companies which forces them to pay fees to operate on the Internet just like everyone that drives a car on the road? If you crash your car every time you get on the highway, or you damaged 140 million cars while driving, you would lose your license for some time. Why hasn't Equifax lost their license to operate on the internet for some time? How about a 2 year hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar can chime in on why Cyber Security Insurance regulation like that is OR is not the answer. He has been working on that lately... >> >> _______________________________________________ >> Dailydave mailing list >> Dailydave at lists.immunityinc.com >> https://lists.immunityinc.com/mailman/listinfo/dailydave >> >> >> _______________________________________________ >> Dailydave mailing list >> Dailydave at lists.immunityinc.com >> https://lists.immunityinc.com/mailman/listinfo/dailydave > > _______________________________________________ > Dailydave mailing list > Dailydave at lists.immunityinc.com > https://lists.immunityinc.com/mailman/listinfo/dailydave From thegrugq at gmail.com Fri Sep 29 18:08:39 2017 From: thegrugq at gmail.com (the grugq) Date: Sat, 30 Sep 2017 01:08:39 +0700 Subject: [Dailydave] Equitablefax In-Reply-To: <59CE719F.8060903@spacerogue.net> References: <31e57448-8477-e3ea-af72-fd21bab3a074@immunityinc.com> <3BBB5ACB-81E8-416A-A3B5-70B7DD2878E2@gmail.com> <59CE719F.8060903@spacerogue.net> Message-ID: Hey > On 29 Sep 2017, at 23:15, spacerog at spacerogue.net wrote: > > Thank you for this timeline because honestly I haven't been paying that close attention. I wasn’t either since it doesn’t impact me, but I had to research it for this week’s news segment on Risky.Biz ==> https://risky.biz/RB471/ During the research it became clear that the public narrative and the facts were diverging quite a bit. In particular this “failure to patch” story line. Yes, they were slow to patch. However, their upstream provider didn’t even make the patch available until weeks after the compromise had already happened. The time from bug -> to exploit -> to the 1st of 30 web shells was about 3-4 days. There is no major enterprise that is able to consistently deploy patches same day -- as would be necessary in this case. Same day deployment of patches just isn’t possible at scale. Business units have to schedule a service window, test the patch against their applications, allocate resources for deploying and testing the patch in place. Even if the enterprise has automated pretty much all of this, they would still have to wait for a maintenance window to minimise business risk. So, yeah, Equifax failed to apply a patch that wasn’t available at the time a mere four days after the announcement is basically a “dog bites man” story. > > Based on this it looks like Equifax did actually patch, just not fast enough, and by the time they got around to it the bad guys where already inside. Based on this list the delta from patch release to install was <91 days. Am I reading this correctly? Yes, they did actually apply the patch. Eventually. Mid April release, and deployed to live about 5-6 weeks later. Thats not terribly fast nor terribly slow for a large company. Granted, they’d been breached for over a month by the time the patch was released so it is a bit of a moot point, but we can take the “failure to patch” narrative behind the shed and put it to sleep. There are things that concern me more than the delay with deploying the patch. Realistically every organisation has to assume that a system will be compromised. Unless they are built to be resilient after an attacker gains a foothold on the system, they are living in a fantasy world and banking on borrowed time. Compromise is inevitable. Whether the systems and networks are hostile to attackers even after they get a shell is the sign of a mature cyber secure posture. I know of companies that do rolling automated re-installs of systems so that the infrastructure is factory fresh daily. This makes persistences really annoying for attackers who migrate onto a stable system or else do the noisy exploit all over again. Attackers tend to give up in frustration. > > If so then the absolute shit ton of criticism heaped on Equifax for not patching is IMO unwarranted. While a 91 day patch cycle for a huge enterprise isn't great it's a hell of a lot better than most other orgs. Agreed. This is where things get kinda interesting. The breach was discovered on July 29th. For people lacking calendar technology, this is July 2017: July 2017 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 <== not a work day 30 31 Note: the breach was discovered on a Saturday. That, to me, reads like a maintenance window was open and the web servers were being serviced. Possibly because of a marketing campaign or something else that would coincide with an “end of July” time frame. What this says to me is that someone was doing work on a compromised server and uttered that phrase dreaded by hackers everywhere “huh? thats weird. . .” I wouldn't be at all surprised if someone was updating a website with new content for a marketing campaign and stumbled over one of the thirty webshells installed all over the place. Regardless of the circumstances surrounding the breach discovery (my money's on web admin updating the site for a Fall marketing campaign) one thing is seems certain — this breach wasn't uncovered by the cybersecurity team in the regular course of their day to day activities. If it were found by the infosec team as part their regular day job via: threat hunting, log monitoring, deception tools, investigating alerts from tripwire etc etc. ; then the discovery would’ve been earlier, and during business hours. If it were found by the infosec team as part of their regular job it would've been discovered on a weekday during any of the preceding 4 months. Based on this data, I assess that the breach was discovered purely by accident -- sheer chance. The Equifax infosec team failed to notice exfiltration of hundreds of millions of records, and they failed to detect the active operational use of 30 web shells. This is far more damning that any patch related tardiness. Was the breach discovery accidental? Was it by a member of the infosec team, or someone else? What systems are being deployed to detect modification to the web servers, notice exfiltration of gigs of data, and might I suggest asset tracking to decommission legacy databases? Maybe rather that shifting the security paradigm to focus on next gen unknown malware threats[0], Equifax should focus on implementing the bare minimum required to detect known malware and techniques. > > Granted your points about bad network design and inability to detect a breach are well made but based on this list the communities criticism on Equifax's inability to patch is inappropriate. > Correct. —gq [0]: http://www.cnmeonline.com/myresources/fireeye/fireeye-cso-less-secure-than-you-think.pdf > - SR > > the grugq wrote: >> I’m not going to address any of the points in the excellent post by Katie but rather put some facts together in a timeline so people can see the Equihax event better. The “if only bug bounty” claptrap is, as Katie points out (much more politely), complete bullshit. >> >> >> Timeline of events: >> >> >> 2017-03-06: Apache announces struts bug >> 2017-03-07: PoC exploit released to public >> >> 2017-03-10: Equihax compromised via struts exploit. Genius hackers use super elite hacker command “whoami” during their sophisticated hacking session. [0] >> >> 2017-03-13: Equihax genius elite hackers install 30 webshells to allow traversing all the different compromised hosts to pass data out of the company >> >> 2017-04-xx: Oracle releases quarterly bundle of patches, including the Struts patch. (They actually crow about this while blasting Equihax for being slow to apply the patch) [1] >> >> 2017-06-30: Equihax patches their struts installs, no longer vulnerable to the struts exploit. They patch the boxes that got popped and almost certainly had webshells installed but notice nothing. [2] >> >> 2017-07-29: Equihax discovers they have been compromised by super elite awesome hackers using one webshell for every day of the month (spares in Feb.) >> 2017-07-30: Equihax evicts the elite hackers and their 30 webshells from their systems >> >> 2017-08-01: Equihax CFO sells $1mm stock, US President of Information sells $600k stock, President of Workforce Solutions sells $250k stock [3] >> >> 2017-09-05: FireEye registers the Equihax domain name as part of a broader PR damage control move, which Equihax will do everything it can to sabotage >> >> 2017-09-07: Equihax mentions that maybe there might have been some sort of hack or something but definitely not a big deal unless you're an American adult with a credit record. >> >> 2017-09-08: Equihax offers an opportunity to sign away your right to sue Equihax in exchange for waiting a week and getting yet another year of free credit reporting. (If you don’t already have 3-5 years of free credit reporting by now, are you even using the Internet??) [4] >> >> 2017-09-11: FireEye (owner of Mandiant, who did the IR + PR for Equihax) quietly pulls the case study white paper about how FireEye 0day protection technology is keeping Equihax safe from unknown threats and "up to 29 webshells” >> >> >> What this looks like to me is a bunch of web app hackers who used a fresh PoC exploit to mass hack everything they could find. Then, while going through their hacked logs, they discover they have an interesting victim. They turn their attention to it and start working on getting deeper into the environment (this is around the 13th, so a couple days after they popped a shell). I’m guessing that what happened was they went on a bit of a rampage inside the DMZ area popping all the shells they could. Then assembled some Rube Goldberg webshell machine to exfil data from the various databases, including, apparently, legacy databases. >> >> I’m calling this mostly a problem with Equihax architecture. This isn’t about a struts bug, this is about a terrible network design that allows random kiddies to scrape the data store clean via a single shell (well, 30, but still). That Equihax was focussing on buying boxes to protect against 0day, and (from stories I’ve read circa 2015) working on ensuring employee phones are compartmented for BYOD. Well, they were clearly spending money out of the security budget. And it wasn’t trivial sums either, FireEye boxes aren’t exactly free. But from the looks of it, the problem wasn’t that they got compromised, the problem was that they couldn’t detect a compromise and prevent it from becoming a breach (seriously: 30 webshells exfiltrating data on 143 million people would have left some pretty hefty “access.log” files). >> >> This is not a “bug” issue, it is an architecture issue. You know, if they threw a canary.io tool into that DMZ and configured it to look like a database, they’d have known about the hack during that first week. If they monitored their logs for unusual activity, such as the installation of 30 webshells, and gigabytes of data going the wrong way. If they had an architecture that prevented a compromise of a web server enabling access to sensitive company data. If they had asset management and decommissioned legacy databases, rather than leaving them in the DMZ. >> >> There are a lot of things here which would have prevented this compromise from becoming a disastrous breach, but spending money on a bug bounty program or FireEye silver bullet boxes, or mobile device management systems — none of those would, or did, help. >> >> >> The important things are always simple. The simple things are always hard. The easy way is always mined. >> — Murphy’s Laws of Enterprise Information Security. >> >> >> —gq >> >> [0]: https://arstechnica.com/information-technology/2017/09/massive-equifax-hack-reportedly-started-4-months-before-it-was-detected/ >> [1]: https://threatpost.com/oracle-patches-apache-struts-reminds-users-to-update-equifax-bug/128151/ >> [2]: a cron job running `find` for new files, AIDE (or Tripwire), would trivially notice the modifications to the file system and alert. >> [3]: https://www.bloomberg.com/news/articles/2017-09-07/three-equifax-executives-sold-stock-before-revealing-cyber-hack >> [4]: https://www.cnbc.com/2017/09/08/were-you-affected-by-the-equifax-data-breach-one-click-could-cost-you-your-rights-in-court.html >> >> >> >>> On 28 Sep 2017, at 02:38, Katie M wrote: >>> >>> Having a bug bounty program wouldn't have helped Equifax. Only Equifax could have helped Equifax. The root cause of the problem wasn't that they didn't know about the bug, it was that they face the same patch prioritization risk vs resource balance that all orgs gamble with. They lost that gamble, which is what every breach represents: a lost bet on the tradeoffs. Simply knowing about a bug, via a bug bounty or otherwise, is just that. And knowing is at best half the battle. >>> >>> But to return to Dave's assertion about the bug bounty ecosystem itself and what it currently is good for and what it's used for - I have many thoughts. And even more songs. >>> >>> "In a sense, the entire bug bounty market is a breeding ground for a species that can collect extremely low impact web vulnerabilities into a life sustaining nutrient cycle, like the crabs on volcanic plumes in the depths of the Pacific." >>> >>> https://en.m.wikipedia.org/wiki/Mariana_Trench >>> >>> Agreed that the bug bounty market has evolved in this *particular stage of it's growth* through its own complex system, the dynamics of which are heavily influenced by factors like: >>> >>> 1. the types of organizations who have been adopting these incentives so far (mostly tech companies), >>> 2. the typical targets (mostly web sites), and >>> 3. the types of vulnerabilities they tend to use bug bounties to find (mostly low hanging fruit that could have been found using common free tools& techniques). >>> >>> Also a factor in this ecosystem is the geolocation and socioeconomic status of the script kiddie bug hunting masses, who, unlike the early professional penetration testers like us, don't have to adapt their techniques to find more interesting, higher quality bugs to continue to be paid relatively small amounts that are worth much more to them in their part of the world. >>> >>> That's good for those bug hunters who are in this category. That's actually bad for the evolution of the bug bounty ecosystem, and is accurate in Dave's characterization of what's happening *right now*. >>> >>> The upside effect though is that the bug hunter masses can now access a safe marketplace for their skills regardless of those facts of where they are and whether they could ever become a "security consultant". That's generally good, but the *dominant* "species" of bug hunter, as Dave accurately points out about them now will remain relatively unskilled if we don't act with higher-order outcomes in mind. >>> >>> It will be like an attempt at brewing beer that gets taken over and soured by undesirable flora before the brewers yeast kicks in and creates the desired effect. And I've been brewing the defensive market for vulnerabilities far too long to watch idly and let the batch sour. >>> >>> We ideally want to create an upward trend in bug hunter population skills, as well as move the bug hunter targets themselves, towards more sophisticated bugs. We are not raising the tide, and we are not causing all ships to rise with it. Just by slapping a bug bounty or vuln disclosure program on something, we are missing the point. >>> >>> One of the papers that we produced out of the MIT Sloan School visiting scholar systems modeling I worked on will come out sometime this fall (2017) as a chapter in an MIT Press book. That paper looks specifically at bug bounty participant data at a specific point in the development of this economy. Bet you're curious about that supply side snapshot of the bug bounty Mariana Trench. :) Look for that book with our research paper when it's out. >>> >>> Bug bounties as they have mostly manifested *right now, at this specific stage in that ecosystem's development,* are a cheap, shiny thing to do, with few exceptions. >>> >>> And no, the exceptional bug bounties are not the ones that pay the most more on that later. The presence of a bug bounty program is being currently used by organizations to virtue signal that they take security seriously by paying for web bugs, but often missing or ignoring aggregate threats, and ignoring their internal failing processes to fix bugs. >>> >>> It matters very much what's on the inside, versus the superficial, shiny, bug bounty exterior. >>> >>> Shiny (but still very insecure): >>> >>> https://www.youtube.com/watch?v=93lrosBEW-Q >>> >>> The alliterative buzz word "bug bounty", deceptively simple and so very misunderstood, needs to evolve as an accepted concept into the more accurate, more strategic "incentives". >>> >>> Straight cash as the only lever for bringing all the (good) bugs to the yard is short-sighted& pollutes the entire defensive reward ocean in this evolution of the vulnerability and exploit markets. Cash is only one lever in this system, and it isn't the most effective one if you're buying bugs for defense purposes, as I've been saying for several years. >>> >>> Perhaps if a strapping demigod of security would just repeat this for me, it would replace the econ 101 BS that has plagued the emerging bug bounty market. Of course, I'm sure they'd happily forget where they heard it first. >>> >>> https://www.youtube.com/watch?v=79DijItQXMM >>> >>> Just kidding, I'll speak for myself, as always: >>> >>> https://www.rsaconference.com/writable/presentations/file_upload/ht-t08-the-wolves-of-vuln-street-the-1st-dynamic-systems-model-of-the-0day-market_final.pdf >>> >>> Better-than-a-bug-bounty incentives that are much more effective for improving defense may not be direct cash, may not be rewards at all. Instead they might be a much harder deep introspective process, to examine what drives the heart of an organization, what they are doing to defend what's important to them, and whether the security choices, tradeoffs, resources, and budgets are actually working for them. What incentives can they use to tease out real risk, rather than being lazy and trendy and calling it a success. >>> >>> No, a bug bounty would not have helped Equifax prevent what happened, and we need to seriously stop the VC-backed tsunami of propaganda that says that it would. That stupid marketing trick employed by at least one of the bug bounty platform vendors should be beneath the critically-thinking readers of this list to entertain in terms of its obvious oversimplicity of a non-trivial problem. >>> >>> I'm not even going to address the cyber insurance idea on this, and by now in this long operetta of a post, it should be obvious as to why. >>> >>> Bug bounties and cyber insurance are not a remedy for a fundamentally unscalable remediation model that most orgs and governments face today. That's precisely why 94% of the Forbes Global 2000 in 2015 didn't even have a front door to report a vulnerability, let alone a bug bounty, and it's not much better now. >>> >>> They struggle to fix the bugs they already know about, and the bottlenecks in that *internal* process are what need work. Putting up a front door to receive bug reports, even without a bug bounty, when there's nothing operationally sufficient inside that org to address what comes through the door, is not the chaos an org needs in the midst of drowning in technical debt. >>> >>> It's time to return the heart of the bug bounty ocean to stop the spread of this intellectual and ecosystem-poisoning darkness. I've been staring at the edge of the water, long as I can remember... >>> >>> https://www.youtube.com/watch?v=GeIHvhnQbbI >>> >>> - k8eM0ana >>> >>> https://www.youtube.com/watch?v=Lg_cweoJXyo >>> >>> >>> 🏝️👩‍💻🐞🌋🌺 @k8em0 @lutasecurity @k8eM0ana 🏝️👩‍💻🐞🌋🌺 >>> >>> >>> >>> On Wed, Sep 27, 2017 at 9:30 AM, Kristian Erik Hermansen wrote: >>> If Equifax had a public bug bounty program, someone would have reported the Java RCE in March 2017 and picked up $10K or more for it. But no, Equifax did not have a public bug bounty program. Say what you will about the pros and cons of a bug bounty program, especially for financial institutions which "know better than the public how to protect themselves", but at least in this case a known issue would have been well documented much earlier. We should encourage other credit and financial companies to consider public or at the very least private bug bounty programs. It's a mess to operate them, but not patching a known critical web flaw ASAP that allows RCE is precisely the legal definition of negligence. Equifax should pay dearly for it. >>> >>> Perhaps it's time to consider federal Cyber Security Insurance laws for such companies which forces them to pay fees to operate on the Internet just like everyone that drives a car on the road? If you crash your car every time you get on the highway, or you damaged 140 million cars while driving, you would lose your license for some time. Why hasn't Equifax lost their license to operate on the internet for some time? How about a 2 year hiatus on their annual revenue to punish them? Just a thought. Maybe Halvar can chime in on why Cyber Security Insurance regulation like that is OR is not the answer. He has been working on that lately... >>> >>> _______________________________________________ >>> Dailydave mailing list >>> Dailydave at lists.immunityinc.com >>> https://lists.immunityinc.com/mailman/listinfo/dailydave >>> >>> >>> _______________________________________________ >>> Dailydave mailing list >>> Dailydave at lists.immunityinc.com >>> https://lists.immunityinc.com/mailman/listinfo/dailydave >> >> _______________________________________________ >> Dailydave mailing list >> Dailydave at lists.immunityinc.com >> https://lists.immunityinc.com/mailman/listinfo/dailydave