[Dailydave] Late Friday thoughts on the Kevin Mandia RSAC keynote.

[Haroon Meer]

Hiya(s)

On Tue, Mar 25, 2014 at 4:27 AM, Dan Guido <dguido at gmail.com> wrote:
> This argument by Bejtlich really makes me angry and it's representative of
> the way that we seem to deal with cyber attribution out in the open. There
> are a lot of gaps in the APT1 report that make those not "in the know"
> question the results. In my opinion, that kind of questioning is deserved.

Fwiw.. Marco Slaviero (marco at thinkst.com) wrote up a snippet on our
blog (and for our ThinkstScapes service) at the time highlighting some
of those gaps and listed data that could have been used to help
support the argument

http://blog.thinkst.com/2013/02/thinkstscapes-2013-ah1-on-china-report.html

-snip-
The Mandiant APT1 report that was released a week ago has been causing
some consternation, which makes it a ripe topic for our ThinkstScapes
service. This morning, we issued an ad-hoc update to our customers
containing our views of the APT1 report. In short, the data is
interesting, but does not conclusively point to Unit 61938. There are
too many open questions to justify the finger pointing.

Take, for example, the markers released for the APT1 group. The report
does not contain sufficient data to replicate the grouping of
attackers bearing those markers into a single cohesive unit. By
Mandiant's own admission the presence of a single marker is
insufficient to tag an attacker as APT1, but thresholds are not
provided for the number of markers required. In the end, it appears as
if the classification boils down to an analyst's opinion, metrics are
absent the public report. The entire report is founded on the notion
that APT1 exists and is definable; should this not be the case, the
report's raison d'être evaporates. Corroboration is needed in the form
of convincing evidence.

In addition, the conclusion that blames hacks supposedly originating
from an area the size of Los Angeles on a military unit's building in
same area is weak. In this regard, the press' use of the word
"neighbourhood" to describe Pudong is misleading. Today's ad-hoc
update examines these and other issues in greater detail, and extracts
the bits we believe matter for corporates.

To be clear, we do not defend China or absolve it from hacking or
espionage; we have little doubt that it conducts such operations as,
presumably, do the US and other sufficiently resourced nations. Permit
me to repeat this: we are not saying the Chinese government does not
hack the US. Our concern is with this specific report; it is the first
concrete public attribution of ongoing espionage against the US, and,
if the report sets the standard for attribution, future events will be
highly muddled as competing hypotheses all meet the low standard set
out in Mandiant’s APT1 report. Unfortunately it seems that contrary
opinions are being subjected to a level of diatribe usually reserved
for arguments of faith, not facts.

Part of the problem is that there is appears to be an information
differential, in which a number of folks with apparent non-public
information are saying "it's totally legitimate", while those without
the information are saying "this does not follow". Mandiant can help
the APT1 debate by releasing more data to reduce this differential,
specifically:

- Is there further evidence that ties the subset of observed IP ranges
to the Unit 61398 Pudong building apart from a WHOIS record? (Note
that the fibre infrastructure was provided by a different company than
the listed owner of the IP ranges.)
- The number of attacks that would be classified as APT1, except for
the fact that their sink address (e.g. HTRAN receiver) was NOT in
Shanghai. What is the method for arriving at this conclusion? Phrased
differently, how much weighting does a Shanghai IP address have in the
APT1 cluster?
- A timestamped listing of known APT1 connections with their
associated IP addresses, which would show us the activity levels of
APT1.
- Metrics showing how many of the APT1 markers are shared with other
groups under observation, and to what degree? (i.e. what is the
overlap of domains, address blocks and malware hashes across the
various groups?)
- How many more profiles of APT1 members were discovered, and what
confidence does Mandiant hold in them? It seems strange that such a
large group with such poor opsec has not leaked many more profiles.
- What is the mapping between APT1-associated domain names and IP
addresses at the time of observation?
- What confidence level is assigned to the APT1⇿Unit 61938 link claim?
- By what reasoning does Mandiant eliminate an explanation for the
attack pattern that argues for small non-government teams operating in
a loosely connected fashion rather than a cohesive and directed group
of operators with a common approach?

These debates are important going forward. Putting aside patriotism
and pride, there are important questions which remain to be asked
about the attribution of online attacks, and the danger in jumping to
conclusions is that, when the shoe is on the other foot, equally weak
claims are possible by an opponent. Hopefully any forthcoming
additional data will settle these questions and we can get back to our
regularly scheduled navel-gazing.
-snip-

/mh
-- 
Haroon Meer | Thinkst Applied Research
http://thinkst.com/pgp/haroon.txt
Tel: +27 83 786 6637