[conspire] (forw) Re: Security breach at multiple Federal agencies via SolarWinds

[Rick Moen]

----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----

Date: Thu, 17 Dec 2020 17:21:41 -0800
From: Rick Moen <rick at linuxmafia.com>
To: BerkeleyLUG <berkeleylug at googlegroups.com>
Subject: Re: Security breach at multiple Federal agencies via SolarWinds
Organization: If you lived here, you'd be $HOME already.

Some comments about links Aaron posted, composing these in-passim as I
read the articles in question:

Quoting goossbears (acohen36 at gmail.com):


> - Gizmodo's 'Feds Still Trying to Determine How Screwed They Are After 
> Massive SolarWinds Hack', 
> https://gizmodo.com/feds-still-trying-to-determine-how-screwed-they-are-aft-1845888076

  A cyberattack that began by targeting an IT firm used by numerous
  federal government agencies, Fortune 500 companies, and other high-value
  targets is shaping up to be a historic event.

It appears so, partly because of the wide scope of SolarWinds, Inc.
users of the Orion Platform software who got shot in the foot, and
partly because of the sensitive nature of what Orion Platform does, 
i.e., network monitoring, which I gather requires that the software 
run with high privilege and access to networks and remote systems.

As with other articles I've seen, here, Gizmodo reporter Tom McKay
doesn't appear to properly grapple with just how grossly negligent
SolarWinds, Inc. was revealed to have been.  Anyone old enough will
remember how much flak Ford Motors took over supposedly exploding gas
tanks on Pintos (which turned out to be a greatly exaggerated story, but
that's not the point).  Somehow, a major software company publishes
software that, when intalled at client sites, destroys client companys'
IT security, and all everyone can talk about is how freaked out the
customers are, i.e., hardly anyone's pointing an appropriate degree 
of blame at the negligent party.

This astonishes me.  It's like the botulism scandal over cans of Bon Vivant 
vichyssoise, except with all the coverage being over the suffering of
victims and nobody saying anything about, looking at, or thinking about
the manufacturer -- and their attitude was just 'Well, sure, some cans
of soup poison and kill people, but, hey, stuff happens.'

  Those responsible built a backdoor into Orion, an IT management
  software produced by SolarWinds, possibly by breaking into Microsoft
  email accounts and other systems, according to the Wall Street Journal.
  [link] They then used it to contaminate software updates provided by the
  company with malware in March and June 2020. In addition to U.S.
  government agencies, the attackers also hit security firm FireEye;
  senior vice president and chief technical officer, Charles Carmakal,
  told Bloomberg [link] the firm was subsequently able to trace the intrusion
  back to SolarWinds before it notified authorities.

The WSJ surmise was as follows:  "How the hackers gained access to
SolarWinds systems to introduce the malicious code is still uncertain.
The company said that its Microsoft email accounts had been compromised
and that this access may have been used to glean more data from the
company’s Office productivity tools."

That's nothing like a complete picture on the vital "How did compromise
and privilege escalation occur?" question (not to mention obviously
involving speculation), but suggests a fatal laxness at SolarWinds, Inc.  
Companies (and projects, like Debian) that take code-signing seriously
treat custody of the signing keys (and the production code repo) like
the crown jewels.  There should have been _no_ path to get to them via
things like phishing and other dumb probes against Microsoft Exchange /
Microsoft Office.



> - CNN's Politics' article 'Massive hack of US government launches search 
> for answers as Russia named top suspect', 
> https://www.cnn.com/2020/12/16/politics/us-government-agencies-hack-uncertainty/index.html

This is what you get when you have an IT story covered by competent
political reports instead of competent IT reporters:  You get an article
about _who_, when the question mainly of interest is _how_.

> - CNN's Business article 'Why the US government hack is literally keeping 
> security experts awake at night', 
> https://www.cnn.com/2020/12/16/tech/solarwinds-orion-hack-explained/index.html 

Despite URL, this generally meritorious article likewise didn't explain
the Orion hack, except in the sense of saying "It was done via
SolarWinds's Orion Platform software, where the bad guys had full 
control for over six months (BBC, below, says eight months) and
piggybacked their code to infiltrate customers via SolarWinds's signed
code for its retail software products."

> - The BBC News' Tech article 'SolarWinds Orion: More US government
> agencies hacked', https://www.bbc.com/news/technology-55318815 

Reasonable layman's overview, adds some deserved praise for the response
of cybersecurity firm FireEyes.

> - The BBC News' Tech article 'SolarWinds: Why the Sunburst hack is so 
> serious', https://www.bbc.com/news/technology-55321643

This is a piece by a different BBC reporter who makes quite a lot of
basic errors, which I probably shouldn't waste time listing.  I have
some sympathy for IT/'tech' reporters, always expected to provide good
coverage on impossible deadlines.  This article is mostly about
ramifications, and shows that the reporter called up a bunch of contacts
in relevant fields, and relied heavily on what they said.  (That is not
a bad thing.  I'm just saying it's that type of article.)

Bruce Schneier is reporting on the reporting.
https://www.schneier.com/blog/archives/2020/12/another-massive-russian-hack-of-us-government-networks.html
https://www.schneier.com/blog/archives/2020/12/how-the-solarwinds-hackers-bypassed-duo-multi-factor-authentication.html
https://www.schneier.com/blog/archives/2020/12/more-on-the-solarwinds-breach.html

Schneier makes the point -- obvious to me, but maybe not to most readers
-- that just replacing the trojaned software installed via SolarWinds's
oopsie with a non-trojaned version is not _nearly_ good enough:  that 
any/all of SolarWinds's ten of thousands of affected customers are going
to have to do _major_ work to rule out and correct persistent, ongoing
penetration of their networks and systems.  Just removing or upgrading 
the trojaned Orion Platform software is closing the barn door after the
horse escaped.

SolarWinds, Inc. retroactively deleted the public list of its customers
from its public-facing Web site shortly after the scandal hit, but,
well, too late, fellahs!
https://web.archive.org/web/20201214143046/https://www.solarwinds.com/company/customers

(As noted in the comments on Schneier's blog, Internet Archive is not a
foolproof repository of things that moneyed interests want to make go
away, in that they sometimes take down their mirror copies in response
to pressure.)

Another commenter says that the customer list captured by Internet
Archive / Wayback Machine is a "very small subset of the SolarWinds
clients".


Found via the second of the above-cited Schneier links:
https://arstechnica.com/information-technology/2020/12/solarwinds-hackers-have-a-clever-way-to-bypass-multi-factor-authentication/

  ADVANCED PERSISTENT THREAT —
  SolarWinds hackers have a clever way to bypass multi-factor
  authentication
  Hackers who hit SolarWinds compromised a think tank three separate
  times.

  DAN GOODIN - 12/14/2020

Oh?  _That's_ interesting.

Article cites researchers at a security firm named Volexity who say
they'd encountered late last year / eartly this year the same attackers
who compromised SolarWinds, and noticed that they'd used a clever trick
to neuter multi-factor authentication on the attacked network of a
think-tank organisation.  However, this hack required that the intruders
first possess 'Administrator' access on the target MS-Windows network,
i.e., root privilege.  The target company used a two-factor
authentication system published by Duo Security.  Having Administrator
access, they simply stole a Duo Security token file from the target
company's Outlook Web Access server, used that to generate a special
'cookie' file, and then figuratively waved that around like Doctor
Who's psychic paper to fake out authentication servers in a 'Oh, you
don't need the second factor in addition to my stolen username and
password' sense.

So, this isn't actually very surprising -- but it does underline the
point that, if subject to a comprehensive breach, the recovering
organisation must asssume that _all_ of the existing security
infrastructure is untrustworthy, including everyone's passwords and
system-internal security tokens.


It should be noted that both Volexity and FireEye are being careful
about the attacker's identity -- in distinction to many in the press who
are saying APT29 / Cosy Bear.  Volexity merely calls the attacker Dark
Halo, and FireEye calls them UNC2452 - both names invented for the
purpose.

https://www.fireeye.com/blog/threat-research/2020/12/evasive-attacker-leverages-solarwinds-supply-chain-compromises-with-sunburst-backdoor.html
https://www.volexity.com/blog/2020/12/14/dark-halo-leverages-solarwinds-compromise-to-breach-organizations/



----- End forwarded message -----