[conspire] xz exploit and backdoor

[Les Faby]

Rick, absolutely right. Features you do not implement do not have bugs.

On Thu, Apr 4, 2024, 9:43 PM Rick Moen <rick at linuxmafia.com> wrote:

> Quoting Ron / BCLUG (admin at bclug.ca):
>
> > It cannot be overstated how devious this hack was!
>
> Yes.  The technical details are impressive and twisty.
> It was very stealthy in execution and well hidden in design -- really a
> masterclass in how to do something covertly in a way well hidden against
> scrutiny.
>
> > It cannot be overstated how devastating this hack would have been.
>
> Yes.  Almost all the world's Linux servers (the ones reachable in public
> via SSH) getting coverly rooted, even for days, would have been
> catastrophic.  And we really did come pretty close.
>
> Also, it's still not completely clear what else the successful bits
> allowed the unknown attackers into, or what other similar projects they
> or others have or had.
>
> > It cannot be overstated how lucky we all are that Adreas just happened
> > to be testing at the right time, noticed delays of *under* ½ second on
> > failed logins, and despite it being outside his area of expertise,
> > decided to look into it further.
>
> Andres has given an appearance on a security podcast, BTW.
> https://risky.biz/RB743/  He's hilariously on-type.
>
> > We'll have the discussion, then carry on as we are currently.
>
> Probably.  But:  I know this is talking big, as I haven't exactly made
> tracks on this larger project, but while thinking about next-gen system
> software for my server, I've often thought I need do better at applying
> the lessons of my Ops / InfoSec background.  And one of my recurring
> frustrations with commodity distro packaged software (especially but not
> limited to binary-package distros) is security-critical, public-facing
> software being overfeatured by selection, by configuration, and by
> choice of compilation options.
>
> Let me back into what I'm thinking about.
>
> Some years ago at $FIRM, an important part of my job was to be one of
> two senior sysadmins for a semi-autonomous division functioning as a
> "merchant bank".  In the USA tech field, that is a term of art meaning
> "credit card transaction processor for e-commerce".  Other companies'
> public-facing online "stores" could back-end into $FIRM's "merchant
> bank" to accept and process the card charges.
>
> All merchant banks must periodically (every 6 mos., if memory serves)
> pass "PCI testing", the Payment Card Industry assocation's Payment Card
> Industry Data Security Standard (PCIDSS) security-quality testing.  And
> I was the lucky sod who had to shepherd our farm of CentOS-based servers
> through that periodic process.
>
> PCIDSS testing was, in a way, a little lame.  A chosen PCIDSS testing
> vendor would, twice a year, port-scan your IP netblocks with some
> secret-sauce perl script, making guesses based on fingerprinting of what
> public-facing software comprised your public attack surface, and then
> you as the customer (and I as the fix-it guy at the customer) would get
> a report saying "Hey, our probe claims you're running Foo daemon version
> XX.YY.Z rev. A, so kindly prove to us that your implementation isn't
> vulnerable to the following CVEs...."  And each of those would be a CVE
> that _might_ apply to Foo XX.YY.Z rev. A.  (Likewise for other
> software.)  My job would be to prove to the vendor that our site
> _lacked_ that possible vulnerability, either because we'd upgraded to
> CentOS update this-or-that, or because we'd specifically disabled that
> functionality entirely in /etc/foo/*.conf .
>
> Sometimes, the vendor's claim would be one of many bizarre types of
> false positive.  Then, my job was to explain why their script had been a
> prat and was mistaken.  Either way, my explanations (rarely, after
> rolling out some upgrades or conffile mods) would be sanity-tested by
> some functionary, and (at the end) accepted as a "pass" for 6 months.
>
> It worked well enough, and we never had security incidents -- to our
> knowledge, at least.
>
> Getting back to distro software choice and packaging:  The "every
> possible use-case must be supported out of the box" syndrome is
> dispiritingly ubiquitous.  Some stuff is compiled into a daemon binary
> even though almost nobody will use it, just because someone might.
> Why is Apache httpd almost always the default, rather than Lighttpd,
> nginx, sthttpd, or Hiawatha?  Because its big dumb software that does
> everything, that everyone's used to.
>
> $FIRM ran Apache httpd, but many PCIDSS complaints got finessed by
> sending them our conffiles where various buggy features were
> specifically disabled.
>
> Network Time Foundation's NTP Project software, used by damned near all
> server-oriented distros by default, is likewise big dumb software that
> does everything, that everyone's used to.  OpenBSD Foundation's OpenNTPd
> was founded with the specific aim of avoiding NTP Project's featuritis,
> and has succeeded (with only minor cavils).  Red hat's Chrony is a
> similar story.
>
> I'm sure you get my point:  You get a smaller attack surface by choosing
> security-relevant software with minimal unneeded servers coded in, and if
> possible further cut back on unwanted functions in the runtime config.
>
> And, last, there are times you want to rebuild a distro package from
> source to use different compile options.  For daemons, this can be a
> reasonable local admin burden, depending.  For example, imagine I were
> looking at the (newest) Debian Portable OpenSSH package's dependencies,
> and said "Eh?  Needs liblzma.so.5?  I don't need that.  I don't _want_
> that."  I'd check, and find that the package got compiled to require it,
> in order to be able to report service-readiness to libsystemd -- which
> I would not have any interest in, therefore compiling a local OpenSSH
> package with that compile-time option disabled would do the trick.
>
> But Portable OpenSSH isn't the only choice, either.  There are respected
> SSHDs that are _much_ smaller/lighter with a smaller attack surface,
> like Dropbear and wolfSSH.  "But they're less well known!"  Do I care?
> Hell no.
>
> Anyway, it seems to me that substantial improvement can be made on a
> local system through active system administration, picking wiser choices
> of software packages, paring down configurations, and in some cases
> compiling locally to reduce runtime dependencies.
>
> All of which is only a _little_ helpful against insider threats such as
> "Jia Tan", whoever that was.  But OTOH shying away from big dumb
> software and featuritis can help defang even the Jia Tans.
>
> > There's been some interesting analysis of the timestamps of JiaTan's git
> > commits showing odd patterns.
> >
> > Jia claims to be from California (GMT -8) but timestamps tended to be
> > GMT +8 - same as Singapore.
> >
> > Many commits were GMT +2 or +3 (DST dependent), pointing to Eastern
> Europe.
>
> APT29 aka Cozy Bear is a popular guess.
> https://en.wikipedia.org/wiki/Cozy_Bear
>
> The list of possiblities is very broad, though.
>
> --
> Cheers,             "Are you sure it’s that simple?  After all my time
> here,
> Rick Moen           I’ve yet to see any problem, however complicated,
> which
> rick at linuxmafia.com when you looked at it the right way, didn’t become
> still
> McQ! (4x80)         more complicated."     -- Poul Anderson, in "Call Me
> Joe"
>
> _______________________________________________
> conspire mailing list
> conspire at linuxmafia.com
> http://linuxmafia.com/mailman/listinfo/conspire
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://linuxmafia.com/pipermail/conspire/attachments/20240405/3772325a/attachment-0001.html>