[sf-lug] South African Linux sites experiencing Ransomware attacks.

[Rick Moen]

Quoting Bobbie Sellers (bliss-sf4ever at dslextreme.com):

> Hi LUGers.
> 
> This is the article ..../A new form of ransomware called Lilocked
> (or Lilu) has been targeting
> Linux servers and encrypting website files, asking for a Bitcoin
> ransom to get these files back./
> /Link to the full article:
> - https://mybroadband.co.za/news/security/319373-south-african-websites-hit-by-new-lilocked-ransomware.html

As usual, this is hot air.  My reaction to a separate (and earlier) but
related news story, this one from Ziff-Davis ('ZDNet').  The story you
are quoting differs primarily in throwing in an idle speculation that
the means of attack is a recent (now fixed) bug in the Exim SMTP daemon
-- but the author doesn't actually have any information about this (and,
further irony, the author quotes an equally unsupported speculation
about that from ZDnet, about which more below).


Quoting Steve Litt via Tech (tech at golug.org):

> Ransomeware just for Linux:
>
> https://www.zdnet.com/article/thousands-of-servers-infected-with-new-lilocked-
lilu-ransomware/

It's time to play one of my favourite games:  'Dissect Self-Promoting
Humbug from a Security/Antimalware Company.'

If you've seen my play this game before, you probably remember that,
IMO, there are only two (related) interesting questions about malware:

1. How does the code _get executed_?

    The way the Lilocked gang breaches servers and encrypts their
    content is currently unknown.

2. If there is _privilege escalation_, how does that happen?

    A thread on a Russian-speaking forum puts forward the theory that
    crooks might be targeting systems running outdated Exim (email)
    software. It also mentions that the ransomware managed to get root
    access to servers by unknown means.

So, basically, 'We dunno, magic happens, I guess'.

If you leave public-facing service running for a long time with
exploitable security bugs, eventually bad things happen.  This should
not be a surprise to anyone.  As I like to say, if you do that, you
actually have bigger problems.

   Servers hit by this ransomware [blah blah blah]

Here's a question, class.  You're about to run a public server that will
house data that you care about.  What is the _first_, most-important
security measure you need to make sure you are doing?

Anyone, anyone?  Bueller?

It's tested, known-restorable, off-system backups.

If you aren't doing that, you have bigger problems.

I you _are_ doing that, then downtime to do a system rebuild after
security compromise is very annoying but not much more than that -- and
system operators who even need to consider paying 'ransom' for their
data contents are implicitly making the humiliating admission that they
failed to do the one single most important task.

   Thousands of web servers have been infected [blah blah]

First of all, the 'infection' metaphor is highly misleading.  Since the
avenue of entry is unstated and ditto the means of privilege escalation,
the most parsimonious assumption is that, like most security 'attacks',
this one leverages sysadmin failure to attend to the basics, i.e.,
involves metaphorical kicking in of a rotten door -- or metaphorical
walking through an unlocked and open door.

Second, who exactly is saying 'thousands'?  What is the Ziff-Davis
author's source for that, and based on whose data?  What specific
thousands are those?  Author Catalin Cimpanu doesn't say.

Last, IT press articles about malware are pretty much uniformly copied
and pasted from press releases / white papers sent to them by
security/antimalware companies, which churn them out by the bushel
basket.  It's good form to credit _which_ of these firms you're cadging
material from -- but Cimpanu doesn't.

Well, it _is_ Ziff-Davis.


> I'm thinking of running a daemon or cron spawned program that
> periodically checks for the existing of any files with a .lilocked
> extension, and throwing up tremendous warnings if found.

You'd be better served by attending to security fundamentals, including
making sure you have complete and correct period offsystem backups --
and a recovery plan in the case of a failed or compromised server.




[follow-up post]

> Better still, do both. Here's my shellscript, called once per hour, to
> warn and log any presence of a .lilocked file:

Well, you do you -- but this is a cron job checking for the afteraffects
of one highly specialised type of utter system disaster, among countless
other minor variants of utter system disaster, has already occurred.
Are you going to do one-off post-disaster checking for every other
possible security meltdown?

Also, as a reminder, there are two maintained and quite-good existing
tools to check for artefacts of security compromise, chkrootkit and
rkhunter.  (A 'rootkit', whose artefacts the tools search for, is a
canned set of replacements for sysadmin administrative utilities altered
to selectively not notice and report, i.e., to hide, a root-level
intrusion already accomplished by other means.  The two rootkit-hunting
tools also check for other related signs of trouble.  It should be noted
in passing that running them on a live system that you suspect is
root-compromised is slightly problematic, since a root-compromised
system should not be trusted as it often has been altered to lie to the
sysadmin in particular ways, including possibly the reports of
rootkit-hunting tools.  However, when run in a suitably paranoid
fashion, their assurances are fairly decent on balance, albeit one must
read results carefully on account of false-positive problems.  A truly
paranoid sysadmin might reboot a suspect system to live-distro media and
run known-valid off-system instance of chkrootkit and/or rkhunter, with
the cost of system downtime and inability to examine the runtime process
state.  But this would be a useful crosscheck against normal system runs
of chkrootkit/rkhunter.)



> I've been making off-system and offsite backups only twice a year,
> clearly not enough. Most of the reason is that it's still murder to
> burn data, especially encrypted data, to blu-ray.

That's IMO expensive and slow.  The best use-case for Blu-Ray, IMO,
isn't backup but rather archival storage.  See, about the distinction:
http://linuxmafia.com/faq/Admin/backup-strategy.html

One alternative is to do backup to a number of USB-connectable external
hard drives, used in rotational fashion.


> I think what I'll do is buy between 2 and 4 256GB thumb drives for
> about $20/apiece. On each one,

My gut-reaction is that that's slow media compared to external hard
drives, and much wwwwmore expensive per gig.


> I'll lay down an ext4 filesystem, use truncate to make one huge file,
> and turn that file into a LUKS loop-mountable filesystem.

D00d, personally I am a huge proponent of KISS where backup/restore is
concerned, and extra layers of abstraction between the OS and the data
is IMO borrowing trouble.

> These will be partial backups,  and won't contain my /boot tree, which
> has millions of old kernels and initramfses. I'll use Kevin Korb's
> methodologies with hardlinks to have the best of both worlds:
> incremental without huge amounts of redundancy.

IMO, some effort distinguishing trees that merit backup from the many
that do not is extremely worthwhile.  Example scheme that deliberately
omits from backup most of the system, e.g., all files that would upon
rebuild be provided by distro software packages:
http://linuxmafia.com/faq/Admin/linuxmafia.com-backup.html

> The one thing I'm still missing is out-of-region backup.

Here in the San Francisco Bay Area, if a wildfire burns out my entire
house _plus_ the outbuildings on my 1/3 acre lot, that means it's
probably burning out all many of the dwellings of stockbrokers and
lawyers who've become my neighbours[1], which means it's a black-swan
regional catastrophe that even vast fortunes could not avert, and my
family has much bigger problems than data loss.  Likewise a meteor hit
or the ISS falling on Chez Moen.  (A great earthquake I've mitigated
against well enough, and our 49.28 metre elevation and 8 km distance
from San Francisco Bay is ample protection against tsuanamis and storm
surges, I'm not near any watercourse likely to overflow, and the local
geography means I don't have a landslide problem.  Burglary is mitigated
by local measures I won't get into here.)

If you want out-of-region backup, make friends with someone out-of-state
who wants to solve the same problem, and ship each other backup media,
say, every quarter.  And personally, I'd make friends with someone I
can have ultimate trust in, and vice-versa, and send unencrypted
storage, because KISS.  To mitigate somewhat the need for ultimate
trust, you can ship media in lockboxes, but that adds weight and
shipping expense.

[1] The point is that there's protective value in living near the
well-heeled, as, e.g., the fire protection is then very above average.
Mom and Dad bought Chez Moen newly built in 1956 for $22,500, comprising
a $20k Veterans Administration GI loan plus $2,500 cash on the
barrel-head, and in my childhood was a neighbourhood of plumbers and
clerks in addition to people like Dad who was then an airline co-pilot.
Rolling forward six decades, today, a rock thrown in any direction has
about 50% chance of beaning a daytrader or CXO, and Zillow says my 1,510
ft^2 suburban house is worth so absurdly much, it would poleaxe my
parents.

The subtle change was lost on me until after (1) I'd be outside digging
in the vegetable garden with my aged mother-in-law siting nearby, and
passers-by would address m-i-l but never me, and act confused when the
man with the shovel _spoke_ and even used fluent English.  (2) A woman
walking her dog past a similar scenario commented 'I appreciate how you
do your own gardening and landscaping' -- to which I said 'Thanks', but
for a moment thought 'WTF?  Who _doesn't_ do his/her own gardening and
landscaping?'  And then it dawned on me:  Actually, everyone else is
now hiring out.  And then I started noticing the au pairs, the
housekeepers, the professional dog walkers, and precious little Biff and
Buffy being picked up from the schoolbus by SUVs idling in front of my house
because God forbid they should walk two blocks.  Sheesh.  Lazy much?

But this means there's a lot of money working to prevent, say, a local
replay of the Oakland Hills Fire
(https://en.wikipedia.org/wiki/Oakland_firestorm_of_1991).





> Here's the actual vulnerability that permits the ransomware to run
> as root and encrypt the files:
> <http://exim.org/static/doc/security/CVE-2019-15846.txt>
> It affects only mail servers using Exim that also have TLS enabled.
> Exim comes with TLS disabled
> by default but many distros' packages configure it as enabled.

According to my best understanding, that doesn't permit anything to run
as root.

The Exim MTA launches initially as root in order to bind to port 25/tcp
(the SMTP port).  But that instance does _not_ accept public traffic
under any circumstances (as that would be ridiculously bad security).  
Instead, the initial Exim instance spawns off 'service' instances
running as a non-privileged user.  (on Debian, this is nonprivileged
user 'Debian-exim'.)

I see a lot of claims on the Internet, including bug-reporter Qualys, 
that CVE-2019-15846 permits remote attackers to execute arbitrary code
as root via a trailing backslash, but I can't see how this would be
possible given Exim's careful dropping of privilege before accepting
incoming SMTP connections.  (Maybe I'm missing something.  Dunno.)

Anyway, FWIW, Debian's response was typical of distros:  a bug-fix
release four days after the CVE was announced. 

Or, of course, your SMTP server can simply not support incoming TLS
connections, in which case there's nothing to attack.