[sf-lug] Risk Sense says Open source vulnerabilities doubled in 2019

[Rick Moen]

Quoting Bobbie Sellers (bliss-sf4ever at dslextreme.com):

> Rick if you read this send me a note about
> your best hours
> and I will try to get the recommended browser.

Calls are welcome after 9am and before midnight.  650-283-7902 cellular.


> Hi LUGers especially Rick,
> 
>     What do you think about this?

Even before reading it (reading just the description), I can see that it
suffers some depressingly familiar (and very lame) problems.

The authors grabbed some thousands of CVE reports from a year's worth of
entries in the National Vulnerability Database and just counted them,
noted which codebases are available under an open source licence, and
asserted that this is a proxy measure for security danger, claiming it
shows that 'commercial' [sic] software is more secure, tra la.

At times, an important part of my job has involved reading and
interpreting CVEs.  


Point 1, CVE categories.  

CVEs document _very_ diverse sort of code problems, and many of them are
just not security vulnerabilities in the usual sense people usually
imagine, but rather in entirely different senses.  Let's take XSS
(cross-site scripting) vulnerabilities, for example.  An XSS is a flaw
in a public-facing Internet site's code that permits hostile parties to
bounce queries off it to attack some other, third-party site.  Point is,
the flaw doesn't permit attacking the hosted code at all, but rather
using the site as a weapon.

For example, I originally built the Linuxmafia.com Knowledgebase
(linuxmafia.com/kb/) using a nifty little bit of demonstration Perl
code called PerlHoo, a means of emulating the old Yahoo Directory
(for those who don't remember it, see https://curlie.org/en for 
its modern offspring) in about a hundred lines of Perl CGI code.
PerlHoo allowed me to gather documents and links by conceptual category, 
and offer them to the public and myself, to more easily find my
squirrel-nut gathering of information.

In September 2015, a security researcher kindly let me know[1] that
PerlHoo had a fatal XSS flaw.  Upon looking closely at the Perl, I
immediately saw how this problem arose:  The author had made his code
parse the URL data passed to it from public networks without bothering 
to validate the input data.  Fixing this would require totally rewriting
its parsing code to not simply trust data from the public.  I needed to
fix this problem quickly, so I asked SF Perl Mongers if anyone could
give me a hand -- and got zero response for two weeks.  At that point,
I found it most prudent to convert the Linuxmafia.com Knowledgebase
to flat HTML rather than it being dynamically generated by Perl any
more.  This sidestepped the fatal composition flaw in PerlHoo, which
thus has never been fixed.

But, point is, the XSS flaw was not any kind of security threat to
linuxmafia.com, nor to its users, nor directly to the public in any way
-- because that's not how XSS problems work.

There are other categories of CVEs that for sundry other reasons are not
security threats to the site or its users, too.  For example, a large
number of them are like this:  "A flaw has been found in version n.nn
of the BIND9 DNS nameserver where carefully crafted queries can cause 
the individual BIND9 instance to terminate and respawn, constituting a
denial of service [DoS] attack.'  

That's undesirable for the site running BIND9, but is it a security hole
in the sense of permitting site compromise or attacking its users, etc.?
No, not at all.  It just means that someone remote on the Internet can
pea-shoot some of the many running instance of BIND9, whereupon the
master instance immediately spawns off new instances to replace them.
DNS responsiveness would almost certainly not even be noticeably
impaired.  But having this be possible is undesirable, so the admin will
want to plug the hole when he/she gets around to it.

Some other CVEs will say '[something] can cause a stack overflow in
Blahblah that might make possible an attack on the code and possibly an
escalation of privilege.'  OK, so as of today, someone can poke at the
code and maybe cause a wild pointer, and maybe some day someone might 
figure out how to use that to do actual harm.  That's like DoS examples,
but possibly more serious in the long term.  Maybe.

I could go on.  There are _lots_ of problems of radically different
types reported in CVEs.  They're all lumped together as CVEs, but
they're very far from equally serious, and many of them are not threats
to the running site at all.


Point 2, Configuration.

Over my years of reading CVEs for a living as part of a DevOps
organisation -- and in particular when I was senior sysadmin for a
division of my company doing online credit-card processing -- about 90% 
of the time, I would read a CVE and say 'Nope, doesn't apply to our site
configuration' -- and document that fact for our PCIDSS vulnerability
scnaning service firm.  (PCIDSS is a required regime for any firm doing
computer handling of credit card data, where you have to be tested and
recertified as compliant every six months.)  A large number of the CVEs 
were flaws (often far-fetched ones) in a codebase's optional components
and functions that had never been enabled at our site, and in many cases 
were components and functions almost never used by anybody.

The rule of thumb with public-facing server code is:  'If you aren't
going to need something, then switch it off.'  This is so ingrained that
it's mostly followed by default even in halfassed Linux distributions.
Code that either isn't present or isn't ever executed is code whose
security flaws, real or theoretical, don't matter.

But the unstated implication of 'studies' like RiskSense's is that all
code is equally security-flaw-exposed, despite the reality that much of
it is either not present, or not enabled, or actively disabled, in
real-world systems.  Which brings us to:


Point 3, Open Source Is Not One Big Hairball.

Back a couple of decades again, one of these goofball security firms did
a 'study' showing that MS-Windows XP was orders of magnitude more
security-safe than was SUSE Linux Professional Edition, by counting
CVEs.

SUSE Linux Professional Edition came, at that point, on something like
eight DVDs with about 4000 packages including multiple office suites and 
server and workstation applications and daemons of every kind.  WinXP
was on one disc and came with... WordPad.  And, of course, one typically
installed a SUSE system with a small percentage of all possible and
provided software.

So, remind me again, why comparing a year's CVEs for all eight DVDs
worth of every conceivable application bundled along with the boxed set
of SUSE Linux Professional Editon with the CVEs for just a bare
Microsoft OS makes sense?

Protip:  It's drivel.

RiskSense didn't literally do that, but they did almost the same thing,
and actually something a bit worse:  They conceptually treat every known
open source codebase one big hairball -- despite the fact that it's
ludicrous to even imagine a server running Jenkins, MySQL, Tomcat,
Magneto, Kubernetes, Elastisearch, and JBoss.


Point 4, Exploitability.

Numerous _real_ studies of security flaws have shown that, as a general
rule, security flaws in open source codebases (the ones that are widely
used) tend to get fixes rolled out long, long before a method is found
to exploit them to do harm.  The CVEs have preventative intent, and that
intent gets effectively implemented by the maintainers, such that actual
security exposure never happens.  And, point is, those studies also show
that discovered flaws in proprietary codebases tend, on average, to 
linger as exposed, _exploitable_ flaws with known exploits, for
significant periods of time.  Those are thus, in the real world, much,
much more serious.

This picture is, of course, totally missed if you just count CVEs.


I could go on, but I'm already at 160 lines of text, and that's enough. 
And, actually, I can see enough of the tediously familiar and damning
flaws pretty much always present in these 'studies' that I'm not going
to bother downloading RiskSense's.  This is all just same-old, same-old,
and I'm not going to spend more time on it.

Why do firms like RiskSense publish such drivel?  Sutton's Law.
(Wille Sutton was a bank-robber.  A reporter asked him why he robbed
banks, and he replied 'That's where the money is.'  Sutton's Law,
then, is:  'Go where the money is.)  Publishing this category of drivel
is a reliable means of getting publicity.  E.g., it's gotten me to spend
hundreds of lines discussing (indiretly) RiskSense, and I'd otherwise
not notice them at all.

And now, I'm done with RiskSense, and lunch beckons.


[1] https://www.openbugbounty.org/search/?search=linuxmafia.com

-- 
Cheers,                            "Rand Paul being patient zero for a Senate 
Rick Moen                          viral outbreak is a sign of a writers' room 
rick at linuxmafia.com                dropping too much acid, late in the season."
McQ! (4x80)                                        -- @owillis (Oliver Willis)