[sf-lug] traffic shaping, ...: Re: [on-list] site up, http[s] down: Re: Wierd problems trying to access linuxmafia.com

[Michael Paoli]

> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: Re: [sf-lug] [on-list] site up, http[s] down: Re: Wierd  
> problems trying to access linuxmafia.com
> Date: Tue, 11 Dec 2018 02:44:40 -0800

> Longer-term, I need to find some more-automated way of throttling, as
> playing whack-a-mole for the world's sociopaths doesn't appeal.

There are traffic shaping packages for at least most major Linux
distributions.  These can be quite advantageous:
o avoids the whack-a-mole (most notably manual per-offender
   configurations being added)
o can be applied to all services and traffic, e.g. not only incoming,
   but also covering any overzealous bandwidth consumption by clients,
   on host itself, etc.
o depending upon the software/package(s), may also be quite dynamic,
   e.g. automagically adjusting based upon congestion (e.g. ping times to
   gateway on remote end of DSL connection), so it can work reasonably
   gracefully/dynamicly, even when shared with other systems using same
   link, but where those other systems don't have any traffic shaping or
   the like, and don't pass through systems that implement traffic
   shaping on the network.

Note also that traffic shaping can be deployed as a pass-through for,
e.g. a home network, giving benefit - and relative fairness to all -
though traffic shaping can also be rather to quite useful even if
deployed on as little as one individual host.

Note however, that such automated traffic shaping doesn't, at least by
itself, especially/particularly punish the "bad actors" - instead, at
least by default, they're mostly treated like any other bandwidth
consumer.

As for Apache, I do rather also like the idea of a robots.txt with some
essentially "honeypot" do not go here stuff (and especially that's not
otherwise linked to or at all or found by a regular web crawl), and
use that (via logs, or CGI, or whatever) to have such offending client
IPs automagically throttled (at least until unused for some while) down
to the bandwidth of sucking chilled molasses through a cocktail straw.

Caveat: I've not yet actually worked with / used such
software/throttling (yet?) (other than fail2ban - which outright
blocks temporarily), but I've seen/read some bits about it on
The Internet.