[conspire] Apache2 bandwidth limiting: fixing 1105 Altschul's connectivity
sean.channel at pacbell.net
Wed Jan 11 12:59:39 PST 2012
just a scant $0.02:
I found it very painless and rewarding to switch from Apache to
Lighttpd, YMMV, and this page includes a link to do the same thing with
Apache, though I'm sort of promoting Lighttpd here, FWIW.
OTOH, I once briefly had the inclination to crawl linuxmafia's knowledge
base myself just to have a local copy (an idea I quickliy got over), so
perhaps a prefab tarball to actually allow such a download might eschew
such crawlers if that be their intent. It is a neat little KB, after all.
OTMA (One Too Many Acronyms),
On 01/11/2012 12:08 AM, Rick Moen wrote:
> CABAL attendees will be happy to hear that we've chased down and fixed
> two major causes for connectivity problems Chez Moen.
> 1. Obsolete DNS nameserver IP.
> The WAPs we have around here, and also /etc/resolv.conf on my server,
> have tended to include two IP addresses of DNS nameservers that Raw
> Bandwidth Communications makes accessible to its customers (such as us)
> for full recursive service:
> That's in addition to my server, IP 18.104.22.168, which is likewise a
> full recursive nameserver. The problem is, it appears that
> 22.214.171.124 has been retired. Depending on round-robin
> implementation, maybe something like half of all DNS queries from
> machines with wireless DHCP leases were going to a nonexistent
> nameserver. The other half were going to my nameserver, which
> was configured to think that 126.96.36.199 was a valid forwarding IP,
> so some of _its_ traffic was likewise going to nowhere.
> This has all been fixed.
> 2. My instance of the Apache2 HTTPd was wide-open to abuse by bandwidth
> hogs. The past two days, almost 100% of incoming bandwidth was in use,
> and logfile analysis revealed that substantively everything was being
> grabbed by many thousands of rapidfire requests to Apache from a single
> IP address in Scotland. Someone there had fired up a mirroring script
> to spider through my entire site and grab every available file without
> exception: photos of my vegetable garden, tarballs of obsolete SSH ports,
> pipermail archives, everything without exception.
> Our immediate tactical measure was to blacklist that IP in
> <Directory />
> Options Indexes FollowSymLinks
> AllowOverride None
> Deny from 188.8.131.52
> However, if one guy today in Scotland with a terabyte array finds it
> easier to wget my entire site than to target just what he wants, that
> pretty much guarantees that there are a million other idiots just like
> him in various other parts of the world.
> So, I looked at throttling options. In a follow-up post, I might write
> about other ways of doing this (including the 'tc' / Traffic Control
> software for doing system-wide throttling at the kernel level), but my
> immediate solution is mod_bw, which is an update/successor to the old
> Apache 1.3 mod_bandwidth module, ported to Apache2.
> Two pretty good articles:
> The first of those two also covers the _other_ currently popular
> solution, an Apache2 module called mod_cband ('Apache2 bandwidth quota
> and throttling module').
> Steps taken here:
> apt-get install libapache2-mod-bw #Fetch Debian package.
> a2enmod bw #Enable the module within the installed Apache instance.
> Add these lines to the port-80 and port443 VirtualHosts stanzas in
> BandwidthModule On
> ForceBandwidthModule On
> Bandwidth all 250000
> LargeFileLimit .mp3 1 30000
> LargeFileLimit .gz 5 30000
> LargeFileLimit .gif 1 30000
> LargeFileLimit .png 1 30000
> LargeFileLimit .zip 1 30000
> LargeFileLimit .pdf 1 30000
> LargeFileLimit .exe 1 30000
> LargeFileLimit .mov 1 30000
> LargeFileLimit .jpeg 1 30000
> /etc/init.d/apache2 restart
> To explain the lines added to the Apache conffiles:
> The 'Bandwidth all 250000' line limits total Apache2 bandwidth to
> The various LargeFileLimit [foo] 1 30000' throttles down to 30kB/s any
> request for a file with the matching filenamed extension that is over 1
> Lots more things can be done, but I'm starting with the low-hanging
> fruit. Module documentation:
> conspire mailing list
> conspire at linuxmafia.com
More information about the conspire