[conspire] Another DNS trick: Making domains go away

[Rick Moen]

One of the invited talks at the 2005 LISA conference was "Internet
Counter-Intelligence: Offense and Defense", by Lance Cottrell, head of
Anonymizer, Inc.  In part, he detailed what one might term just how low 
Internet-using companies tend to go, in their manipulation of their 
customers through technical means.  Jim Dennis, who was attending with
me, also found the talk very worthwhile and technically valuable, but 
mentioned as we left that his biggest reaction was one of irritation --
not at Cottrell, but rather at Internet businesses.

Cottrell went into a fair amount of detail about how a surprisingly high
percentage of firms deploy user-tracking and IP-geolocation services to
ensure their ability to set differential pricing.  In part, that means
offering lower prices in some places than others:  He gave the example
of a real-life purchase of some expensive computer gear that the same
firm offered at _twice_ the price to people browsing from European IP 
addresses than to those coming from American IPs.

Mostly, though, the firms work really hard to ensure that competitive,
attractive pricing is offered _only_ to newer customers, and that they
gradually (but invisibly) jack up greatly the prices offered to you
once you're an established customer.  He commented:  Forget about
rewarding customer loyalty.  The opposite is the general rule.

There's some danger that you, reading this, might think the syndrome
occurs with companies some _other_ people deal with, in part because I
can't remember many of the numerou everyday company names he cited:  I
remember that Amazon.com and Barnes & Noble were among them, and many
others -- and we're not talking trivial price differences, either.

To Cottrell's credit, he didn't present this talk primarily as a sale
pitch for Anonymizer's proxying service, that among other things 
completely hides your network location.  But because of his experience
with that project, his analysis was fully credible.


On the drive back from LISA, aspects of Cottrell's talk (of which there
were others, such as IP-blocking and forging for political and business
reasons, information leakage, and the uses of those data in competitive
business intelligence efforts) kept colliding in my head with a
longstanding project of mine:

Many, many years ago, I started wanting for various reasons to want to
make particular hostnames, domains, and IP addresses evaporate from my
experience of the Internet.  Among the first Internet entities to annoy
me to that extent was Doubleclick.net (now owned by Microsoft
Corporation).  Around the 1980s, something they were doing annoyed me
enough that they were the first crash-test dummy for my "make things go
away" project.  At first, this was in /etc/hosts and similar
static-lookup files, which on account of some obvious drawbacks didn't
work too well:

127.0.0.2  ad.doubleclick.net
127.0.0.3  missed-me.doubleclick.net
127.0.0.4  another-one.doubleclick.net
127.0.0.5  tom.doubleclick.net
127.0.0.6  dick.doubleclick.net
127.0.0.7  harry.doubleclick.net

This sort of thing made a huge number of banner ads (etc.) go away, by
mapping their hostnames to my loopback network interface, but the supply
of new hostnames was endless, plus it helped only that one machine (that
had the /etc/hosts file), plus it didn't catch traffic fetched by IP
address.

The more-comprehensive solution (including catching references by IP) was
something elaborate like Junkbuster, but I wanted to see if there was an
easy 95% solution.


Pretty soon, I remembered:  "Oh, wait!  I run a DNS nameserver."  Which
provided an easy way to make all of *.doubleclick.net go bye-bye, in 
one easy step, inside /etc/bind/named.conf.local:

  //doubleclick.net must die.  Internet advertisers (DoubleClick, Inc.).
  zone "doubleclick.net" {
          type master;
          allow-query { any; };
          file "/etc/bind/advertisers.zone";
  };

Even if you never create /etc/bind/advertisers.zone at all, it still
works because you've said "Pay no attention to any other nameserver's 
information about Doubleclick.net hostnames:  I know all."  But here's 
the advertisers.zone file I created, anyway:

  $TTL 86400
  ;Generic make-advertisers-go-away zonefile.  Put YOUR IP address in the 
  ;A line, and YOUR nameserver name in the NS line.
  @	IN	SOA	ns1.linuxmafia.COM.		rick.deirdre.NET. (
  			2005112300		; serial
  			7200			; refresh 3 hours
  			3600			; retry 1 hour
  			2419200			; expire 1000 hours
  			86400 			; minimum 24 hours
  			)		
  ;
  		IN	NS	ns1.linuxmafia.com.
  *		IN	A	198.144.195.186

The wildcard "A" line resolves *.doubleclick.net to my hostname.  You 
could of course map it to somewhere else creative, or whatever you wish. 
The point is, _no_ call to those hostnames to pick up a cookie, a banner 
ad, a "Web bug" (or "beacon") 1x1 pixel GIF, or anything else is going
to get out to those bloodsuckers.  Effectively, they get summarily and
completely vanished.


Over the years since the '80s, occasionally one of those firms would
come to my attention with some "cute" variation on Doubleclick.net's 
advertising-blitz-and-spy-on-users business model, and get dropped into
the same oubliette.  "yimg.com" (Yahoo Images) was an early addition.

You learn some of the euphemisms as you go along:  Some of the firms
sell "market intelligence", "research", "Web metrics", "dynamic personal
messages", "measured response", "targeted promotions", and so on.

Now, I certainly can't spend significant time on this stuff:  I don't
have that time to waste.  However, I'm occasionally willing to devote a
little on a high-bang-for-the-buck basis, especially if I can "bottle"
what progress I've made, and offer it up to others.  Thus this posting:


As I've found domains used entirely or almost entirely for the more
scummy sorts of Internet-spying and advertising activities, I've 
been declaring my nameserver "authoritative" for them in exactly the way
shown above for doubleclick.net -- which means they get included in my 
prototype BIND9 example files, that I publish for the public's benefit.

If you want to see the whole set, in a format that can be dropped
effortlessly into BIND8/BIND9 nameserver configurations, download
http://linuxmafia.com/pub/linux/network/bind9-examples-linuxmafia.tar.gz

Following is the comments lines (only) from the "make domains go away"
section of /etc/bind/named.conf.local :



//Domains Killed Dirt Cheap:
//(advertising and similar domains mapped via local DNS to nowhere at all)

//"2o7.com" issues traffic-tracking cookies (run by Omniture, Inc.).

//"360i.com" are Internet advertisers (360 Integrated, run by 360i LLC.).

//"3dstats.com" are Web-bug advertisers (ImagineNET Company)

//"ad-up.com" are Internet advertisers and sell e-mail address lists
// (Ad-Up Corporation).

//"adbot.com" were big Internet advertisers, but went broke and are now
// probably harmless:  Owned by Cameron Gregory, Web/Java developer.

//"adjuggler.com" are Internet advertisers (Thruport Technologies, Inc.).

//"adknowledge.com" are Web-bug advertisers (Adknowledge, Inc.).

//"adlegend.com" are Web-bug advertisers (run by TruEffect, Inc.).

//"adrevolver.com" are Web-bug advertisers (run by BlueLithium, Inc.).

//"adriver.ru" are Internet advertisers 

//"adserver.com" are Internet advertisers (Fastclick, Inc.).

//"adsmart.com" are Internet advertisers (run by Web holding company CMGI).

//"adtech.de" are Internet advertisers (ADTECH AG).

//"alexa.com" are Internet advertisers (Alexa Internet, Inc.)

//"advertising.com" are Web-bug advertisers (Advertising.com, Inc.).

//"apmebf.com" are Web-bug advertisers (part of ValueClick, Inc.).

//"atdmt.com" issue traffic-tracking cookies (part of Atlas, 
// division of aQuantive, Inc.).

//"atlas.cz" are Czech-language Internet advertisers (ATLAS.CZ, a.s.).

//"atwola.com" are Web-bug advertisers (part of AOL, Inc.).

//"belnk.com" are Internet advertisers (BehaviorLink, in Claria's Vista 
//division).

//"bfast.com" are Internet advertisers (part of ValueClick, Inc.)

//"bizrate.com" are Internet advertisers (Shopzilla, Inc.).

//"blm.net" are Internet advertisers (BrowserMedia, LLC).

//"bluelithium.com" are Internet advertisers (BlueLithium, Inc.).

//"bluestreak.com" are Internet advertisers (Bluestreak, Inc.).

//"bravenet.com" issue traffic-tracking cookies (Bravenet Web Services Inc.).

//"burstnet.com" are Internet advertisers (Burst Media LLC).

//"burstbeacon.com" are Internet advertisers (Burst Media LLC).

//"casalemedia.com" are Internet advertisers (Casale Media, Inc.).

//"centrport.net" are Internet advertisers (CentrPort, Inc.).

//"checkm8.com" are Internet advertisers (Checkm8 Technologies, Inc.)

//"clickability.com" are Internet advertisers (Clickability, Inc.)

//"clicktracks.com" issue traffic-tracking cookies (ClickTracks Analytics Inc.)

//"clickz.com" issue traffic-tracking cookies (Incisive Interactive Marketing,
//LLC).

//"cnnaudience.com" are Internet advertisers (Turner Broadcasting System, Inc.)

//"contextweb.com" are Internet advertisers (ContextWeb, Inc.).

//"coremetrics.com" are Internet advertisers (Coremetrics, Inc.).

//"criticalmass.com" are Internet advertisers (Critical Mass, part of
// Omnicom Group, Inc.).

//"did-it.com" are Internet advertisers (Did-it.com, LLC).

//"dogpile.com" are Internet advertisers run by infospace.com (InfoSpace, Inc.).

//"domainsponsor.com" are Internet advertisers (Oversee.net)

//doubleclick.net must die.  Internet advertisers (DoubleClick, Inc.).

//"esomniture.com" are Web-bug publishers (Omniture, Inc.).

//"falkag.net" are Internet advertisers (Falk eSolutions AG).

//"fastclick.com" are Internet advertisers (Fastclick, Inc.).

//"fastclick.net" are Internet advertisers (Fastclick, Inc.).

//"focalink.com" are Internet advertisers (Focalink Communications).

//"gemius.pl" issue traffic-tracking cookies (Gemius S.A.)

//"gureport.co.uk" issue traffic-tracking cookies (Guardian Newspapers, Ltd.)

//"hitbox.com" are Web-bug publishers (WebSideStory, Inc.).

//"hitslink.com" are Web-bug publishers (Net Applications, Inc.).

//"hitsprocessor.com" are Web-bug publishers (Net Applications, Inc.).

//"humanclick.com" are Internet advertisers (LivePerson, Inc.).

//"imrworldwide.com" are Internet advertisers (NetRatings, Inc., in 
//collaboration with AC Nielsen and Nielsen Media Research). 

//"indextools.com" are Internet advertisers (IndexTools, Inc.)

//"information.com" are Internet advertisers (Oversee.net)

//"infospace.com" serve up ads from ads.infospace.com (InfoSpace, Inc.).

//"insightexpressai.com" are Internet advertisers (InsightExpress, LLC).

//"itadnetwork.co.uk" are Internet advertisers (Net Communities Limited)

//"kanoodle.com" are Internet advertisers (Kanoodle.com, Inc.).

//"linkexchange.com" come across as sleazemeisters.  Advertisers dealing
// in questionable page-rank deals (Microsoft Corporation).

//"liveperson.com" are Internet advertisers (LivePerson. Inc.).

//"liveperson.net" are Internet advertisers (LivePerson. Inc.).

//"maxserving.com" are Internet advertisers (Ask Jeeves, Inc.).

//"medialand.ru" are Internet advertisers (Medialand.Ru, Ltd.).

//"mediaplex.com" are Internet advertisers (Mediaplex, Inc.).

//"myaffiliateprogram.com" are Internet advertisers (KowaBunga Technologies,
// part of Think Partnership Inc. / CGI Holding Corporation)

//"nbcupromotes.com" are Internet advertisers (NBC/Universal Promotions).

//"netservice.de" are German-language Internet advertisers.

//"nozonedata.com" are Internet advertisers (NoZone, Inc.).

//"nytdigital.com" are Internet advertisers (The New York Times Company)

//"omniture.com" are Web-bug advertisers (Omniture, Inc., the people 
// who run 2o7.com).

//"onestat.com" are a Dutch traffic-tracking company.

//"optimost.com" are Internet advertisers (Optimost LLC)

//"poindextersystems.com" are Internet advertisers (Poindexter Systems, Inc.)

//"pointroll.com" are Web-bug advertisers (run by Gannett Company, Inc.).

//"preferences.com" appear to serve up ads from ads.preferences.com 
// (RHCDirect LLC).

//"questionmarket.com" issue traffic-tracking cookies (Dynamic Logic).

//"realmedia.com" are Internet advertisers (24/7 Real Media, Inc.)

//"remoteapproach.com" collect spy-on-users data from Acrobat 7.x 
// and later for the benefit of Adobe Systems, Inc.

//"revenue.net" are Internet advertisers (Oversee.net)

//"revsci.net" are Internet advertisers (Revenue Science, Inc.)

//"riddler.com" advertise from various subdomains (Riddler LLC).

//"rightmedia.com" are Web-bug advertisers (the people who run yieldmanager.com,
// Right Media, LLC).

//"ru4.com" are Web-bug advertisers (Pointdexter Systems).

//"sageanalyst.net" are Internet advertisers (sbasoft, Inc./SageMetrics Corp.).

//"seeq.com" are Internet advertisers (BrowserMedia, LLC).

//"serving-sys.com" are Internet advertisers (Ilissos).

//"sitestat.com" issues traffic-tracking cookies (Nedstat BV)

//"smartadserver.com" are Internet advertisers (auFeminin.com SA).

//"specificclick.com" are Web-bug advertisers (SpecificCLICK).

//"specificclick.net" are Web-bug advertisers (SpecificCLICK).

//"spylog.com" issue traffic-tracking cookies (OOO Spylog)

//"statcounter.com" issues traffic-tracking cookies (Aodhan Cullen of Dublin).

//"tacoda.com" are Internet advertisers (TACODA Systems, Inc.).

//"techbuyer.com" are Web-bug advertisers (YesDirect, Inc.)

//"techtarget.com" are Internet advertisers (TechTarget, Inc.).

//"trafficmp.com" are Internet advertisers (Vendare Group, Inc.)

//"tribalfusion.com" are Internet advertisers (Tribal Fusion, Inc.)

//"trueffect.com" are Web-bug advertisers (TruEffect, Inc. the people 
// who run adlegend.com).

//"ultramercial.com" are Internet advertisers (Ultramercial, LLC).

//"valueclick.com" serve up ads from oz.valueclick.com (ValueClick, Inc.).

//"valueclick.net" are Internet advertisers (ValueClick, Inc.).

//"webads.nl" are Internet advertisers (Webads Europe)

//"webstats4u.com" are Web-bug publishers (Web Measurement Servicews B.V.).

//"webtrendslive.com" are Web-bug advertisers (WebTrends, Inc.).

//"wiredminds.com" are Internet advertisers (WiredMinds, Inc.).

//"wiredminds.de" are Internet advertisers (WiredMinds AG).

//"yadro.ru" are Internet advertisers 

//"yieldmanager.com" are Web-bug advertisers (run by RightMedia, Inc).

//yimg.com must die, too; same reasons as for Doubleclick (Yahoo, Inc.).

//"zedo.com" are Internet advertisers (ZEDO, Inc.).


(Those characterisations aren't very exact, so don't take them as
gospel.  I looked at each domain's known activity just long enough to 
class them as "Should be made to go away", and wrote a quick guess at
what each one seemed to be mostly about.)

One neat thing is:  _Any_ DNS-client machine that uses your nameserver 
will be under your umbrella.  Their processes, like yours, will have
those same bloodsucker domains globally mapped to oblivion.


This can be a two-edged sword, given contrary-minded local users:  My
mother-in-law Cheryl, who lives with us and initially had her
workstations set up to use my nameserver, kept coming to me and
complaining that she was being blocked from reaching desirable content 
by my proxy.

I explained I had no proxy.  She continued to complain, and was certain
it was my fault.

It occurred to me that I had mapped *.doubleclick.net hostnames to
nowhere -- but I stressed that there was _nothing_ but undesirable
crudware ever retrieved from those URLs, and she really shouldn't want
to have that rubbish back.

She continued to complain:  My nameserver was generating "broken links",
so obviously I must be depriving her of stuff she wants.  She knew that
those those links weren't broken anywhere except from home, so obviously
I was impairing her Internet experience.

I looked:  Indeed, there were 404s being generated (because of the
particular variety of oblivion I was then mapping the domain to).  Every
one of those 404s was objectively undesirable junk.

She complained.  I stressed that I wasn't filtering her Internet traffic,
just resolving certain domains locally.  If she didn't like my
nameserver policy, she was welcome to use any of millions of others, or
run her own.

She complained.  I reiterated that her shortage of banner ads, Web bugs,
and spy cookies wasn't my problem.

She complained.  I sat down at her machine and repointed them to Raw
Bandwidth's nameservers.

Moral:  No good deed goes unpunished.



It would be nice if Deirdre had the house Apple Airport base station
referencing my nameserver IP (only) for the DNS IP that it passes to
DHCP clients.  However, I'm betting it doesn't.  Crying shame, that.



http://linuxmafia.com/pub/linux/network/bind9-examples-linuxmafia.tar.gz 
has one other special feature:  a file called "maps-lawsuits".

Many years ago, I managed to make the day of Paul Vixie, DNS expert and
founder of the anti-spam MAPS Project, Inc., at the end of one of his 
lectures at BayLISA:  I told him that, the day Yesmail, Inc. got a
temporary restraining order against MAPS for announcing an intention to
put Yesmail's IPs in its DNS blocklist (alleging tortious interference
in Yesmail's business affairs), I sent an e-mail to several high
executives at Yesmail and its entire sales department:  Paul Vixie and
MAPSs, I said, would eventually get around to forgiving them for their
actions.  By contrast, I pointed out, they'd just managed to piss off
just about every sysadmin in the world, and the oceans would dry up, the
sun would burn out, and the universe would suffer heat death before
_they_ would either forgive or forget.

Therefore, I said, I predicted a long eternity of their IP addresses
gracing a large number of sysadmins' null-route lists, and hoped they
considered the sacrifice worthwhile.

The "maps-lawsuits" file details each of the five firms that sued MAPS
on what in my view were specious and disreputable grounds -- including, 
in some cases, who owns those firms now.  I hope to add specific IP
address lists, soon.

Being mindful of the restraint-of-trade statutes, I certainly won't tell
anyone _else_ to null-route those firms' IP addresses.  You could, for
example, send them Christmas cards.  Let your conscience be your guide.