[conspire] What happens when ... DNS ...
Michael Paoli
Michael.Paoli at cal.berkeley.edu
Sun Mar 6 23:01:14 PST 2016
> Date: Sun, 6 Mar 2016 16:53:15 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: conspire at linuxmafia.com
> Subject: [conspire] What happens when your hosting provider has zero
> regard for truth
>
> ----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----
>
> Date: Sun, 6 Mar 2016 16:51:41 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: Edie Stern
> Cc: Mark Olson, Joe Siclari
> Subject: Re: Help me Obi-Wan!
> Organization: If you lived here, you'd be $HOME already.
>
> Quoting estern770 at aol.com (estern770 at aol.com):
>
>> Rick, once again, thanks so much for your help. I wouldn't be
>> surprised if x7 contacts you some time for freelance problem
>> determination.
>
> You're all very welcome -- except I suspect that x7hosting at this point
> doesn't like me at all. ;-> FWIW, I just rechecked all the results I
> sent before -- and nothing has been fixed or changed in any way. All
> the errors I found are still there, unchanged.
>
>> I spoke to X7hosting and gave them Rick's analysis. They came back (a
>> couple days later) and said it was a domain registration problem.
>
> In case you didn't know: That makes no sense.
>
> What my results showed from direct, real-time examination is major
> deficiencies in the authoritative DNS -- which is published by four DNS
> nameservers at x7hosting. Domain registration (which you have placed at
> Network Solutions) has no relevance.
>
> The only connection _generically_ of domain registration to
> authoritative DNS is that the registrar updates (along with the domain's
> other data in the public WHOIS records) the list of authoritative
> nameservers within the parent DNS zone (the parent for fanac.org being
> .org).
>
> Using WHOIS client software, one can query what's in these records:
>
> $ whois fanac.org
> [...]
>
> Name Server: NS1.X7HOSTING.COM
> Name Server: NS2.X7HOSTING.COM
> Name Server: NS3.X7HOSTING.COM
> Name Server: NS4.X7HOSTING.COM
> [...]
> $
>
> These lines appear to be correct, i.e., they competently state what four
> x7hosting.com DNS nameservers are authoritative for fanac.org's DNS.
> The problem isn't them being _specified_ incorrectly (by Network
> Solutions). The problem is them _functioning_ incorrectly -- which is
> very clearly x7hosting's fault. Attempting to blame the registrar is
> more than a bit pathetic, and nonsensical.
>
> ----- End forwarded message -----
Do not presume malice for that which can be explained by incompetence[1]
So, let's check/confirm ...
and I typically wouldn't *start* with registrar and whois, if it is or
presumably is a DNS proble/issue ... though that is *also* a fine
starting point (and especially fine if one wants to check through
everything from top, to bottom).
So ... jumping in at slightly different point/approach.
fanac.org. & its DNS. Let's make semi-reasonable presumption
(unless/until evidence to the contrary may be seen), that org. DNS is
mostly or entirely working as it should, and at least isn't generally
hosed. And let's also start by presuming my local DNS isn't seriously
messed up. :-) (hey, I can be an optimist, right?)
So, first, let's start by finding NS records for org. I do this via
+trace, to get us the basic top-down, from DNS on that. Even if
something were screwed up in my local DNS (e.g. bad data cached), that
would bypass most of that - at least if I've got basic working
connectivity, and proper configuration of the root servers).
The +trace option of dig(1), well, man page words it better than I could:
+[no]trace
Toggle tracing of the delegation path from the root name servers
for the name being looked up. Tracing is disabled by default. When
tracing is enabled, dig makes iterative queries to resolve the name
being looked up. It will follow referrals from the root servers,
showing the answer from each server that was used to resolve the
lookup.
+dnssec is also set when +trace is set to better emulate the
default queries from a nameserver.
$ dig -t NS org. +trace
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> -t NS org. +trace
;; global options: +cmd
. 515734 IN NS b.root-servers.net.
. 515734 IN NS d.root-servers.net.
. 515734 IN NS c.root-servers.net.
. 515734 IN NS i.root-servers.net.
. 515734 IN NS h.root-servers.net.
. 515734 IN NS l.root-servers.net.
. 515734 IN NS e.root-servers.net.
. 515734 IN NS f.root-servers.net.
. 515734 IN NS k.root-servers.net.
. 515734 IN NS a.root-servers.net.
. 515734 IN NS g.root-servers.net.
. 515734 IN NS j.root-servers.net.
. 515734 IN NS m.root-servers.net.
. 518254 IN RRSIG NS 8 0 518400
20160316170000 20160306160000 54549 .
sujpVnxCntUPa0GwlFB7CBDcojpXXACQ+4BG/jck+Zcb5uNcnYQr/EYH
kycXgxjuay3q1wwmHo1ezoKRJECn0GNHhi2lAXaH8UdXaEda0NiY2m+K
36MJH4KDdeVbtgUQtepzIAlucMWBmfR4syn/O1VsFD/2QSSZ5aOvfpcD s4Y=
;; Received 397 bytes from 127.0.0.1#53(127.0.0.1) in 2135 ms
org. 172800 IN NS a2.org.afilias-nst.info.
org. 172800 IN NS a0.org.afilias-nst.info.
org. 172800 IN NS c0.org.afilias-nst.info.
org. 172800 IN NS b0.org.afilias-nst.org.
org. 172800 IN NS b2.org.afilias-nst.org.
org. 172800 IN NS d0.org.afilias-nst.org.
org. 86400 IN DS 9795 7 2
3922B31B6F3A4EA92B19EB7B52120F031FD8E05FF0B03BAFCF9F891B FE7FF8E5
org. 86400 IN DS 9795 7 1
364DFAB3DAF254CAB477B5675B10766DDAA24982
org. 86400 IN RRSIG DS 8 1 86400
20160316170000 20160306160000 54549 .
OUjabHkb3wYyYQhgbQb1+Wh4JZbyJYi1cLN6fT3OyGM2nzOEFdQfQsMq
RfFTMnWPpKCGSFf1qpQAUDqI+uX7TkCRHP7KcjC3Lmlgepl0QM0+eO8n
EV+UaJBF67Fehmk+2A6eAUgdapuZpOQqUuJ+mouwTSFG2sQlmLoe9FTn rHI=
;; Received 677 bytes from 192.33.4.12#53(c.root-servers.net) in 688 ms
org. 86400 IN NS d0.org.afilias-nst.org.
org. 86400 IN NS a2.org.afilias-nst.info.
org. 86400 IN NS b0.org.afilias-nst.org.
org. 86400 IN NS b2.org.afilias-nst.org.
org. 86400 IN NS c0.org.afilias-nst.info.
org. 86400 IN NS a0.org.afilias-nst.info.
org. 86400 IN RRSIG NS 7 1 86400
20160322145911 20160301135911 3248 org.
lxt0xpqjEmD5czxaNzdCVTXj8hfL1Truu6E2WwHtlnr0DMpIO1fc4OnX
pgDKUYTGUz6xdLYEH1f/hUSbe9KEORj+Jy8TfyGb6ql4jahWW03SjvQ7
Guqqt+byZ6eXq3BWtJh64QFTa9/e8KdnEX95vP8BrauUEwMlbZL1N9rK sxY=
;; Received 597 bytes from 199.19.54.1#53(b0.org.afilias-nst.org) in 27 ms
$
So, now I check against one of those NS servers for org. This is teensy
bit of shortcut, as I've not looked at whois(1) data for the domain, but
if things look as expected at org., then I don't particularly need to
look at whois(1), unless perhaps I want to also confirm that that
matches as expected to data I'm seeing from the org. nameserver(s).
And in brief, +noall shuts off most all the response data (just about
tell me nothin' about what responses one got back), +norecurse - don't
do request recursion - so ask the nameserver, but don't ask it to query
other nameserver(s), but just tell us what it has. And +authority, tell
us about any authority data/records in the response. Here I'm expecting
authority data, and not direct answers, as I expect the org. nameserver
to fully delegate any subdomain(s) thereof, so I'd expect it to have
authority, but not answer records for a subdomain directly under it.
So I ask for just that data (mostly to keep the "chatter" down and make
the results shown by dig(1) a bit more clear, relative to what I'm
looking for and interested in in this case).
$ dig @d0.org.afilias-nst.org. -t NS fanac.org. +noall +norecurse +authority
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @d0.org.afilias-nst.org. -t NS
fanac.org. +noall +norecurse +authority
; (2 servers found)
;; global options: +cmd
fanac.org. 86400 IN NS ns1.x7hosting.com.
fanac.org. 86400 IN NS ns2.x7hosting.com.
fanac.org. 86400 IN NS ns4.x7hosting.com.
fanac.org. 86400 IN NS ns3.x7hosting.com.
$
So, we see it fully delegated to NS servers ns[1-4].x7hosting.com.
Let's see what IP addresses those have:
$ (for ns in 1 2 3 4; do ns=ns"$ns".x7hosting.com.; echo $(dig +short
"$ns" A "$ns" AAAA) "[$ns]"; done)
64.29.149.100 [ns1.x7hosting.com.]
64.29.149.100 [ns2.x7hosting.com.]
216.251.37.96 [ns3.x7hosting.com.]
64.49.104.96 [ns4.x7hosting.com.]
$
Looking okay *so* far. Now let's see what those are (and aren't)
serving up for data of the actual domain in question.
$ (for ns in 1 2 3 4; do ns=ns"$ns".x7hosting.com.; ips=$(dig +short
"$ns" A "$ns" AAAA); for ip in $ips; do echo $(dig @"$ip" -t SOA
fanac.org. +short +norecurse) "[$ip $ns]"; done; done)
ns1.carrierzone.com. admin.carrierzone.com. 1007 86403 3600 3600000
86400 [64.29.149.100 ns1.x7hosting.com.]
ns1.carrierzone.com. admin.carrierzone.com. 1007 86403 3600 3600000
86400 [64.29.149.100 ns2.x7hosting.com.]
ns1.carrierzone.com. admin.carrierzone.com. 1007 86403 3600 3600000
86400 [216.251.37.96 ns3.x7hosting.com.]
[64.49.104.96 ns4.x7hosting.com.]
$
Well, three of the four gave us SOA record answers, and of those three,
the serial numbers match, but not good that we didn't get SOA record
result from one of the four ... how not good depends a bit more upon the
details.
Peeking also at TCP, as they should support TCP - just to see if they
may be clueless/incompetent about operating DNS.
$ (for ns in 1 2 3 4; do ns=ns"$ns".x7hosting.com.; ips=$(dig +short
"$ns" A "$ns" AAAA); for ip in $ips; do echo $(dig @"$ip" -t SOA
fanac.org. +short +tcp +norecurse) "[$ip $ns]"; done; done)
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @64.29.149.100 -t SOA fanac.org.
+short +tcp +norecurse ; (1 server found) ;; global options: +cmd ;;
connection timed out; no servers could be reached [64.29.149.100
ns1.x7hosting.com.]
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @64.29.149.100 -t SOA fanac.org.
+short +tcp +norecurse ; (1 server found) ;; global options: +cmd ;;
connection timed out; no servers could be reached [64.29.149.100
ns2.x7hosting.com.]
ns1.carrierzone.com. admin.carrierzone.com. 1007 86403 3600 3600000
86400 [216.251.37.96 ns3.x7hosting.com.]
;; Connection to 64.49.104.96#53(64.49.104.96) for fanac.org. failed:
connection refused. [64.49.104.96 ns4.x7hosting.com.]
$
Well, that's seriously messed up. Three of the four fail at accepting
a TCP connection - two of them timing out, and one refusing. Only one
accepted a TCP connection, but at least that one did give us a resultant
SOA record - but only one out of four ... that's pretty darn poor results.
So ... ignoring for a second the obvious TCP problem, let's look at the
one of four that didn't give us expected results even without asking
dig(1) to use TCP:
$ dig @64.49.104.96 -t SOA fanac.org. +norecurse
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @64.49.104.96 -t SOA fanac.org.
+norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30619
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 2800
;; QUESTION SECTION:
;fanac.org. IN SOA
;; Query time: 234 msec
;; SERVER: 64.49.104.96#53(64.49.104.96)
;; WHEN: Sun Mar 06 21:11:20 PST 2016
;; MSG SIZE rcvd: 38
$
Not good, but we'll get back to that ...
Likewise checking that same DNS server for NS records for our domain in
question.
$ dig @64.49.104.96 -t NS fanac.org. +norecurse
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @64.49.104.96 -t NS fanac.org.
+norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23756
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 2800
;; QUESTION SECTION:
;fanac.org. IN NS
;; Query time: 98 msec
;; SERVER: 64.49.104.96#53(64.49.104.96)
;; WHEN: Sun Mar 06 21:27:18 PST 2016
;; MSG SIZE rcvd: 38
$
Well, that's both a bit odd/unexpected (until we take their incompetence
into account).
We see above, that the DNS server *is* answering, and not that the
domain doesn't exist (which would have shown as NXDOMAIN), but it's
response has exactly no records - so the answer is, according to that
DNS server, that the record doesn't exist - no SOA record for that
domain, and likewise no NS records. Both incorrect answers, but a
rather odd one. It's not NXDOMAIN, so the domain exits. But no SOA
record and no NS records. So that implies it's directly under org., and
not delegated ... not true, but that's what the data implies. And DNS
clients will presume the data is correct. So, from that, they may cache
that result. How long? Well, not delegated (per what the server
claims), so we go up to a domain that exists - org. in this case - and
use its
MINIMUM a.k.a. Negative Cache TTL
So, for org., that's ...
$ dig -t SOA org. +short
a0.org.afilias-nst.info. noc.afilias-nst.info. 2011898256 1800 900
604800 86400
$
86400 seconds which is 24 hours. So, once that response is received
from the DNS server, any and all clients and caching, etc., may
legitimately presume there is no SOA nor NS records for that domain, for
up to 24 hours.
Curious, that ill-behaved DNS server, what does it show for an ANY
query?
$ dig @64.49.104.96 -t ANY fanac.org. +norecurse
; <<>> DiG 9.9.5-9+deb8u5-Debian <<>> @64.49.104.96 -t ANY fanac.org.
+norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 32566
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 2800
;; QUESTION SECTION:
;fanac.org. IN ANY
;; Query time: 96 msec
;; SERVER: 64.49.104.96#53(64.49.104.96)
;; WHEN: Sun Mar 06 22:01:14 PST 2016
;; MSG SIZE rcvd: 38
$
Well, that's rather bizarre in any case. Dear knows how they messed
that up. In any case, it is answering, and in the answer it's not
telling us the domain doesn't exist (NXDOMAIN would tell us the domain
doesn't exist), yet we haven't found any records for the domain (but ANY
isn't all-inclusive so it may have some data for the domain we're just
not finding e.g. if there's a DNS record for
www.fanac.org., but no DNS records for fanac.org (would never
legitimately be the case - but if org. directly hosted without
delegation, in such a hypothetical case, then such could legitimately
possibly exist that way)
). Anyway, given the (incorrect) data that is served up, DNS
clients/caching and such, may very legitimately presume there is are no
SOA nor NS records for that domain, and may cache that (mis)fact for up
to 24 hours.
So, for every query that hits the one in four of their DNS servers, your
domain may be screwed by any and all clients that continue to use that
data that they served up, for up to 24 hours.
Oh, and possible ugly kludge of a work-around? If the DNS server that's
giving bad answers is consistent in its DNS name, remove the NS records
that delegate to it. So, remove the NS record:
fanac.org. IN NS ns4.x7hosting.com.
and likewise remove the delegation ... so for fanac.org.,
that would be remove ns4.x7hosting.com from among its listed DNS
servers. Might take a while for all that to percolate through DNS,
but if one can do all that, then within 24 to 48 hours or so (depending
upon various involved TTLs and such). at least then one would
effectively "cut off" paying attention to the DNS server that's serving
up bad data. That doesn't fix or work around the TCP issue, though.
As only one of the four accepts TCP connections, well, if one reduced to
single DNS server, there are at least two potential problems. That's
generally a no-no per RFCs (and for good reason - only one and there's no
redundancy or fail-over capability if that IP can't be reached) - should
have at least two, and preferably at least three. Also, some registrars
may not let one reduce to only two or only one. But I don't think org.
is that persnickety, but one's registrar may or may not be so persnickety
on that with a org. registration or updates thereof.
Oh, and if a provider sucks and can't fix their stuff, especially given
ample notice and time, take one's business elsewhere (best not to reward
incompetence), and leave ample clues behind (reviews, diagnostic,
summaries of their failure to correct, etc.) for others to find - at
least that way others have at least half a chance of finding out the
provider is problematic and clueness, and best be avoided or abandoned
for better.
Or, counterexample, as I sometimes say:
"If you want to see more of something, feed it money."
One can also get good competently managed DNS for free*!
*well, at least in simpler cases. If one has huge number of records
and/or very high volumes of DNS traffic, it's generally not going to be
found for free.
The incompetence levels I seem to see at x7hosting.com, make me think
that free would be far too high a price to pay for their disservice.
How much are they paying you to take their abuse and incompetence?
1. https://en.wikipedia.org/wiki/Hanlon%27s_razor
More information about the conspire
mailing list