[sf-lug] resolver problem

[Michael Paoli]

I don't think we're seeing an IPv6 issue, in this specific case.

And why do I say that?:
$ dig +short github.com. A github.com. AAAA
192.30.252.122
$

Chromium works, but Firefox doesn't?  That sounds odd.
And in both cases, with full URLs?  E.g. starting with:
http://
or:
https://
and having a / after the hostname (and optional port) portion?
That sounds a bit odd to me, if that's the case.  Some difference in
cache with them (have you cleared cache, and still same results?),
or proxy settings, or possible (partial?) application/system compromise?
Or some plugin/addon in browser that's causing issues?
What if one boots off of known good clean ISO image on DVD/CD/USB?
Do the same problems still persist and in same manner?

If you temporarily remove your "workaround" of having added
192.30.252.131 github.com
to /etc/hosts
problem still there?
And if you then also temporarily edit your /etc/nsswitch.conf file
changing the line that reads:
hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4
to:
hosts:          files dns mdns4
Does it then work or does the problem persist?
What if you temporarily edit it to just:
hosts:          dns
for that hosts line? (and yes, you wouldn't want to leave it like that,
as that will generally break other things ... I did say "temporarily").
Anyway, in such case, do you still have one browser working, and the
other not, for github.com URL(s)?  And what of the git command using
github.com?

Could be (way) overkill, but use of strace(1) on the browsers - or even
ping, could (among the voluminous output) provide some useful clues -
notably how gethostbyname - or similar calls are being handled.  But
that still wouldn't necessarily tell one which bits of whatever's
handling the resolution are necessarily being returned by which layer.
Also, gethostbyname(3) is library call, not system call, so strace(1)
likely wouldn't show it directly, but rather some lower layer(s) as
program interacts with resolver.  Let's see ...
$ type Strace
Strace is /home/m/michael/bin/Strace
$ cat /home/m/michael/bin/Strace
#!/bin/sh
exec strace -fv -eall -s2048 ${1+"$@"}
$ 2>&1 Strace ping -c 3 github.com. | fgrep github | tail -n 1
recvfrom(3,  
".5\201\200\0\1\0\1\0\4\0\0\6github\3com\0\0\1\0\1\300\f\0\1\0\1\0\0\0R\0\4\300\36\374\201\300\f\0\2\0\1\0\2\215\340\0\24\3ns3\3p16\6dynect\3net\0\300\f\0\2\0\1\0\2\215\340\0\6\3ns1\300<\300\f\0\2\0\1\0\2\215\340\0\6\3ns4\300<\300\f\0\2\0\1\0\2\215\340\0\6\3ns2\300<", 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("127.0.0.1")}, [16]) =  
130
$
If I dug into and decoded that last bit, that's probably DNS response
from my local DNS server, as bit earlier on that, I see it's opened up
communication with 127.0.0.0 port 53 UDP, and that's the file descriptor
/ socket 3 that it's receiving some data from at that point.
I could probably also utilize tcpdump and perhaps also wireshark, or the
like, to capture and analyze that wee bit of data.

So ... hopefully with that, enough additional clues/hints/suggestions to
usefully proceed further with divide-and-conquer to isolate the cause of
the issue.

I also heard someone was lost?  ;-) ... may have provided a partial "map"
(or at least some more context clues) further below.

references/excerpts:
avahi-daemon(8)
resolver(3)
resolver(5)
http://linuxmafia.com/pipermail/sf-lug/2016q2/date.html
http://linuxmafia.com/pipermail/sf-lug/2016q2/011811.html
et. seq.

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 10:51:43 -0700
>
> My ubuntu 14.4 system has been booting without network configuration
> which I have been _assuming_ was due to a failed wifi module.
> So I switched to an ethernet cable connection which seems to work sort
> of!
>
> Chromium works but Firefox seems not to "see" the internet!
> So I stopped using Firefox and switched to Chromium.
>
> Then I had occasion to push something to github and got into
> problems:
>
> git push
> fatal: unable to access 'https://github.com/alexKleider/debk.git/':
> Could not resolve host: github.com
>
> ping github.com   # succeeds; (although last quad varies)
> PING github.com (192.30.252.130) 56(84) bytes of data.
> 64 bytes from github.com (192.30.252.130): icmp_seq=1 ttl=53 time=86.9 ms
>
> dig github.com @127.0.0.1 +short
> ;; connection timed out; no servers could be reached
>
> dig github.com @10.0.0.2 +short   # My router
> 192.30.252.131
>
> I'm able to get around the problem by adding the line
> 192.30.252.131 github.com
> to my /etc/hosts file

> From: "Daniel Gimpelevich" <daniel at gimpelevich.san-francisco.ca.us>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 10:54:54 -0700
>
> Try:
> dig github.com @127.0.1.1 +short

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 10:58:58 -0700
>
> alex at x301:~$ dig github.com @127.0.1.1 +short
> ;; connection timed out; no servers could be reached

> From: "Daniel Gimpelevich" <daniel at gimpelevich.san-francisco.ca.us>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 11:02:40 -0700
>
> The reason I said this was that my Ubuntu desktop has a dnsmasq instance
> that listens on 127.0.1.1 only. If yours isn't listening, NetworkManager
> is goobered in some way.

> From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 18:33:48 -0700
>
> What does your nsswitch.conf file say about hosts?
> What's in your /etc/resolv.conf file?

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 18:44:31 -0700
>
> I checked resolv.conf but clearly the way Ubuntu is configured they
> don't want me messing with it:
> # Dynamic resolv.conf(5) file for glibc resolver(3) generated by
> resolvconf(8)
> #     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
> nameserver 10.0.0.2
> search sonic.net
>
> As for nsswitch.conf, it's Greek to me but here it is:
> hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4
>
> Further info: I moved the hard drive from my computer to another
> (same model thinkpad x301) but the symptoms were the same so have to
> assume the problem is NOT to do with a bad wifi hardware component,
> which is what I initially suspected.

> From: "Daniel Gimpelevich" <daniel at gimpelevich.san-francisco.ca.us>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 18:55:51 -0700
>
>> nameserver 10.0.0.2
> This means NetworkManager is not even being used. The issue is almost
> certainly on your router at this point.

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 19:26:22 -0700
>
> 10.0.0.2 is the router.
>
> If it was the problem, would I not expect all other computers in the
> household to also malfunction and also should I not expect my
> computer to work fine when in another place?
> Neither of these are the case.
>
> I'm still curious to know how it is possible for ping to work while
> dig @127.0.0.1 does not.

> From: "Daniel Gimpelevich" <daniel at gimpelevich.san-francisco.ca.us>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 19:34:48 -0700
>
> On Mon, 2016-04-04 at 19:26 -0700, Alex Kleider wrote:
>> I'm still curious to know how it is possible for ping to work while
>> dig
>> @127.0.0.1 does not.
>
> dig @127.0.1.1 cannot work because you are not using NetworkManager.
> dig @127.0.0.1 cannot work because you are not running a nameserver.
>
> You're just using the glibc resolver to talk to 10.0.0.2 as resolv.conf
> says.

> From: jim <jim at well.com>
> Subject: Re: [sf-lug] resolver problem
> Date: Tue, 5 Apr 2016 02:42:02 +0000
>
>     Your /etc/resolv.conf file shows that you have
> a single nameserver with an IP address of 10.0.0.2
>
>     When you boot your computer, is it using
> dhclient to get a dhcp IP address or is your
> computer configured for a static address?
>     If static, then look at your /etc/network/interfaces
> file to see if it has a gateway directive in your
> interface clause.
>     If dhcp then inspect your router to see the
> gateway (external) network IP address, and maybe
> ISP docs to verify the nameserver IP address.

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: Re: [sf-lug] resolver problem
> Date: Mon, 04 Apr 2016 23:17:43 -0700
>
> So how is it possible that I can ping github.com
> but can not get git clone https://github.com/alexKleider/etc
> to work
> and how is it possible that Chromium can locate urls
> but Firefox can not??
>
> That is the mystery!  And if I knew how that is possible, there in
> probably will lie the clue what is the underlying problem.

> From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
> Subject: Re: [sf-lug] resolver problem
> Date: Tue, 05 Apr 2016 07:36:46 -0700
>
> The resolver libraries first check files (/etc/hosts),
> then mdns4_minimal (whatever that is - likely some DNS or nameservice
> caching thingy)
> if that gives a NOTFOUND by that point - I'm presuming an
> authoritative not found
> (NXDOMAIN or similar) at that point, rather than a failure, at that
> point it returns
> those results.  But if it got some other result - other than answer,
> it continues
> to DNS - where you haev 10.0.0.2 per /etc/resolv.conf, and then it may also
> use mdns4 after that.  So ... depending what results are received
> from files and
> mdns4_minimal, it may not even try DNS.

> From: "Daniel Gimpelevich" <daniel at gimpelevich.san-francisco.ca.us>
> Subject: Re: [sf-lug] resolver problem
> Date: Tue, 05 Apr 2016 08:28:49 -0700
>
> There is a reply from mDNS only when the query is in the .local TLD,
> that is, NOTFOUND means it's for a bogus .local hostname, which should
> then also be shielded from being put through real DNS servers.

> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: Re: [sf-lug] resolver problem
> Date: Tue, 5 Apr 2016 13:21:26 -0700
>
> This is a longshot, but perhaps some weird IPV6 failure?  Try testing
> again using ping6 and not just ping.

> From: "Alex Kleider" <akleider at sonic.net>
> Subject: Re: [sf-lug] resolver problem
> Date: Tue, 05 Apr 2016 18:43:43 -0700
>
> ping github.com does work
> but git push doesn't (although it used to.)
> Chromium can resolve URLs but Firefox can not!
> That's what I don't understand.
>
> git push
> fatal: unable to access 'https://github.com/alexKleider/debk.git/':
> Could not resolve host: github.com
>
> ping github.com   # succeeds; (although last quad varies)
> PING github.com (192.30.252.130) 56(84) bytes of data.
> 64 bytes from github.com (192.30.252.130): icmp_seq=1 ttl=53 time=86.9
> ms
>
> dig github.com @10.0.0.2 +short   # My router
> 192.30.252.131
>
> content of /etc/resolv.conf:
> nameserver 10.0.0.2
> search sonic.net
>
> I'm able to get around the problem by adding the line
> 192.30.252.131 github.com
> to my /etc/hosts file