Re: [CNS-NS #16701] DNS b0rk3n?

Michael Sinatra via RT wrote:
> rossd@quantum.me.berkeley.edu via RT wrote:
>
>
>>Hmmm... I was trying to hit www.namesys.com to forward
>>someone the speed results of Reiser4 (WELL worth the trouble, btw).
>>But it looks like the Campus DNS boxxen must be on a smoke break
>>or something....
>>
>>rossd@rossd ~ $ dig www.namesys.com
>>; <<>> DiG 9.2.3 <<>> www.namesys.com
>>;; global options: printcmd
>>;; connection timed out; no servers could be reached
>>
>>but PacBell's DNS has it...
>>rossd@rossd ~ $ dig @206.13.31.12 www.namesys.com
>>
>>; <<>> DiG 9.2.3 <<>> @206.13.31.12 www.namesys.com
>>;; global options: printcmd
>>;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49141
>>;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3
>>;; QUESTION SECTION:
>>;www.namesys.com. IN A
>>;; ANSWER SECTION:
>>www.namesys.com. 49484 IN A 212.16.7.65
>>;; AUTHORITY SECTION:
>>namesys.com. 49484 IN NS ns.namesys.com.
>>namesys.com. 49484 IN NS ns0.comstar.ru.
>>;; ADDITIONAL SECTION:
>>ns.namesys.com. 49484 IN A 212.16.7.65
>>ns0.comstar.ru. 41525 IN A 212.248.0.3
>>ns0.comstar.ru. 41525 IN A 195.210.128.3
>>;; Query time: 19 msec
>>;; SERVER: 206.13.31.12#53(206.13.31.12)
>>;; WHEN: Mon Jan 24 14:24:22 2005
>>;; MSG SIZE rcvd: 142
>>
>>and I can get other addresses from campus....
>>
>>rossd@rossd ~ $ dig www.yahoo.com
>>; <<>> DiG 9.2.3 <<>> www.yahoo.com
>>;; global options: printcmd
>>;; Got answer:
>>;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62833
>>;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 9, ADDITIONAL: 8
>>
>>;; QUESTION SECTION:
>>;www.yahoo.com. IN A
>>;; ANSWER SECTION:
>>www.yahoo.com. 219 IN CNAME www.yahoo.akadns.net.
>>www.yahoo.akadns.net. 48 IN A 66.94.230.44
>>www.yahoo.akadns.net. 48 IN A 66.94.230.48
>>www.yahoo.akadns.net. 48 IN A 66.94.230.49
>>www.yahoo.akadns.net. 48 IN A 66.94.230.50
>>www.yahoo.akadns.net. 48 IN A 66.94.230.32, etc, etc....
>>
>>but I did notice this...
>>rossd ~ $ nslookup www.namesys.com 128.32.206.12
>>Note: nslookup is deprecated and may be removed from future releases.
>>Consider using the `dig' or `host' programs instead. Run nslookup with
>>the `-sil[ent]' option to prevent this message from appearing.
>>;; reply from unexpected source: 128.32.136.12#53, expected 128.32.206.12#53
>>;; Warning: ID mismatch: expected ID 63012, got 24375
>>;; reply from unexpected source: 128.32.136.12#53, expected 128.32.206.12#53
>>;; Warning: ID mismatch: expected ID 63012, got 24375
>>;; connection timed out; no servers could be reached
>>
>>
>>Hope that helps... Reiser4 is definitely worth telling people about! :-)
>>
>>but i think it statistically anomalous to think it only this one site
>>is problematic...
>>
>>Anyone else notice it?
>
>
> Nobody else has noticed it, but I'll look into it. Usually when this
> sort of thing happens, it's because there's something subtly wrong with
> the domain in question, and it's effectively inadvertantly poisoning the
> caches of the nameservers of everyone who tries to go there.
>
> PacBell does something really weird with their nameservers and I need to
> find out what that is, but it effectly changes the caching effects that
> users see.

I may have spoken a bit too soon about the PacBell nameservers. They're
just using a really big anycast or load-balanced cluster and you can get
them to fail in the same way if you keep trying. I suspect that it's
just a question of which nameserver still has info about the domain in
its cache, and none of the UCB servers do (and only about 75% of the
time does PacBell, and that will drop to 0% as the caches expire, if the
problem isn't fixed).

The problem isn't that subtle, it's that all of the nameservers that are
supposed to be authoritative for namesys.com are either lame or
unreachable. Of all of those servers, only one actually responded when
I queried it, and I had to use TCP to get it to respond. Here's a
tcpdump of that conversation:

16:22:03.522737 128.32.155.8.3708 > 212.248.0.3.53: S
676578196:676578196(0) win 57344 <mss 1460,nop,wscale
0,nop,nop,timestamp 2185361 0> (DF)
16:22:03.723476 212.248.0.3.53 > 128.32.155.8.3708: S
1657155741:1657155741(0) ack 676578197 win 1460 <mss 1460> (DF)
16:22:03.723546 128.32.155.8.3708 > 212.248.0.3.53: . ack 1 win 58400 (DF)
16:22:03.723605 128.32.155.8.3708 > 212.248.0.3.53: P 1:36(35) ack 1 win
58400 3367+ A? www.namesys.com. (33) (DF)
16:22:04.020137 212.248.0.3.53 > 128.32.155.8.3708: . ack 36 win 32767 (DF)
16:22:11.144783 212.248.0.3.53 > 128.32.155.8.2113: P
2890942749:2890942784(35)ack 748629138 win 32767 38350 ServFail 0/0/0
(33) (DF)
16:22:11.144856 128.32.155.8.2113 > 212.248.0.3.53: R
748629138:748629138(0) win 0 (DF)
16:22:31.143719 212.248.0.3.53 > 128.32.155.8.3708: P 1:36(35) ack 36
win 327673367 ServFail 0/0/0 (33) (DF)
16:22:31.145842 128.32.155.8.3708 > 212.248.0.3.53: F 36:36(0) ack 36
win 58400(DF)
16:22:31.346369 212.248.0.3.53 > 128.32.155.8.3708: F 36:36(0) ack 37
win 65535(DF)

Actually, now that I look at it, I did get a response in UDP as well:

16:25:02.003608 128.32.155.8.1292 > 212.248.0.3.53: 65359+ A?
www.namesys.com.(33)
16:25:07.013193 128.32.155.8.1292 > 212.248.0.3.53: 65359+ A?
www.namesys.com.(33)
16:25:30.147934 212.248.0.3.53 > 128.32.155.8.1292: 65359 ServFail
0/0/0 (33)

In both cases, the "ServFail" message is definitely NOT a good sign.
Most likely the zone is either corrupted on that server or it's not even
configured to serve the zone. And this is the only server that deigned
to respond to queries.

Hopefully someone at namesys.com will notice after more caches start to
expire on the internet.

michael

------------------------------------------------------------------------
The following was automatically added to this message by the list server:

For information about Micronet, including subscribing to
or unsubscribing from its mailing list and finding out
about upcoming meetings, please visit the Micronet Web site:
<http://micronet.berkeley.edu/>.
Received on Tue Jan 25 10:29:59 2005