submit tech/20180706 Anatomy of a Linux DNS Lookup - Part III.md

This commit is contained in:
songshunqiang 2018-08-29 16:41:58 +08:00
parent 2b71f8b567
commit c98efe6238
2 changed files with 300 additions and 316 deletions

View File

@ -1,316 +0,0 @@
pinewall is translating
Anatomy of a Linux DNS Lookup Part III
============================================================
In [Anatomy of a Linux DNS Lookup Part I][1] I covered:
* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `ping` vs `host` style lookups
and in [Anatomy of a Linux DNS Lookup Part II][2] I covered:
* `systemd` and its `networking` service
* `ifup` and `ifdown`
* `dhclient`
* `resolvconf`
and ended up here:
* * *
![linux-dns-2 (2)](https://zwischenzugs.files.wordpress.com/2018/06/linux-dns-2-2.png?w=525)
_A (roughly) accurate map of whats going on_
Unfortunately, thats not the end of the story. Theres still more things that can get involved. In Part III, Im going to cover NetworkManager and dnsmasq and briefly show how they play a part.
* * *
# 1) NetworkManager
As mentioned in Part II, we are now well away from POSIX standards and into Linux distribution-specific areas of DNS resolution management.
In my preferred distribution (Ubuntu), there is a service thats available and often installed for me as a dependency of some other package I install called [NetworkManager][3]. Its actually a service developed by RedHat in 2004 to help manage network interfaces for you.
What does this have to do with DNS? Install it to find out:
```
$ apt-get install -y network-manager
```
In my distribution, I get a config file.
```
$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile,ofono
dns=dnsmasq
[ifupdown]
managed=false
```
See that `dns=dnsmasq` there? That means that NetworkManager will use `dnsmasq` to manage DNS on the host.
* * *
# 2) dnsmasq
The dnsmasq program is that now-familiar thing: yet another level of indirection for `/etc/resolv.conf`.
Technically, dnsmasq can do a few things, but is primarily it acts as a DNS server that can cache requests to other DNS servers. It runs on port 53 (the standard DNS port), on all local network interfaces.
So where is `dnsmasq` running? NetworkManager is running:
```
$ ps -ef | grep NetworkManager
root     15048     1  0 16:39 ?        00:00:00 /usr/sbin/NetworkManager --no-daemon
```
But no `dnsmasq` process exists:
```
$ ps -ef | grep dnsmasq
$
```
Although its configured to be used, confusingly its not actually installed! So youre going to install it.
Before you install it though, lets check the state of `/etc/resolv.conf`.
```
$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.2
search home
```
Its not been changed by NetworkManager.
If `dnsmasq` is installed:
```
$ apt-get install -y dnsmasq
```
Then `dnsmasq` is up and running:
```
$ ps -ef | grep dnsmasq
dnsmasq  15286     1  0 16:54 ?        00:00:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq/dnsmasq.pid -u dnsmasq -r /var/run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
```
And `/etc/resolv.conf` has changed again!
```
root@linuxdns1:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search home
```
And `netstat` shows `dnsmasq` is serving on all interfaces at port 53:
```
$ netstat -nlp4
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address     Foreign Address State   PID/Program name
tcp        0      0 127.0.0.1:53      0.0.0.0:*       LISTEN  15286/dnsmasq 
tcp        0      0 10.0.2.15:53      0.0.0.0:*       LISTEN  15286/dnsmasq
tcp        0      0 172.28.128.11:53  0.0.0.0:*       LISTEN  15286/dnsmasq
tcp        0      0 0.0.0.0:22        0.0.0.0:*       LISTEN  1237/sshd
udp        0      0 127.0.0.1:53      0.0.0.0:*         15286/dnsmasq
udp        0      0 10.0.2.15:53      0.0.0.0:*               15286/dnsmasq  
udp        0      0 172.28.128.11:53  0.0.0.0:*               15286/dnsmasq  
udp        0      0 0.0.0.0:68        0.0.0.0:*               10758/dhclient
udp        0      0 0.0.0.0:68        0.0.0.0:*               10530/dhclient
udp        0      0 0.0.0.0:68        0.0.0.0:*               10185/dhclient
```
* * *
# 3) Unpicking dnsmasq
Now we are in a situation where all DNS queries are going to `127.0.0.1:53` and from there what happens?
We can get a clue from looking again at the `/var/run` folder. The `resolv.conf` in `resolvconf` has been changed to point to where `dnsmasq` is being served:
```
$ cat /var/run/resolvconf/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search home
```
while theres a new `dnsmasq` folder with its own `resolv.conf`.
```
$ cat /run/dnsmasq/resolv.conf
nameserver 10.0.2.2
```
which has the nameserver given to us by `DHCP`.
We can reason about this without looking too deeply, but what if we really want to know whats going on?
* * *
# 4) Debugging Dnsmasq
Frequently Ive found myself wondering what dnsmasqs state is. Fortunately, you can get a good amount of information out of it if you set change this line in `/etc/dnsmasq.conf`:
```
#log-queries
```
to:
```
log-queries
```
and restart `dnsmasq`
Now, if you do a simple:
```
$ ping -c1 bbc.co.uk
```
you will see something like this in `/var/log/syslog` (the `[...]` indicates that the lines start is the same as the previous one):
```
Jul  3 19:56:07 ubuntu-xenial dnsmasq[15372]: query[A] bbc.co.uk from 127.0.0.1
[...] forwarded bbc.co.uk to 10.0.2.2
[...] reply bbc.co.uk is 151.101.192.81
[...] reply bbc.co.uk is 151.101.0.81
[...] reply bbc.co.uk is 151.101.64.81
[...] reply bbc.co.uk is 151.101.128.81
[...] query[PTR] 81.192.101.151.in-addr.arpa from 127.0.0.1
[...] forwarded 81.192.101.151.in-addr.arpa to 10.0.2.2
[...] reply 151.101.192.81 is NXDOMAIN
```
which shows what `dnsmasq` received, where the query was forwarded to, and what reply was received.
If the query is returned from the cache (or, more exactly, the local time-to-live for the query has not expired), then it looks like this in the logs:
```
[...] query[A] bbc.co.uk from 127.0.0.1
[...] cached bbc.co.uk is 151.101.64.81
[...] cached bbc.co.uk is 151.101.128.81
[...] cached bbc.co.uk is 151.101.192.81
[...] cached bbc.co.uk is 151.101.0.81
[...] query[PTR] 81.64.101.151.in-addr.arpa from 127.0.0.1
```
and if you ever want to know whats in your cache, you can provoke dnsmasq into sending it to the same log file by sending the `USR1` signal to the dnsmasq process id:
```
$ kill -SIGUSR1 <(cat /run/dnsmasq/dnsmasq.pid)
```
and the output of the dump looks like this:
```
Jul  3 15:08:08 ubuntu-xenial dnsmasq[15697]: time 1530630488                                                                                                                                 
[...] cache size 150, 0/5 cache insertions re-used unexpired cache entries.                                                                           
[...] queries forwarded 2, queries answered locally 0                                                                                                 
[...] queries for authoritative zones 0                                                                                                               
[...] server 10.0.2.2#53: queries sent 2, retried or failed 0                                                                                         
[...] Host           Address        Flags      Expires                            
[...] linuxdns1      172.28.128.8   4FRI   H                                                                
[...] ip6-localhost  ::1            6FRI   H                                                                
[...] ip6-allhosts   ff02::3         6FRI   H                                                                
[...] ip6-localnet     fe00::         6FRI   H                                                                
[...] ip6-mcastprefix  ff00::         6FRI   H                                                                
[...] ip6-loopback     :               6F I   H                                                                
[...] ip6-allnodes    ff02:           6FRI   H                                                                
[...] bbc.co.uk        151.101.64.81  4F         Tue Jul  3 15:11:41 2018                                     
[...] bbc.co.uk        151.101.192.81 4F         Tue Jul  3 15:11:41 2018                                     
[...] bbc.co.uk        151.101.0.81    4F         Tue Jul  3 15:11:41 2018                                     
[...] bbc.co.uk        151.101.128.81 4F         Tue Jul  3 15:11:41 2018                                     
[...]                  151.101.64.81  4 R  NX    Tue Jul  3 15:34:17 2018                                     
[...] localhost        127.0.0.1      4FRI   H                                                                
[...] <Root>           19036   8   2  SF I                                                                    
[...] ip6-allrouters   ff02::2        6FRI   H        
```
In the above output, I believe (but dont know, and ? indicates a relatively wild guess on my part) that:
* 4 means IPv4
* 6 means IPv6
* H means address was read from an `/etc/hosts` file
* I ? Immortal DNS value? (ie no time-to-live value?)
* F ?
* R ?
* S?
* N?
* X
#### Alternatives to dnsmasq
`dnsmasq` is not the only option that can be passed to dns in NetworkManager. Theres `none` which does nothing to `/etc/resolv,conf`, `default`, which claims to update `resolv.conf`  to reflect currently active connections, and `unbound`, which communicates with the `unbound` service and `dnssec-triggerd`, which is concerned with DNS security and is not covered here.
* * *
### End of Part III
Thats the end of Part III, where we covered the NetworkManager service, and its `dns=dnsmasq` setting.
Lets briefly list some of the things weve come across so far:
* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `/run/resolvconf/resolv.conf`
* `systemd` and its `networking` service
* `ifup` and `ifdown`
* `dhclient`
* `resolvconf`
* `NetworkManager`
* `dnsmasq`
--------------------------------------------------------------------------------
via: https://zwischenzugs.com/2018/07/06/anatomy-of-a-linux-dns-lookup-part-iii/
作者:[ZWISCHENZUGS][a]
译者:[译者ID](https://github.com/译者ID)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://zwischenzugs.com/
[1]:https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-lookup-part-i/
[2]:https://zwischenzugs.com/2018/06/18/anatomy-of-a-linux-dns-lookup-part-ii/
[3]:https://en.wikipedia.org/wiki/NetworkManager

View File

@ -0,0 +1,300 @@
Linux DNS 查询剖析(第三部分)
============================================================
在 [Linux DNS 查询剖析(第一部分)][1]中,我们介绍了:
* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `ping` 与 `host` 查询方式的对比
and in [Linux DNS 查询剖析(第二部分)][2],我们介绍了:
* `systemd` 和对应的 `networking` 服务
* `ifup` 和 `ifdown`
* `dhclient`
* `resolvconf`
剖析进展如下:
* * *
![linux-dns-2 (2)][4]
_大致准确的关系图_
很可惜,故事还没有结束,还有不少东西也会影响 DNS 查询。在第三部分中,我将介绍 `NetworkManager``dnsmasq`,简要说明它们如何影响 DNS 查询。
* * *
### 1) NetworkManager
在第二部分已经提到,我们现在介绍的内容已经偏离 POSIX 标准,涉及的 DNS 解析管理部分在各个发行版上形式并不统一。
在我使用的发行版 Ubuntu有一个名为 [NetworkManager][3] 的服务,它通常作为一些其它软件包的依赖被安装而且处于<ruby>激活<rt>available</rt></ruby>状态。它实际上是 RedHat 在 2004 年开发的一个服务,用于帮助你管理网络接口。
它与 DNS 查询有什么关系呢?让我们安装这个服务并找出答案:
```
$ apt-get install -y network-manager
```
对于 Ubuntu在软件包安装后你可以发现一个新的配置文件
```
$ cat /etc/NetworkManager/NetworkManager.conf
[main]
plugins=ifupdown,keyfile,ofono
dns=dnsmasq
[ifupdown]
managed=false
```
看到 `dns=dnsmasq` 了吧?这意味着 `NetworkManager` 将使用 `dnsmasq` 管理主机上的 DNS。
* * *
### 2) dnsmasq
`dnsmasq` 程序是我们很熟悉的程序:只是 `/etc/resolv.conf` 之上的又一个间接层。
理论上,`dnsmasq` 有多种用途,但主要被用作 DNS 缓存服务器,缓存到其它 DNS 服务器的请求。`dnsmasq` 在本地所有网络接口上监听 53 端口(标准的 DNS 端口)。
那么 `dnsmasq` 运行在哪里呢?`NetworkManager` 的运行情况如下:
```
$ ps -ef | grep NetworkManager
root     15048     1  0 16:39 ?        00:00:00 /usr/sbin/NetworkManager --no-daemon
```
但并没有找到 `dnsmasq` 相关的进程:
```
$ ps -ef | grep dnsmasq
$
```
令人迷惑的是,虽然 `dnsmasq` 被配置用于管理 DNS但其实并没有安装在系统上因而你需要自己安装它。
安装之前,让我们查看一下 `/etc/resolv.conf` 文件的内容:
```
$ cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.2
search home
```
可见,并没有被 `NetworkManager` 修改。
如果安装 `dnsmasq`:
```
$ apt-get install -y dnsmasq
```
这时,`dnsmasq` 已经启动运行:
```
$ ps -ef | grep dnsmasq
dnsmasq  15286     1  0 16:54 ?        00:00:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq/dnsmasq.pid -u dnsmasq -r /var/run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
```
然后,`/etc/resolv.conf` 文件内容又改变了!
```
root@linuxdns1:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search home
```
运行 `netstat` 命令,可以看出 `dnsmasq` 在所有网络接口上监听 53 端口:
```
$ netstat -nlp4
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address     Foreign Address State   PID/Program name
tcp        0      0 127.0.0.1:53      0.0.0.0:*       LISTEN  15286/dnsmasq 
tcp        0      0 10.0.2.15:53      0.0.0.0:*       LISTEN  15286/dnsmasq
tcp        0      0 172.28.128.11:53  0.0.0.0:*       LISTEN  15286/dnsmasq
tcp        0      0 0.0.0.0:22        0.0.0.0:*       LISTEN  1237/sshd
udp        0      0 127.0.0.1:53      0.0.0.0:*         15286/dnsmasq
udp        0      0 10.0.2.15:53      0.0.0.0:*               15286/dnsmasq  
udp        0      0 172.28.128.11:53  0.0.0.0:*               15286/dnsmasq  
udp        0      0 0.0.0.0:68        0.0.0.0:*               10758/dhclient
udp        0      0 0.0.0.0:68        0.0.0.0:*               10530/dhclient
udp        0      0 0.0.0.0:68        0.0.0.0:*               10185/dhclient
```
* * *
### 3) 分析 dnsmasq
在目前的情况下,所有的 DNS 查询都会使用 `127.0.0.1:53` 这个 DNS 服务器,下一步会发生什么呢?
我再次查看 `/var/run` 目录,可以发现一个线索:
`resolvconf` 目录下 `resolv.conf` 文件中的配置也相应变更,变更为 `dnsmasq` 对应的 DNS 服务器:
```
$ cat /var/run/resolvconf/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 127.0.0.1
search home
```
同时,出现了一个新的 `dnsmasq` 目录,也包含一个 `resolv.conf` 文件:
```
$ cat /run/dnsmasq/resolv.conf
nameserver 10.0.2.2
```
LCTT 译注:这里依次提到了 `/var/run``/run`,使用 Ubuntu 16.04 LTS 验证发现,`/var/run` 其实是指向 `/run/` 的软链接)
该文件包含我们从 `DHCP` 获取的 `nameserver`
虽然可以推导出这个结论,但如何查看具体的调用逻辑呢?
* * *
### 4) 调试 dnsmasq
我经常思考 `dnsmasq` (在整个过程中)的功能定位。幸运的是,如果你将 `/etc/dnsmasq.conf` 中的一行做如下调整,你可以获取大量 `dnsmasq` 状态的信息:
```
#log-queries
```
修改为:
```
log-queries
```
然后重启 `dnsmasq`
接下来,只要运行一个简单的命令:
```
$ ping -c1 bbc.co.uk
```
你就可以在 `/var/log/syslog` 中找到类似的内容(其中 `[...]` 表示行首内容与上一行相同):
```
Jul  3 19:56:07 ubuntu-xenial dnsmasq[15372]: query[A] bbc.co.uk from 127.0.0.1
[...] forwarded bbc.co.uk to 10.0.2.2
[...] reply bbc.co.uk is 151.101.192.81
[...] reply bbc.co.uk is 151.101.0.81
[...] reply bbc.co.uk is 151.101.64.81
[...] reply bbc.co.uk is 151.101.128.81
[...] query[PTR] 81.192.101.151.in-addr.arpa from 127.0.0.1
[...] forwarded 81.192.101.151.in-addr.arpa to 10.0.2.2
[...] reply 151.101.192.81 is NXDOMAIN
```
可以清晰看出 `dnsmasq` 收到的查询、查询被转发到了哪里以及收到的回复。
如果查询被缓存命中(或者说,本地的查询结果还在<ruby>存活时间<rt>time-to-live</rt></ruby>内,并未过期),日志显示如下:
```
[...] query[A] bbc.co.uk from 127.0.0.1
[...] cached bbc.co.uk is 151.101.64.81
[...] cached bbc.co.uk is 151.101.128.81
[...] cached bbc.co.uk is 151.101.192.81
[...] cached bbc.co.uk is 151.101.0.81
[...] query[PTR] 81.64.101.151.in-addr.arpa from 127.0.0.1
```
如果你想了解缓存中有哪些记录,可以向 `dnsmasq` 进程 id 发送 `USR1` 信号,这样 `dnsmasq` 会将缓存记录导出并写入到相同的日志文件中:
```
$ kill -SIGUSR1 $(cat /run/dnsmasq/dnsmasq.pid)
```
LCTT 译注:原文中命令执行报错,已变更成最接近且符合作者意图的命令)
导记录对应如下输出:
```
Jul  3 15:08:08 ubuntu-xenial dnsmasq[15697]: time 1530630488
[...] cache size 150, 0/5 cache insertions re-used unexpired cache entries.
[...] queries forwarded 2, queries answered locally 0
[...] queries for authoritative zones 0
[...] server 10.0.2.2#53: queries sent 2, retried or failed 0
[...] Host           Address        Flags      Expires
[...] linuxdns1      172.28.128.8   4FRI   H
[...] ip6-localhost  ::1            6FRI   H
[...] ip6-allhosts   ff02::3         6FRI   H
[...] ip6-localnet     fe00::         6FRI   H
[...] ip6-mcastprefix  ff00::         6FRI   H
[...] ip6-loopback     :               6F I   H
[...] ip6-allnodes    ff02:           6FRI   H
[...] bbc.co.uk        151.101.64.81  4F         Tue Jul  3 15:11:41 2018
[...] bbc.co.uk        151.101.192.81 4F         Tue Jul  3 15:11:41 2018
[...] bbc.co.uk        151.101.0.81    4F         Tue Jul  3 15:11:41 2018
[...] bbc.co.uk        151.101.128.81 4F         Tue Jul  3 15:11:41 2018
[...]                  151.101.64.81  4 R  NX    Tue Jul  3 15:34:17 2018
[...] localhost        127.0.0.1      4FRI   H
[...] <Root>           19036   8   2  SF I
[...] ip6-allrouters   ff02::2        6FRI   H
```
在上面的输出中,我猜测(并不确认,`?` 代表我比较疯狂的猜测)如下:
* `4` 代表 IPv4
* `6` 代表 IPv6
* `H` 代表从 `/etc/hosts` 中读取 IP 地址
* `I` “永生”的 DNS 记录 (例如,没有设置存活时间数值 )
* `F`
* `R`
* `S`
* `N`
* `X`
LCTT 译注:查看 `dnsmasq` 的源代码 [`cache.c`][5] 可知,`4` 代表 `IPV4``6` 代表 `IPV6``C` 代表 `CNAME``S` 代表 `DNSSEC``F` 代表 `FORWARD``R` 代表 `REVERSE``I` 代表 `IMMORTAL``N` 代表 `NEG``X` 代表 `NXDOMAIN``H` 代表 `HOSTS`。更具体的含义需要查看代码或相关文档)
#### dnsmasq 的替代品
`NetworkManager` 配置中的 `dns` 字段并不是只能使用 `dnsmasq`,可选项包括 `none``default``unbound` 和 `dnssec-triggered` 等。使用 `none` 时,`NetworkManager` 不会改动 `/etc/resolv.conf`;使用 `default` 时,`NetworkManager` 会根据当前的<ruby>活跃连接<rt>active connections</rt></ruby>更新 `resolv.conf`;使用 `unbound` 时,`NetworkManager` 会与 `unbound` 服务通信;`dnssec-triggered` 与 DNS 安全相关,不在本文讨论范围。
* * *
### 第三部分总结
第三部分到此结束,其中我们介绍了 `NetworkManager` 服务及其 `dns=dnsmasq` 的配置。
下面简要罗列一下我们已经介绍过的全部内容:
* `nsswitch`
* `/etc/hosts`
* `/etc/resolv.conf`
* `/run/resolvconf/resolv.conf`
* `systemd` 及对应的 `networking` 服务
* `ifup` 和 `ifdown`
* `dhclient`
* `resolvconf`
* `NetworkManager`
* `dnsmasq`
--------------------------------------------------------------------------------
via: https://zwischenzugs.com/2018/07/06/anatomy-of-a-linux-dns-lookup-part-iii/
作者:[ZWISCHENZUGS][a]
译者:[pinewall](https://github.com/pinewall)
校对:[校对者ID](https://github.com/校对者ID)
本文由 [LCTT](https://github.com/LCTT/TranslateProject) 原创编译,[Linux中国](https://linux.cn/) 荣誉推出
[a]:https://zwischenzugs.com/
[1]:https://linux.cn/article-9943-1.html
[2]:https://linux.cn/article-9949-1.html
[3]:https://en.wikipedia.org/wiki/NetworkManager
[4]:https://zwischenzugs.files.wordpress.com/2018/06/linux-dns-2-2.png?w=525
[5]:https://github.com/imp/dnsmasq/blob/master/src/cache.c