TranslateProject/sources/tech/20221106 Making a DNS query in Ruby from scratch.md

Making a DNS query in Ruby from scratch

Hello! A while back I wrote a post about how to write a toy DNS resolver in Go.

In that post I left out “how to generate and parse DNS queries” because I thought it was boring, but a few people pointed out that they did not know how to parse and generate DNS queries and they were interested in how to do it.

This made me curious – how much work is it do the DNS parsing? It turns out we can do it in a pretty nice 120-line Ruby program, which is not that bad.

So here’s a quick post on how to generate DNS queries and parse DNS responses! We’re going to do it in Ruby because I’m giving a talk at a Ruby conference soon, and this blog post is partly prep for that talk :). I’ve tried to keep it readable for folks who don’t know Ruby though, I’ve only used pretty basic Ruby code.

At the end we’re going to have a very simple toy Ruby version of dig that can look up domain names like this:

    $ ruby dig.rb example.com
    example.com    20314    A    93.184.216.34

The whole thing is about 120 lines of code, so it’s not that much. (The final program is dig.rb if you want to skip the explanations and just read some code.) We won’t implement the “how a DNS resolver works” from the previous post because, well, we already did that. Let’s get into it!

Along the way I’m going to try to explain how you could figure out some of this stuff yourself if you were trying to figure out how DNS queries are formatted from scratch. Mostly that’s “poke around in Wireshark” and “read RFC 1035, the DNS RFC”.

step 1: open a UDP socket

We need to actually send our queries, so to do that we need to open a UDP socket. We’ll send our queries to 8.8.8.8, Google’s DNS server.

Here’s the code to set up a UDP connection to 8.8.8.8, port 53 (the DNS port).

    require 'socket'
    sock = UDPSocket.new

    sock.bind('0.0.0.0', 12345)
    sock.connect('8.8.8.8', 53)

a quick note on UDP

I’m not going to say too much about UDP here, but I will say that the basic unit of computer networking is the “packet” (a packet is a string of bytes), and in this program we’re going to do the simplest possible thing you can do with a computer network – send 1 packet and receive 1 packet in response.

So UDP is a way to send packets in the simplest possible way.

It’s the most common way to send DNS queries, though you can also use TCP or DNS-over-HTTPS instead.

step 2: copy a DNS query from Wireshark

Next: let’s say we have no idea how DNS works but we want to send a working query as fast as possible. The easiest way to get a DNS query to play with and make sure our UDP connection is working is to just copy one that already works!

So that’s what we’re going to do, using Wireshark (an incredible packet analysis tool)

The steps I used to this are roughly:

1. Open Wireshark and click ‘capture’
2. Enter udp.port == 53 as a filter (in the search bar)
3. Run ping example.com in my terminal (to generate a DNS query)
4. Click on the DNS query (“Standard query A example.com”)
5. Right click on “Domain Name System (query”) in the bottom left pane
6. Click ‘Copy’ -> ‘as a hex stream’
7. Now I have “b96201000001000000000000076578616d706c6503636f6d0000010001” on my clipboard, to use in my Ruby program. Hooray!

step 3: decode the hex stream and send the DNS query

Now we can send our DNS query to 8.8.8.8! Here’s what that looks like: we just need to add 5 lines of code

    hex_string = "b96201000001000000000000076578616d706c6503636f6d0000010001"
    bytes = [hex_string].pack('H*')
    sock.send(bytes, 0)

    # get the reply
    reply, _ = sock.recvfrom(1024)
    puts reply.unpack('H*')

[hex_string].pack('H*') is translating our hex string into a byte string. At this point we don’t really know what this data means but we’ll get there in a second.

We can also take this opportunity to make sure our program is working and is sending valid data, using tcpdump. How I did that:

1. Run sudo tcpdump -ni any port 53 and host 8.8.8.8 in a terminal tab
2. In a different terminal tab, run this Ruby program (ruby dns-1.rb)

Here’s what the output looks like:

    $ sudo tcpdump -ni any port 53 and host 8.8.8.8
    08:50:28.287440 IP 192.168.1.174.12345 > 8.8.8.8.53: 47458+ A? example.com. (29)
    08:50:28.312043 IP 8.8.8.8.53 > 192.168.1.174.12345: 47458 1/0/0 A 93.184.216.34 (45)

This is really good - we can see the DNS request (“what’s the IP for example.com”) and the response (“it’s 93.184.216.34”). So everything is working. Now we just need to, you know, figure out how to generate and decode this data ourselves.

step 4: learn a little about how DNS queries are formatted

Now that we have a DNS query for example.com, let’s learn about what it means.

Here’s our query, formatted as hex.

    b96201000001000000000000076578616d706c6503636f6d0000010001

If you poke around in Wireshark, you’ll see that this query has 2 parts:

• The header (b96201000001000000000000)
• The question (076578616d706c6503636f6d0000010001)

step 5: make the header

Our goal in this step is to generate the byte string b96201000001000000000000, but with a Ruby function instead of hardcoding it.

So: the header is 12 bytes. What do those 12 bytes mean? If you look at Wireshark (or read RFC 1035), you’ll see that it’s 6 2-byte numbers concatenated together.

The 6 numbers correspond to the query ID, the flags, and then the number of questions, answer records, authoritative records, and additional records in the packet.

We don’t need to worry about what all those things are yet though – we just need to put in 6 numbers.

And luckily we know exactly which 6 numbers to put because our goal is to literally generate the string b96201000001000000000000.

So here’s a function to make the header. (note: there’s no return because you don’t need to write return in Ruby if it’s the last line of the function)

    def make_question_header(query_id)
      # id, flags, num questions, num answers, num auth, num additional
      [query_id, 0x0100, 0x0001, 0x0000, 0x0000, 0x0000].pack('nnnnnn')
    end

This is very short because we’ve hardcoded everything except the query ID.

what’s nnnnnn?

You might be wondering what nnnnnn is in .pack('nnnnnn'). That’s a format string telling .pack() how to convert that array of 6 numbers into a byte string.

The documentation for .pack is here, and it says that n means “represent it as “16-bit unsigned, network (big-endian) byte order”.

16 bits is the same as 2 bytes, and we need to use network byte order because this is computer networking. I’m not going to explain byte order right now (though I do have a comic attempting to explain it)

test the header code

Let’s quickly test that our make_question_header function works.

    puts make_question_header(0xb962) == ["b96201000001000000000000"].pack("H*")

This prints out “true”, so we win and we can move on.

step 5: encode the domain name

Next we need to generate the question (“what’s the IP for example.com?“). This has 3 parts:

• the domain name (for example “example.com”)
• the query type (for example “A” is for “IPv4 Address”
• the query class (which is always the same, 1 is for IN is for INternet)

The hardest part of this is the domain name so let’s write a function to do that.

example.com is encoded in a DNS query, in hex, as 076578616d706c6503636f6d00. What does that mean?

Well, if we translate the bytes into ASCII, it looks like this:

    076578616d706c6503636f6d00
     7 e x a m p l e 3 c o m 0

So each segment (like example) has its length (like 7) in front of it.

Here’s the Ruby code to translate example.com into 7 e x a m p l e 3 c o m 0:

    def encode_domain_name(domain)
      domain
        .split(".")
        .map { |x| x.length.chr + x }
        .join + "\0"
    end

Other than that, to finish generating the question section we just need to append the type and class onto the end of the domain name.

step 6: write make_dns_query

Here’s the final function to make a DNS query:

    def make_dns_query(domain, type)
      query_id = rand(65535)
      header = make_question_header(query_id)
      question =  encode_domain_name(domain) + [type, 1].pack('nn')
      header + question
    end

Here’s all the code we’ve written before in dns-2.rb – it’s still only 29 lines.

now for the parsing

Now that we’ve managed to generate a DNS query, we get into the hard part: the parsing. Again, we’ll split this into a bunch of different

• parse a DNS header
• parse a DNS name
• parse a DNS record

The hardest part of this (maybe surprisingly) is going to be “parse a DNS name”.

step 7: parse the DNS header

Let’s start with the easiest part: the DNS header. We already talked about how it’s 6 numbers concatenated together.

So all we need to do is

• read the first 12 bytes
• convert that into an array of 6 numbers
• put those numbers in a class for convenience

Here’s the Ruby code to do that.

    class DNSHeader
      attr_reader :id, :flags, :num_questions, :num_answers, :num_auth, :num_additional
      def initialize(buf)
        hdr = buf.read(12)
        @id, @flags, @num_questions, @num_answers, @num_auth, @num_additional = hdr.unpack('nnnnnn')
      end
    end

Quick Ruby note: attr_reader is a Ruby thing that means “make these instance variables accessible as methods”. So you can call header.flags to look at the @flags variable.

We can call this with DNSHeader(buf). Not so bad.

Let’s move on to the hardest part: parsing a domain name.

step 8: parse a domain name

First, let’s write a partial version.

    def read_domain_name_wrong(buf)
      domain = []
      loop do
        len = buf.read(1).unpack('C')[0]
        break if len == 0
        domain << buf.read(len)
      end
      domain.join('.')
    end

This repeatedly reads 1 byte and then reads that length into a string until the length is 0.

This works great, for the first time we see a domain name (example.com) in our DNS response.

trouble with domain names: compression!

But the second time example.com appears, we run into trouble – in Wireshark, it says that the domain is represented cryptically as just the 2 bytes c00c.

This is something called DNS compression and if we want to parse any DNS responses we’re going to have to implement it.

This is luckily not that hard. All c00c is saying is:

• The first 2 bits (0b11.....) mean “DNS compression ahead!”
• The remaining 14 bits are an integer. In this case that integer is 12 (0x0c), so that means “go back to the 12th byte in the packet and use the domain name you find there”

If you want to read more about DNS compression, I found the explanation in the DNS RFC relatively readable.

step 9: implement DNS compression

So we need a more complicated version of our read_domain_name function

Here it is.

      domain = []
      loop do
        len = buf.read(1).unpack('C')[0]
        break if len == 0
        if len & 0b11000000 == 0b11000000
          # weird case: DNS compression!
          second_byte = buf.read(1).unpack('C')[0]
          offset = ((len & 0x3f) << 8) + second_byte
          old_pos = buf.pos
          buf.pos = offset
          domain << read_domain_name(buf)
          buf.pos = old_pos
          break
        else
          # normal case
          domain << buf.read(len)
        end
      end
      domain.join('.')

Basically what’s happening is:

• if the first 2 bits are 0b11, we need to do DNS compression. Then:
  • read the second byte and do a little bit arithmetic to convert that into the offset
  • save the current position in the buffer
  • read the domain name at the offset we calculated
  • restore our position in the buffer

This is kind of messy but it’s the most complicated part of parsing the DNS response, so we’re almost done!

step 10: parse a DNS query

You might think “why do we need to parse a DNS query? This is the response!”. But every DNS response has the original query in it, so we need to parse it.

Here’s the code for parsing the DNS query.

    class DNSQuery
      attr_reader :domain, :type, :cls
      def initialize(buf)
        @domain = read_domain_name(buf)
        @type, @cls = buf.read(4).unpack('nn')
      end
    end

There’s not very much to it: the type and class are 2 bytes each.

step 11: parse a DNS record

This is the exciting part – the DNS record is where our query data lives! The “rdata field” (“record data”) is where the IP address we’re going to get in response to our DNS query lives.

Here’s the code:

    class DNSRecord
      attr_reader :name, :type, :class, :ttl, :rdlength, :rdata
      def initialize(buf)
        @name = read_domain_name(buf)
        @type, @class, @ttl, @rdlength = buf.read(10).unpack('nnNn')
        @rdata = buf.read(@rdlength)
      end

We also need to do a little work to make the rdata field human readable. The meaning of the record data depends on the record type – for example for an “A” record it’s a 4-byte IP address, for but a “CNAME” record it’s a domain name.

So here’s some code to make the request data human readable:

      def read_rdata(buf, length)
        @type_name = TYPES[@type] || @type
        if @type_name == "CNAME" or @type_name == "NS"
          read_domain_name(buf)
        elsif @type_name == "A"
          buf.read(length).unpack('C*').join('.')
        else
          buf.read(length)
        end
      end

This function uses this TYPES hash to map the record type to a human-readable name:

    TYPES = {
      1 => "A",
      2 => "NS",
      5 => "CNAME",
      # there are a lot more but we don't need them for this example
    }

The most interesting part of read_rdata is probably the line buf.read(length).unpack('C*').join('.') – it’s saying “hey, an IP address is 4 bytes, so convert it into an array of 4 numbers and then join those with “.“s”.

step 12: finish parsing the DNS response

Now we’re ready to parse the DNS response!

Here’s some code to do that:

    class DNSResponse
      attr_reader :header, :queries, :answers, :authorities, :additionals
      def initialize(bytes)
        buf = StringIO.new(bytes)
        @header = DNSHeader.new(buf)
        @queries = (1..@header.num_questions).map { DNSQuery.new(buf) }
        @answers = (1..@header.num_answers).map { DNSRecord.new(buf) }
        @authorities = (1..@header.num_auth).map { DNSRecord.new(buf) }
        @additionals = (1..@header.num_additional).map { DNSRecord.new(buf) }
      end
    end

This mostly just calls the other functions we’ve written to parse the DNS response.

It uses this cute (1..@header.num_answers).map construction to create an array of 2 DNS records if @header.num_answers is 2. (which is maybe a little bit of Ruby magic but I think it’s kind of fun and hopefully isn’t too hard to read)

We can integrate this code into our main function like this:

    sock.send(make_dns_query("example.com", 1), 0) # 1 is "A", for IP address
    reply, _ = sock.recvfrom(1024)
    response = DNSResponse.new(reply) # parse the response!!!
    puts response.answers[0]

Printing out the records looks awful though (it says something like #<DNSRecord:0x00000001368e3118>). So we need to write some pretty printing code to make it human readable.

step 13: pretty print our DNS records

We need to add a .to_s field to DNS records to make them have a nice string representation. This is just a 1-line method in DNSRecord:

      def to_s
        "#{@name}\t\t#{@ttl}\t#{@type_name}\t#{@parsed_rdata}"
      end

You also might notice that I left out the class field of the DNS record. That’s because it’s always the same (IN for “internet”) so I felt it was redundant. Most DNS tools (like real dig) will print out the class though.

and we’re done!

Here’s our final main function:

    def main
      # connect to google dns
      sock = UDPSocket.new
      sock.bind('0.0.0.0', 12345)
      sock.connect('8.8.8.8', 53)

      # send query
      domain = ARGV[0]
      sock.send(make_dns_query(domain, 1), 0)

      # receive & parse response
      reply, _ = sock.recvfrom(1024)
      response = DNSResponse.new(reply)
      response.answers.each do |record|
        puts record
      end

I don’t think there’s too much to say about this – we connect, send a query, print out each of the answers, and exit. Success!

    $ ruby dig.rb example.com
    example.com   18608   A   93.184.216.34

You can see the final program as a gist here: dig.rb. You could add more features to it if you want, like

• pretty printing for other query types
• options to print out the “authority” and “additional” sections of the DNS response
• retries
• making sure that the DNS response we see is actually a response to the query we sent (the query ID has to match!

Also you can let me know on Twitter if I’ve made a mistake in this post somewhere – I wrote this pretty quickly so I probably got something wrong.

---

via: https://jvns.ca/blog/2022/11/06/making-a-dns-query-in-ruby-from-scratch/

作者：Julia Evans 选题：lujun9972 译者：译者ID 校对：校对者ID

本文由 LCTT 原创编译，Linux中国 荣誉推出