Mikrotik (6.0rc14) DNS bug (TCP/53 vs. >512-byte answers)

  • Problem:

Mikrotik (6.0rc14) DNS accepts ‘truncated’ DNS responses as answer instead of retrying the query via TCP.

  • Symptoms:

When Mikrotik is configured to use Ziggo DNS (213.51.129.37), com-services.pandonetworks.com is not resolvable.
When Mikrotik is configured to use UUnet DNS (193.79.242.39), com-services.pandonetworks.com is resolvable.

When Ziggo or UUnet is used directly from the client, bypassing the mikrotik, there is no problem.

  • Explanation:

Contrary to Windows and Linux, Mikrotik does not retry a DNS query over TCP when it receiver an answer with ‘tc’ (truncated) flag set.
It will use the answer and forward it to the client.

I believe that threads like http://forum.mikrotik.com/t/mikrotik-dns-server-issues-with-amazon-s3-low-ttl-60sec/53196/1 are rooted in this problem.

But whether or not this difference will cause an issue for you depends on the behavior of your upstream DNS resolvers.

When resolvers need to reply a >512-byte answer to a DNS query,

type-A will respond with 0 answers and ‘truncated’ flag, resulting in the following mikrotik log:

May  5 02:15:39 dns,packet --- got query from 192.168.1.254:64887:
May  5 02:15:39 dns,packet id: 2  rd 1  tc 0  aa 0  qr 0  ra 0  QUERY 'no error'
May  5 02:15:39 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 02:15:39 dns query from 192.168.1.254: #4106 com-services.pandonetworks.com. A
May  5 02:15:39 dns,packet --- sending query to 213.51.129.37:53:
May  5 02:15:39 dns,packet id: b2b8  rd 1  tc 0  aa 0  qr 0  ra 0  QUERY 'no error'
May  5 02:15:39 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 02:15:39 dns,packet --- got answer from 213.51.129.37:53:
May  5 02:15:39 dns,packet id: b2b8  rd 1  tc 1  aa 0  qr 1  ra 1  QUERY 'no error'
May  5 02:15:39 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 02:15:39 dns done query: #4106 dns name exists, but no appropriate record
May  5 02:15:39 dns,packet --- sending reply to 192.168.1.254:64887:
May  5 02:15:39 dns,packet id: 2  rd 1  tc 0  aa 0  qr 1  ra 1  QUERY 'no error'
May  5 02:15:39 dns,packet question: com-services.pandonetworks.com.   A   IN

type-B will respond with >0 answers and ‘truncated’ flag.

May  5 01:58:35 dns query from 192.168.1.254: #3944 com-services.pandonetworks.com. A
May  5 01:58:35 dns,packet --- sending query to 193.79.242.39:53:
May  5 01:58:35 dns,packet id: 92b1  rd 1  tc 0  aa 0  qr 0  ra 0  QUERY 'no error'
May  5 01:58:35 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 01:58:38 dns,packet --- got answer from 193.79.242.39:53:
May  5 01:58:38 dns,packet id: 84ad  rd 1  tc 1  aa 0  qr 1  ra 1  QUERY 'no error'
May  5 01:58:38 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 01:58:38 dns,packet answer:
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.111.219
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.182.155
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.119.16
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.26.189
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.214.210
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.206.161
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.30.40
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.229.197
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.102.245
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.197.135
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.72.13.132
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.112.55
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.159.90
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.42.109
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.200.125
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.214.33
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.222.22
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.188.191
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.235.79
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.110.17
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.153.41
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.214.26
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.125.183
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.188.170
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.10.23
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.127.117
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 208.78.158.8
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.118.22
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.189.231
May  5 01:58:38 dns done query: #3944 com-services.pandonetworks.com 54.215.111.219
May  5 01:58:38 dns,packet --- sending reply to 192.168.1.254:59071:
May  5 01:58:38 dns,packet id: 2  rd 1  tc 0  aa 0  qr 1  ra 1  QUERY 'no error'
May  5 01:58:38 dns,packet question: com-services.pandonetworks.com.   A   IN
May  5 01:58:38 dns,packet answer:
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.111.219
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.182.155
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.119.16
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.26.189
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.214.210
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.206.161
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.30.40
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.229.197
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.102.245
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.197.135
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.72.13.132
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.112.55
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.159.90
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.42.109
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.200.125
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.214.33
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.222.22
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.188.191
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.235.79
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.110.17
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.153.41
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.214.26
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.241.125.183
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 184.169.188.170
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 50.18.10.23
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.127.117
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 208.78.158.8
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 54.215.118.22
May  5 01:58:38 dns,packet com-services.pandonetworks.com. A 600: 204.236.189.231

Most likely Mikrotik has not been able to reproduce this problem, because he has a type-B resolver.

I have made a packetcapture while manually querying 3 example DNS resolvers to illustrate extra.

URL: http://cloudshark.org/captures/67988e7ba517

Type-A: Ziggo (Dutch ISP) - 213.51.129.37
Type-A: Google Public DNS - 8.8.4.4
Type-B: UUnet/Worldcom - 193.79.242.39

Ziggo: (nslookup com-services.pandonetworks.com 213.51.129.37)
Packet #3: UDP/53 request to 213.51.129.37
Packet #4: 213.51.129.37 responds with 0 answers and ‘truncated’ flag set.
Packet #5-18: TCP/53 DNS request resulting in 31 answers.

Google Public DNS: (nslookup com-services.pandonetworks.com 8.8.4.4)
Packet #21: UDP/53 request to 8.8.4.4
Packet #22: 8.8.4.4 responds with 0 answers and ‘truncated’ flag set.
Packet #23-36: TCP/53 DNS request resulting in 31 answers.

UUnet/Worldcom: (nslookup com-services.pandonetworks.com 193.79.242.39)
Packet #39: UDP/53 request to 193.79.242.39
Packet #40: 193.79.242.39 responds with 29 answers and ‘truncated’ flag set.
Packet #41-54: TCP/53 DNS request resulting in 31 answers.

  • Solution:

When receiving a DNS response with ‘truncated’ set, Mikrotik should not forward this answer to the client, but instead retry the query over TCP.

+1 the exact issue i get regularly. It appears most New Zealand isp dns servers respond with TC flag…

Obviously this is only an issue if you are using the Mikrotik as a DNS proxy.

Please email support@mikrotik.com with the issue. The forums are a user forum, official support will take care of it.

Support are well aware of this issue. I have had an open ticket with them for several months now.