Community discussions

 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

MikroTik ignoring DHCP Discover Requests from devices

Fri Sep 20, 2019 9:38 pm

I'm experiencing some peculiar issues on 6.40.9 (we haven't upgraded just yet. We definitely plan to.) But as it stands MikroTiks are ignoring DHCP discovers. Granted the devices don't even bother asking for DHCP until we run an IP scan via the MikroTik (BOOTP Request.) Any ideas as to why the MikroTik might be ignoring these?

See attached packet captures for further insight. On the LarrkMTK.pcap you can see the device via the filter eth.addr==94:57:a5:4b:de:f8 is trying to discover via DHCPv6 before we reboot the AP. Our issues look to be pretty wild. Help is greatly appreciated!
You do not have the required permissions to view the files attached to this post.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1435
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: MikroTik ignoring DHCP Discover Requests from devices

Sat Sep 21, 2019 12:13 am

I'm experiencing some peculiar issues on 6.40.9 (we haven't upgraded just yet. We definitely plan to.)

Surely the above should be the first step before asking help?
MTCNA, MTCTCE, MTCRE & MTCINE
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Tue Sep 24, 2019 9:49 pm

I'm experiencing some peculiar issues on 6.40.9 (we haven't upgraded just yet. We definitely plan to.)

Surely the above should be the first step before asking help?
We've upgraded it on a few MikroTiks. It possibly fixed it; as the problem disappeared. But so has rebooting the equipment in other cases so it doesn't appear consistent. We are looking to replicate the problem so we can find the silver bullet we need that consistently fixes the problem. Any ideas as what is causing the MikroTik to ignore DHCP discover requests? We have a significant number of these deployed so pitching "upgrade them all" without the smoking gun is going to look dubious at best. We have plans to upgrade them but this would be an urgent push we'd need to send out as it is service impacting.

Attached a few more packet captures from other devices.
You do not have the required permissions to view the files attached to this post.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1435
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: MikroTik ignoring DHCP Discover Requests from devices

Fri Sep 27, 2019 8:26 pm

without seeing the config, my guess would be there are no DHCP service running on VID 316
MTCNA, MTCTCE, MTCRE & MTCINE
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 7:50 pm

In our deployments, we are experiencing significant DHCP issues. These usually occur after RADIUS outages; we can’t be entirely sure, but the common symptom is that the DHCP server function of our MikroTiks ceases to work as intended. The most common method to initiate communication between the device and the MikroTik is using the IP scan tool to send a BOOTP via DHCP to all possible clients off of the ether5 interface.

We’ve attempted to simulate RADIUS outages by placing a firewall between a test controller and the RADIUS server to filter RADIUS traffic, and we’ve confirmed with our third-party group who supports our controllers that we are filtering all traffic between their equipment and the RADIUS server. Sadly this hasn’t replicated the problem in our lab, and we cannot work toward creating a solution without figuring out what the exact cause is. Below is a summary of some of our efforts thus far:

Checked the provided PCAP for BOOTP and saw this single packet that is sent by the router to synchronize clients:
1550 10.779919 255.255.255.255 255.255.255.255 BOOTP 288 Boot Request from b8:69:f4:a5:8b:38 (Routerbo_a5:8b:38)

(((See attached bootp.jpg image)))

We accessed the test AP and verified this packet appears on both the Ethernet WAN interface (eth0) and the WLAN interface where you had clients (wlan32)

# tcpdump -i eth0 -vvv | grep -i bootp
255.255.255.255.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from b8:69:f4:9a:70:8c, length 246, xid 0x8, Flags [none] (0x0000)

# tcpdump -i wlan32 -vvv | grep -i bootp
255.255.255.255.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from b8:69:f4:9a:70:8c, length 246, xid 0x8, Flags [none] (0x0000)

Checked the PCAP taken on the routers ethernet 5 interface for a client that was not getting a valid IP but reporting the APIPA 169.x IP and we see the DHCP discovery packets on the correct VLAN 227 but no replies from the router or DHCP server:


(((See Attached DHCP.jpg and Tell0000.png image)))



Based on the packet capture analysis, we need to focus on digging deep on MikroTik DHCP function.
We currently don’t see any external factors heavily contributing to this issue.
Capture has traffic on two VLAN’s: 50 and 108 from two clients concurrently.
• Client on VLAN 50 – struggling to get an IP address from respective DHCP-50/VLAN-50/Unauthenticated pool
- AP is doing good with a packet received from Client to network, properly tagged (VLAN 50). However, don’t see any packet coming back from MikroTik on VLAN 50 ( except LLDP_Multicast)
- Could be four reasons:
o MikroTik DHCP server app didn’t receive the discovery packet
o MikroTik DHCP server app received the discovery packet but dropped
o MikroTik DHCP server app received the discovery packet, but failed to send back with proper VLAN TAG/50
o MikroTik Transmit inactive state

• Client on VLAN 108 – exchanging bidirectional traffic without any issue, while Client on VLAN-50 struggling with DHCP discovery retransmits.

Moreover, it makes some correlation with magic for resolving the issue using IP-SCAN or reboot of MikroTik, probably bringing that stranded logical interface or DHCP Server process.

Below is a snapshot of what we believe is happening.

(((See attached simu.png image)))

Attached are the packet captures of two separate devices captured off of their ether5 interfaces respectively.

Below is our current lab setup:
The typical MikroTik deployment consists of a MikroTik acting as gateway for a WattBox PDU, Ruckus Access Point, and client devices, and it is usually bridged to a modem; which receives DHCP from a node. The Access Points are managed via a vSZ (virtual SmartZone) controller; which send their configurations, and handle a lot of the RADIUS process.

(((See attached lab.png image)))
You do not have the required permissions to view the files attached to this post.
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 7:58 pm

See Attached:
You do not have the required permissions to view the files attached to this post.
 
Exiver
Member Candidate
Member Candidate
Posts: 113
Joined: Sat Jan 10, 2015 6:45 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 8:12 pm

From an external view your setup is relatively complex. This means without seeing your configuration it would be just a guess into the blue. And that does not help at all since it just wastes your and our time ;-) If you expect help - post your complete configuration.
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 8:18 pm

From an external view your setup is relatively complex. This means without seeing your configuration it would be just a guess into the blue. And that does not help at all since it just wastes your and our time ;-) If you expect help - post your complete configuration.
Please see the attached files earlier. example.rsc is an export of the configuration of one of our deployed MikroTiks.

I'll re-attach here.
You do not have the required permissions to view the files attached to this post.
 
Exiver
Member Candidate
Member Candidate
Posts: 113
Joined: Sat Jan 10, 2015 6:45 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 8:57 pm

Im sorry - i have missed your configuration.

It looks like you have set "authorative=after-2sec-delay" to your client vlans. This means the router will ignore all dhcp-requests from a device if the requests are not coming in bigger time intervals than 2 seconds. If we look at your screenshots it seems like the clients take more than 2 seconds to re-request DHCP. Can you please try to set authorative to "after-10sec-delay" and check whether this solves your problem? If that doesnt help i would try to remove that option and recheck whether its working then.

If that does not help as well i would suggest to start with the mikrotik packet sniffer and check whether the dhcp request arrives at your router. If this is the case the packet is at least not lost on its way.
Last edited by Exiver on Wed Oct 02, 2019 11:46 pm, edited 1 time in total.
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Mon Sep 30, 2019 9:13 pm

Im sorry - i have missed your configuration.

It looks like you have set "authorative=after-2sec-delay" to your client vlans. This means the router will ignore all dhcp-requests from a device if the requests are not coming in smaller time intervals than 2 seconds. If we look at your screenshots it seems like the clients take more than 2 seconds to re-request DHCP. Can you please try to set authorative to "after-10sec-delay" and check whether this solves your problem? If that doesnt help i would try to remove that option and recheck whether its working then.

If that does not help as well i would suggest to start with the mikrotik packet sniffer and check whether the dhcp request arrives at your router. If this is the case the packet is at least not lost on its way.
Absolutely. I'll give it a go when I get into another affected MikroTik. It'll be interesting to see what the pcaps look like if this does fix it.
 
Zetera
Frequent Visitor
Frequent Visitor
Topic Author
Posts: 52
Joined: Sat Apr 21, 2018 6:57 pm

Re: MikroTik ignoring DHCP Discover Requests from devices

Tue Oct 01, 2019 11:08 pm

Sadly it doesn't appear that adjusting the authoritative delay in any direction made a difference. Do we happen to have any other suggestions that I can test out?

A temporary fix appears to be sending a boot request via DHCP using he IP-Scan tool on the MikroTik then rebooting the device. But I'm definitely looking to test a more permanent fix; if possible a way to replicate this issue on lab equipment would be awesome.
 
User avatar
CZFan
Forum Guru
Forum Guru
Posts: 1435
Joined: Sun Oct 09, 2016 8:25 pm
Location: South Africa, Randburg
Contact:

Re: MikroTik ignoring DHCP Discover Requests from devices

Wed Oct 02, 2019 11:36 pm

Have you applied at least 6.44.5 long term version?

6.40.9 is fairly old, and not sure why time is wasted on that
MTCNA, MTCTCE, MTCRE & MTCINE

Who is online

Users browsing this forum: No registered users and 105 guests