Another DHCP question

RouterOS General
1

Preface with saying that I’ve searched for days with no resolution. The closest thing I could find to my issue was in this link
http://forum.mikrotik.com/t/dhcp-server-on-bridge-interface-issue/136970/1
And at the end, I tried what they did. Tore my DHCP servers apart and rebuilt them from WinBox instead WebFig.
I’ll also preface by saying I’ve been around networks for a while, so while my skillset is beginner at best, I have a general understanding of fundamentals (I think). Started to take the CCNA, past the first test, then life work and all things other prevented me from ever going back (4 years ago)

The setup I’m using (not all inclusive) is the RB4011, Netgear/Xfinity on port 1, DAC on SFP to Cisco 3560X (10g module installed). Two Pi’s, one of which is PiHole (the other is PiKVM (jury still out on that)).

What I’m after
The RB4011 to be used only as a router, firewall and DHCP server for my VLans (stick with me here, I don’t think this is a typical question, but maybe). All my devices are on the Cisco and as of now, I plan to stay that way. Don’t have intentions of using the ports on the RB unless a need arises. The 3560X is a 48PoE+ switch, so I have more than enough. And if I really need more ports, I have an older 3560E 48PoE sitting in a box (I say older as if the X is new lol )

The problem
The 4011 DHCP isn’t handing out addresses (again, stick with me)

I’m fully aware of RoaS, but that’s not what I’m looking to do.

The Cisco is working perfectly. All my devices talk to each other (haven’t done anything about blocking vlan traversal as of yet). Because I couldn’t get the 4011 to hand out addresses, the 3560X is currently pulling DHCP duty. Works great and all, but, for me, pulling DHCP information is a bit of a PITA. Very doable, and I can get everything I want, but I very much prefer to grab all my analytics from something like the MT GUI (WinBox, WebFig, either or) than logging into the switch and digging my way through while looking for something (eg; IP assigned to a freshly added headless device, etc). So, bluntly, it’s a preference thing. Not a OMG my stuff is broken and I need help! Every one can get out on the internet and do what they need, so all is right in the world (well, sort of, but that’s for a different forum).

I’ve tried putting the IP helper address in, and it’s definitely doing it’s job. Using Packet Sniffer on the RB, I can see the packets coming in and hitting the IP/DHCP I setup, just the server NEVER responds. In coming packets count goes up, responding packet count stays and a nice solid, circular 0 (guess that’s more oblong).

I’ve tried all combinations that make sense to me. Doesn’t mean I’ve tried them all, just everything that looks like it should work. Crumb and crackered my DuckDucks until I had no crackers and crumbs left. Tried every solution that I’ve found on the nets that was even remotely close to what I’m trying to do.

I’ve tried creating VLans, even though I’m not trunking (uplink port Cisco side is configured as a routed “no switchport” port). Put IP’s on them, put addresses on them, created the pools, networks and servers. Put the servers on the vlan interface, the SFP port itself, even created a bridge to just put the DHCP’s on.

My i’s are crossing and my t’s have all lost their dots.

I see RoaS posts all over the place, so, that leads me to believe
1- There’s not solution so everyone goes the RoaS way
2- Not many try and do what I do
D- I’m bashing my face into a wall for no reason.

In all seriousness, I don’t want anyone to spoon feed me the answer, and if there is a post I missed, just tell me where it is. I learn best by flipping switches and pushing big red buttons, but I think I’ve push flipped everything? If it can be done, if you could just point me in the right direction with a “Yeah, it’s been done.” and then tell me where I need to/should be looking, the area or thing I should be paying more attention to. That would be fee-nom-nom-a-nal.

If I can’t do it, my next solution to try will be to go the route of maybe throwing another Pi in the rack and standing up some kind of DHCP. That’s not my preferred method, but I’ll give that a go. I really want to keep all the switching/heavy lifting on the Cisco and have my stuff and household hit the router only when needed.

I haven’t tossed any config code up because I’m not sure what all anyone would want to look at? All of it (minus sensitive stuff)? Just the DHCP setup? The blonde next door?

Really appreciate anyone who looks at this and anyone who feels up to helping an old fart trying to learn a new dog.

Swany

2

Small update - As I still can’t get anywhere with using the RB4011 as a remote DHCP server, I decided to try and use my Unifi Controller as a temp solution until I can find something that works (or give up and just go RoaS). Didn’t work. Same issue. This time though, the request doesn’t have to traverse from the switch to the router as the controller is on the switch (on a VM). Same thing, I can ping it, it can ping other things, it just wont issue IP’s. I’m beginning to wonder if something is blocking the DHCP request or reply. I’m researching, and so far, everything I’m finding says it should be working, yet it’s not.

Still looking…

edited
Didn’t even think about just doing a screen cap. I’ve attached a screen cap of the relative parts. If I’m missing something someone wants to see, let me know. I’m also attaching, for those that Cisco, my Cisco setup. If you see something wrong, let me know. I do believe I’ve scrubbed the important stuff out. If you spot something I should have yanked, let me know and I’ll fix, but again, I think I got it.

There is one change I’ve made that isn’t reflected in the Cisco config - The ip helper no longer points to the IP 172.23.1.165, currently it’s pointing to the router interface as 172.23.151.2 - I had placed the DHCP on the interface directly, as suggested to me by someone else

edited to add RB4011 config
RB4011_Config.txt (8.62 KB)
CURRENT CONFIG.txt (6.31 KB)
CURRENT ROUTE.txt (986 Bytes)
CURRENT VLAN.txt (2.62 KB)
MIKROTIK DHCP.jpg

3

Learned a new skill - How to use the log. Still deciphering what it means, but when I try to get a IP lease from the MikroTik, I get the following

received discover from 8C:AE:4C:D6:33:36 with unknown giaddr 172.23.165.25

http://forum.mikrotik.com/t/dhcp/21154/11

But I’m not sure how to add 255.255.255.255 to the relay? As any time I try, I am hit with an error

edit
I was using the wrong “relay”. I added the 255.255.255.255 to the DHCP relay and it started handing out IP’s. The log showed it was my test laptop, which it should have been, but it never received the IP, so it kept asking until all IP’s were exhausted.
When checking the ARP table/cache on the Cisco, I see lines for each of the IP’s, listed as incomplete. I flush the ARP table, they go away. As soon as the laptop starts trying to ask for IP’s again, the 4 in the vlan get used up on the MT side, and the incomplete in the arp table on the Cisco comes back
Repeat Mikrotik.jpg

4

A little more progress

If I go into the DHCP server, set the MAC from the laptop to a predefined IP address (guess you call it static by DHCP?) it gets to the laptop without issue. I can ipconfig /release, then /flushdns, then /renew and it pulls it again, no problems. If I go back to dynamic, it fails to get through again.

It has something to do with the DHCP server pinging the next avail IP. As soon as it does that, the ARP table on the Cisco 3560X puts an entry in for that IP, listed as “Incomplete”. I base this assumption on the error in the 4011 log stating

ping
"Detected conflict by ARP response for from
ping done

5

Your last post indicates that there might be a device in your network (the cisco switch) configured to perform proxy ARP? That function can kill DHCP and its check if IP address is unused.

Which device is that MAC address mentioned in warning?

6

Post your RB4011 config,
and how does the RB4011 pass the vlans to the switch??

/export hide-sensitive file=anynameyouwish

7

mkx-
There’s no ARP device turn on on the router that I’m aware of. I’ll go digging for info on Cisco about ARP Proxy (didn’t know that was a thing, thank you). The MAC it’s listing the warning from is the Gateway for the VLan (165) on the Cisco switch. So the SVI it’s what’s throwing back the message I guess?

anav-
I posted a picture above of my DHCP config, but I will work on getting the export you asked for posted as quickly as I can.
I’m not passing VLan’s to the router. I have no VLan setup on the router as I am not using RoaS. The routers only functions (if I can get it all working) will be to route traffic for the internet, firewall, and DHCP services. All VLan routing is being done on the Cisco 3560X. On the Cisco, I’m using it’s relay, “ip helper-address x.x.x.x” to point to the router interface. Everything about this current configuration is working except the ability to assign dynamic IPs. Internet traffic is working for everything else on my network (I’m typing this message from my desktop, which is connected to the Cisco, routing through the RB4011, the only difference is that this machine is getting it’s IP from the Cisco itself due to the issues from the MikroTik).

And as from my previous post, if I tell the RB4011 to take the MAC address of my test laptop, reserve an IP for it, when the laptop requests an IP, it works. It has something to do only with the dynamic process.

I found a post from back in 10? Maybe 11? Where some people were saying MikroTik replaced their routers as broken, receiving the same ARP error message. I have a feeling that’s not it.

8

Added RB4011 config to second post

Thank you both for taking the time to look at this. I feel I am so close, just need to find that last little hiccup in my config. I really do appreciate your time.

9

The RB4011 config makes no sense to me.

(1) For example you have (besides ether1) disabled all the ethernet port EXCEPT sfp+ port
Then on bridge ports you disabled the sfp+ port while the rest are ‘active’

???


(2) Even if the RB4011 is not assigning/creating vlans or DHCP etc…
you still need to define each vlan and use interface bridge.
Ensure the ethernet port and bridge port ARE NOT disabled for that particular port.

(3) You dont need addresses for vlans or dhcp either…

(4) You have no route to the internet as the only route I see is disabled.

(5) Would recommend rp-filter be set to loose vice strict (but only a single WAN maybe not a concern??

(6) Never seen this rule used, why is it there??
dd action=accept chain=input comment=DHCP in-interface=sfp-sfpplus1
protocol=udp src-port=67-68

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

In summary, I dont quite understand what you are trying to accomplish with the RB4011 and thus the advice above is not certain…

10

(1) For example you have (besides ether1) disabled all the ethernet port EXCEPT sfp+ port
Then on bridge ports you disabled the sfp+ port while the rest are ‘active’

I disabled all the ports physical interfaces because I’m not using them. I didn’t turn them off at the bridge because there is no access to them, so did not see the point in it. I can understand it being a best practice, shut it down here, so then shut it down there. I just didn’t, no good reason one way or the other

As for the SFP+ and eth0, 0 leads to the modem/internet and the SFP+ is the link between the RB4011 and the the Cisco 3560X. They need to be at L3, not L2. It’s a routed port, not a switch port. If I go L2, I need to trunk the connection, switch the Cisco 3560X to sub interfaces instead of SVI’s and use the RB4011 as the router/switch for inner VLan. RoaS; not what I want to do. I want all the switching for the VLans to be done on the Cisco. The RB4011’s only function is to route traffic to and from the internet, be a firewall between my network and the internet, and, if I can get it working, be my DHCP server for my VLans. Nothing more.


(2) Even if the RB4011 is not assigning/creating vlans or DHCP etc…
you still need to define each vlan and use interface bridge.
Ensure the ethernet port and bridge port ARE NOT disabled for that particular port.

Why would I need VLans on the RB? The traffic coming through the SFP+ port is untagged; it has no VLan information. I can see the need if I was to trunk, but to trunk would lead to RoaS and again, avoiding that.


(3) You dont need addresses for vlans or dhcp either

I discovered the no need for IP’s on the DHCP last night. Part of the test when I put the broadcast in the DHCP relay was to remove the IP from the DHCP.


(4) You have no route to the internet as the only route I see is disabled.

Not true, I do (see attached image showing all routes learned through OSPF, public IP redacted). Traffic to and from the internet works perfectly. As with all previous messages, I’m on this forum typing this message from a system that is attached the Cisco. The only difference is this machine has been given an IP from the Cisco DHCP server built into the switch (which I don’t want to use as it’s a very simplified server and does not fit my needs)


(5) Would recommend rp-filter be set to loose vice strict (but only a single WAN maybe not a concern??

I am unfamiliar with the rp-filter, so I will research that. Thank you.


(6) Never seen this rule used, why is it there??

I know DHCP does not need any firewall rules to work. This rule is only in place to capture packet/byte count. Something I put on a bit back for troubleshooting/trying to debug this problem.


Overall, everything is working as it should on my network. All routing, all my VLans on the switch and inner-vlan switching on the Cisco. Communication from the Cisco to the MikroTik. From the farthest connected device that has to go through two switches (the Cisco being one) all the way through the to the internet. The ONLY thing that does not work is Dynamic IP assignments from the RB4011. If I tell it to reserve X IP for Y MAC address, the laptop I’ve been using, in the VLan I’ve been testing with, works phenomenally. It’s ONLY when it’s set to Dynamic it fails to get an IP back to the Cisco (see screen capture in post #3).

Over all, what I’m asking the router to do is a very simple task and should not be so difficult to do. Something large companies do all day long. Set up a DHCP server and have everyone point to it in order to use it. An every day task. Except, for some reason, the RB can’t seem to do it. There is nothing odd about my configuration. It may not be “Main Stream” or the way every HomeLab is setup, but neither is it uncommon.

I submitted a trouble ticket to MT because, at this point, while I am no expert (nor do I profess to be one, which is why I’m here seeking advice), if I am understanding the documentation I’ve read on the MT and the Cisco (and, as for how this function works on the Cisco, it really is a single command, ip helper-address x.x.x.x, so, kind of hard to mess that up so long as you type the correct IP), and even my buddy, whom I found out this morning tried to do this exact setup himself, only to give up, and it would be fair to call him an engineer (not his title, but for the work he does, it will suffice, and I wont talk about who/what/where he works). So, this leads me to believe that while I may a little off my nogg’n, I DO have everything setup right (reinforced by the fact that, again, if I reserve an IP for said test environment it works perfect, just the dynamic, generic broadcast is failing), that there is something not working correctly.

Unfortunately, I believe without even looking at the notes, they pretty much told me to go pound sand and hire someone unless I’m having a hardware problem or reporting a bug (to which I replied exactly, I’m reporting what I believe to be a bug).
ROUTES.jpg

11

I tried setting rp-filter to none as a test - no change, still no IP making it through. Thank you for the suggestion though

12

Ok, paid a little more attention to the logs on the Mikrotik - It doesn’t transmit the IP when under dynamic because it receives a message from the Cisco saying, I believe, that that IP is in it’s ARP table. The listings in the ARP table are all listed as “Incomplete”.


It assigns the IP when I reserve it because it never “pings” the IP, so it never receives the message from the Cisco regarding the ARP table. I’m not sure if there is a way to tell the Mikrotik to ignore the ARP or to get Cisco to not send the message

edit
It all comes down to that ARP. If I let it assign by reservation, then remove the reservation, it will request the same IP again, so the message goes through, no ping/ARP look up. Flush it from the laptop and we’re back at the dynamic ping/ARP

As a “band-aid” I can get it to work dynamically if, on the Mikrotik side, I turn off “Conflict Detection”. While this can be used, temporarily, as a work around, I would think this is a bad thing, especially if in larger environments.


I am still going to work towards a better solution.

13

Looks as if I spoke too soon. It did start handing out IPs, but when I added other pools to the DHCP server, it just kept handing out the same one over and over, even if I flushed the dns from the laptop. So I am back at square one, and I honestly don’t know how much more energy I can spend on this.

14

I now have a 95% satisfactory resolution.

Configure all VLan’s and VLan SVI’s on the Cisco

On the Cisco, set the ip helper-address to the router interface on the SFP+ link

On the MikroTik build a DHCP server for each VLan you want to pull an IP

In the Relay option within the DHCP server options, set it to the gateway IP of the VLan on the Cisco. When the Cisco forwards the dhcp broadcast request, it sends it with the VLan gateway IP as the return. Which, if you only have one VLan/Network, you can actually set it to the broadcast, 255.255.255.255. If you have 2 or more, you need to use the gateway IP (I hope I’m explaining this well)

The important part (at least for me), and why it’s only 95% and not 100% - You need to uncheck the Conflict Detection option. For me, it’s what was causing the pools to exhaust because it was thinking the IP/APR was in use. Once I did that, it would hand out IP’s. This can lead to issues down the road when needing to troubleshoot because if a system has an ip of 192.168.1.10 for whatever reason (static set, assigned by DHCP) and your DHCP isn’t tracking it (reserve the IP on the DHCP server, DHCP server reboots, loses all track of IPs assigned, etc), it will just gladly hand out the same IP address again.

Until I can figured out what exactly is going on with the ARP messages, this will suffice for now. It gives me my GUI so I can see everything easily at a glance, will allow me to plug in new devices and let them be discovered, etc. For my use case, this solution is an acceptable temporary solution as most of my devices will end up with statics (but not all) and I can reserve those off to the side in DHCP or just not allow them into the pool again.

Again, not a 100%, but a good enough for now (especially given all the time I’ve already put in to this issue and ignored other things going on that need my focus, both network and home)

Thank you guys for trying to help. It honestly was and is appreciated.