mikrotik routing between two dhcp servers

[stefki]

Hello all, I have mikrotik router CRS125-24G-1S which is very nice device. Now I have setup two DHCP servers and two WAN connections. I am not using any load balancer or failover only two Independent WAN and two dhcp networks.
this is short example
Home PC connected to DHCP server network 192.168.1.0/24
Work PC is connected to DHCP server 192.168.2.0/24
HomePC ip is 192.168.1.2
Work PC ip is 192.168.2.93

WAN1 interface =ether1-wan1
WAN2 interface = wan2
Both pcs have internet but they can’t communicate each other.
When I try to ping each other , ping no reply no communication between them at all.
Also both pc are not able to ping the default gateway 192.168.1.1
this is my full setup config

If someone experienced user can give me some help and hints, thank you all the best.

/interface pppoe-client
add add-default-route=yes default-route-distance=10 disabled=no interface=ether24 name=wan2 password=xxx user=xxxx

/ip pool
add name=dhcp_pool1 ranges=192.168.1.2-192.168.1.254
add name=dhcp_pool2 ranges=192.168.2.2-192.168.2.254

/ip dhcp-server
add address-pool=dhcp_pool1 authoritative=after-2sec-delay disabled=no interface=bridge1 lease-time=1w name=dhcp1
add address-pool=dhcp_pool2 disabled=no interface=dhcp2-brdige name=dhcp2

/interface bridge port
add bridge=bridge1 interface=ether3
add bridge=bridge1 interface=ether4
add bridge=bridge1 interface=ether5
add bridge=dhcp2-brdige interface=ether17
add bridge=dhcp2-brdige interface=ether18

/ip firewall connection tracking
set enabled=yes

/ip address
add address=213.xx.xx.xx/24 interface=ether1-wan1 network=xx.135.xx.0
add address=192.168.1.1/24 interface=bridge1 network=192.168.1.0
add address=192.168.2.1/24 interface=dhcp2-brdige network=192.168.2.0

/ip dhcp-server network
add address=192.168.1.0/24 dns-server=x.135.xx.18,xx.xx.160.19 gateway=192.168.1.1
add address=192.168.2.0/24 dns-server=xx.17.xx.5,81.xx.225.5 gateway=192.168.2.1

/ip dns
set servers=xx.xx.xx.18,xx.xx.xx.19

/ip firewall filter
add action=fasttrack-connection chain=forward connection-nat-state="" connection-state=established,related out-interface=ether1-wan1
add action=fasttrack-connection chain=forward connection-nat-state="" connection-state=established,related out-interface=wan2
add action=accept chain=forward connection-nat-state="" connection-state=established,related

/ip firewall mangle
add action=mark-connection chain=input in-interface=wan2 new-connection-mark=internet2 passthrough=no
add action=mark-routing chain=output connection-mark=internet2 new-routing-mark=ruta-internet2 passthrough=no

/ip firewall nat
add action=dst-nat chain=dstnat  dst-port=9166 in-interface=ether1-wan1 protocol=tcp to-addresses=192.168.1.2 to-ports=9166
add action=dst-nat chain=dstnat  dst-port=2251 in-interface=wan2 protocol=tcp to-addresses=192.168.2.93 to-ports=22

add action=masquerade chain=srcnat out-interface=ether1-wan1 src-address=192.168.1.0/24
add action=masquerade chain=srcnat out-interface=wan2 src-address=192.168.2.0/24

/ip route
add distance=10 gateway=wan2 routing-mark=ruta-internet2
add distance=1 gateway=ether1-wan1 routing-mark=internet1
add distance=10 gateway=wan2 routing-mark=internet2
add distance=1 gateway=ether1-wan1

/ip route rule
add src-address=192.168.1.0/24 table=internet1
add src-address=192.168.2.0/24 table=internet2

Thank you.

[Zacharias]

Both subnets should be able to communicate unless blocked…
From a quick look in your config i dont see something wrong…

[mkx]

Just brain-storming: does routing table internet1 have any route for destination 192.168.2.0/24 ? And ditto for routing table internet2 and 192.168.1.0/24 … I’m not sure if direct routes are used automatically when one starts with multiple routing tables …

[Zacharias]

@mkx, normally, no extra routes are needed…
Also as OP says, even the PC on 1.0/24 cannot ping the gateway 1.1…
There must be something else…

[stefki]

@mkx: I don’t have additional routes added.

[Zacharias]

@mkx: I don’t have additional routes added.

Not needed anyways for two directly connected networks…

[Sob]

The rule is clear, if the source is 192.168.1.0/24, route will be looked up in table internet1 and it contains only one default route going to ether1-wan1. You need another rule before those two, e.g.:

/ip route rule
add dst-address=192.168.0.0/16 src-address=192.168.0.0/16 table=main

[Zacharias]

@sob wait because i am totally confused now… since when do i need a route to communicte with the gateway IP of my own subnet ?
Also, if you setup two subnets in the same router, unless blocked, those subnets can totally reach each other with no extra routes or anything…

Maybe i am confused with the route rules… what is their purpose ? Never used them..

[Sob]

Routing rules are mighty tool. They work in prerouting phase (both prerouting and output) and instruct router where exactly it should look for route.

In this case, everything is fine for incoming packet. It’s coming from 192.168.1.2 to 192.168.1.1 and it matches:

/ip route rule add src-address=192.168.1.0/24 table=internet1

So at first sight, only internet1 table should be used and it doesn’t contain connected routes. But everything is fine so far, because there’s exception (hidden rule) that when target is local address, router uses main routing table.

But then there’s response from 192.168.1.1, it matches same rule and in this case it will do what it says, it will look up destination in routing table internet1, which contains only:

/ip route add distance=1 gateway=ether1-wan1 routing-mark=internet1

So even though destination is 192.168.1.2, it will be sent to ether1-wan1.

[Zacharias]

@sob totally understood…
Some questions though…
Why should i use routing rules since i can mark all my connections in the mangles facility and route everything according my needs… I could do the same implementation just using mangles only and the routing table… so what extra does it offer ?

But everything is fine so far, because there’s exception (hidden rule)

Hidden in the exported config or hidden in general ?

[Sob]

It allows you to not use firewall rules. If you know that no matter what, some subnets should always use specific routing table, it’s easier to use routing rules than make firewall exceptions to not mark something.

For example, let’s say you have dual WAN config and you want one LAN client to use only specific WAN. It’s easy, you can use mangle rule and mark routing for it. But what if you also have local server publicly available using forwarded ports and this client needs to connect to it? That’s what hairpin NAT is for and it works great. But in this case it won’t, because you marked routing for this client, to use different routing table, which does not contain local connected subnets. So you can either add exceptions to not mark routing when it will be forwarded back to server, but that could be a lot of rules if there are more ports, you will have to keep it synchronized with dstnat rules, and it won’t be clear at all. Or you can use one routing rule, to tell router that local subnet should be always looked up only in main routing table.

Also, unlike using mangle rules, routing rules should be compatible with fasttrack (I didn’t test is myself).

And about routing rules for local destination, I don’t remember details, but from older thread:

@mducharme

… I can create different routing table and can mark packets to use it (that’s what you do too), but local addresses seem to be special case and ignore this. I had a look how Linux (which RouterOS is based on) does this. It has special routing table “local” for local addresses. I’m pretty sure that RouterOS has the same thing internally. The difference is that Linux allows you to work with this table (change routing rules or the table itself), but RouterOS doesn’t. It looks like in RouterOS “local” routing table is hardcoded to always have the highest priority.

[Zacharias]

@sob although i ve read this https://wiki.mikrotik.com/wiki/Manual:IP/Route a couple of times, i had never noticed it… we always learn…
So you were right, it is mentioned in the wiki as well:
By default (when no routing-mark values are used) all active routes are in the main table, and there is only one hidden implicit rule (“catch all” rule) that uses the main table for all destination lookups.

So this is the part i missed from the beginning, those route rules, using routing marks, bypassed the main routing table… Thus, even the connected routes could not reach their gate…

[Sob]

It’s two things:

• Hidden rule mentioned by manual is similar to implicit accept at the end of firewall chain (even though it’s not actually a rule there), i.e. what happens if nothing else before it matches.
• If destination is local address, it gets special treatment, you can’t tell RouterOS to use different routing table for it and route it elsewhere (that’s what the linked older thread was about).

[Zacharias]

Yes, well i mean that the route rules send the traffic to a different path, according to the routing marks, when the destination was not a local address ( address asigned to the routers interface, like the 192.168.1.2) although there is a connected route for .1.0/24 …

[gnro]

The Home PC must be connected to one of the ether3, 4, 5, or to a switch connected on these ports.
The Work PC must be connected to one of the ether17, 18 or to a switch connected on these ports.

You must add:

/ip route rule
add dst-address=192.168.0.0/16 src-address=192.168.0.0/16 table=main place-before=0

or a similar rule or rules, as Sob suggested, but be sure this rule is processed before the marked ones, hence the “place-before=0”.

[stefki]

Thank you so much !
Yes it’s works.

Now ping works

ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=63 time=0.767 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=63 time=0.726 ms
^C

this rule was solution.
/ip route rule
add dst-address=192.168.0.0/16 src-address=192.168.0.0/16 table=main place-before=0

[Zacharias]

Actually /16 is too much…
/22 is enough…

@sob, i made a little more research…
In linux kernel there are 3 routing tables, the local the main and the default…

So, it is not actually a hidden route inside the main routing table but a tottally different table that has the locally hosted IPs and broadcast addresses, that is the Local Routing Table…
It is a Table that we do not see, i dont know if through terminal would be possible, and can not be modified. You can only delete routes in that Table…

If no matching route is found in the Local routing Table then the Main Routing Table is Checked…
And as it proves at least in ROS, this Table can not be bypassed by any Routing Rule…

To sum up, first is checked the Local Routing Table, then if we have added some other route rules pointing to some custom Tables, those Rules are checked and if no match has been found so far lastly the Main routing Table gets checked…

[stefki]

Thank you very much for routing solution between dhcp servers.

Now I have another question which is related to this post, I dont want to open new thread :slight_smile: When I try to access to my local server or my local PC's over Public IP WAN1 or WAN2 I can’t.
I got rejected by winscp or chrome. When I type my public ip in google chrome , the page is not accessible, buy If I put local 192.168.2.93 yeah its works
example screenshot from Winscp, of course all ports are forwarded correctly. Also I can access to my server or pc’s when I am connected from outside network, another ISP line, for example 4G from my cellphone. But from local mikrotik it doesn`t connect by public ip.

If I replace public ip address with this one 192.168.2.93 I get connected with WInscp to my server.
thank you!

[Sob]

You’re looking for https://wiki.mikrotik.com/wiki/Hairpin_NAT

[Zacharias]

You’re looking for https://wiki.mikrotik.com/wiki/Hairpin_NAT

Oh yes…