Unable to route between two networks

I am running a RB/2011L-RM with RouterOS v6.0rc13. I have two WANs and two networks running through my router. The WANs use separate gateways and are from different ISPs. My configuration currently works so that when I am connected to a local port it routes my traffic through the corresponding WAN. I have port 5 set up so that I can switch between networks when I need to.

Here is my network diagram:

My problem is that I am unable to route between the two networks, specifically form a laptop connected to port 5 to a webserver. I have a webserver at 192.168.88.15 and a domain pointing to the WAN plugged into ether1.

My configuration is as follows:
I have two bridges bridge-local and bridge-W.

bridge-local 192.168.88.0/24
bridge-W 192.168.1.0/24

Port 1: ether1-gateway (WAN1)
Port 2: ether2-TC-Local bridge-local
Port 3: ether3-W-WAN (Wan2)
Port 4: ether4-W-Local bridge-w
Port 5: ether5 bridge-local/bridge-w (Switches bridge)

Firewall/Mangle:
Mark Routing src-address=192.168.88.0/24 routing-mark=TC
Mark Routing src-address=192.168.1.0/24 routing-mark=W

Routes:
0.0.0.0/0 gateway=ether1-gateway routing-mark=TC
0.0.0.0/0 gateway=ether3-W-WAN routing-mark=W


Any help would be appreciated, and if more information is needed just tell me.

By default Mikrotik will route between two different connected networks. Unless you put in a filter rule to stop it.

When you say can’t route what exactly do you mean?

  1. Can’t ping server via router?
  2. Can’t ping server via PC?
  3. Ping works I can’t do something else… Explain…

Specifically I can’t ping the server from the PC. Pinging from the Router to the Server works fine.

Disable you firewall filter rules and test again if it succeeds resolve the issue with an allow above the drop that is causing the issue.

Assuming that is not the issue at all…

It is possible you have a route that is hijacking that traffic. Do a print on your routes routes and post them.



******One more thing the switch may have some kind of snooping shutting you down… Bypass it (for testing) to isolate the issue.

I disabled all of my firewall rules, it did not solve the problem. I then tried and disabled my mangle rules and I can now access the webserver. I still have the correct IP on my laptop (192.168.1.2) and my external IP is the one for the correct WAN (W). Is my setup going to continue to work without these mangle rules?

P.S. Thanks for your help.

I assuming you are talking about these rules you posted earlier


Firewall/Mangle:
Mark Routing src-address=192.168.88.0/24 routing-mark=TC
Mark Routing src-address=192.168.1.0/24 routing-mark=W

Routes:
0.0.0.0/0 gateway=ether1-gateway routing-mark=TC
0.0.0.0/0 gateway=ether3-W-WAN routing-mark=W

Based on these rules they should have never caused an issue because the route is distance 1. When a request comes from the 192.168.88.0/24 subnet to the 192.168.1.0/24 subnet it will use the distance ZERO connected route. So those rule should not be causing the issue unless you have some element in them I am not aware of. Post the mangle rules New Terminal IP Firewall Mangle Print



[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=prerouting action=mark-routing new-routing-mark=TC
     passthrough=yes src-address=192.168.88.0/24 

 1   chain=prerouting action=mark-routing new-routing-mark=W
     passthrough=yes src-address=192.168.1.0/24

Sorry for the long wait time. After disabling these routes I was able to get traffic between the networks, but External traffic into my webserver was not routing properly. I have currently re-enabled these mangle rules and am back where I started.

Ah I see the issue now those mangle rules don’t exclude your local traffic.

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic
0 chain=prerouting action=mark-routing new-routing-mark=TC
passthrough=yes src-address=192.168.88.0/24

1 chain=prerouting action=mark-routing new-routing-mark=W
passthrough=yes src-address=192.168.1.0/24

Lets add NOT ! 192.168.0.0/16 for DST Address to both rules and test it.

Or

We could add an Accept local Passthough NO above those two rules.

Or

We could add a distance zero connected route that supports your routing marks as is.

I’m into the realm where I’m not quite sure what I am doing. I added these two rules. My though was when the source address is form the TC network and the destination is the W network accept the connection and vice versa.

 
 0   chain=prerouting action=accept src-address=192.168.88.0/24 
     dst-address=192.168.1.0/24 

 1   chain=prerouting action=accept src-address=192.168.1.0/24 
     dst-address=192.168.88.0/24 

 2   chain=prerouting action=mark-routing new-routing-mark=TC
     passthrough=yes src-address=192.168.88.0/24 dst-address=!192.168.0.0/16 

 3   chain=prerouting action=mark-routing new-routing-mark=W
     passthrough=yes src-address=192.168.1.0/24 dst-address=!192.168.0.0/16



Lets add NOT ! 192.168.0.0/16 for DST Address to both rules and test it.

We could add an Accept local Passthough NO above those two rules.

We could add a distance zero connected route that supports your routing marks as is.

I tried the NOTs as you see above, nothing changed. I tried the accept rules, again no change. I tried a route but I don’t think I configured it right. Could you give me an example of what the route should look like? Thanks.

Doesn’t make sense information is missing. Please post the Mangle Rules as well as the routes.

Mangle rules

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=prerouting action=mark-routing new-routing-mark=TC
     passthrough=yes src-address=192.168.88.0/24 

 1   chain=prerouting action=mark-routing new-routing-mark=W
     passthrough=yes src-address=192.168.1.0/24

Routes

[admin@MikroTik] /ip route> print
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 #      DST-ADDRESS        PREF-SRC        GATEWAY            DISTANCE
 0 A S  0.0.0.0/0                          24.229.41.1               1
 1 A S  0.0.0.0/0                          70.44.216.1               1
 2 ADS  0.0.0.0/0                          70.44.216.1               0
 3  DS  0.0.0.0/0                          24.229.41.1               0
 4 ADC  24.229.41.0/24     24.229.41.95    ether1-gateway            0
 5 ADC  70.44.216.0/21     70.44.218.184   ether3-W-WAN        0
 6 ADC  192.168.1.0/24     192.168.1.1     bridge-W            0
 7 ADC  192.168.88.0/24    192.168.88.1    bridge-local              0

Okay I see the issue. You have 0.0.0.0/0 routes at distance ZERO. Distance Zero should be all your local routes and distance 1+ should be used for your out to internet routes.

At no time should a 0.0.0.0/0 DST route be distance 0…

I presume you are talking about these two default routes at a distance of 0. These two routes were dynamically assigned, hence the D on the far left. Should I still get rid of them?

 2 ADS  0.0.0.0/0                          70.44.216.1               0
 3  DS  0.0.0.0/0                          24.229.41.1               0

I got rid of them but there was no change.

I fooled around with mangles a bit and managed to get routing by internal IP working by adding the !(Address Range) to the rules.

[admin@MikroTik] /ip firewall mangle> print
Flags: X - disabled, I - invalid, D - dynamic 
 0   chain=prerouting action=mark-routing new-routing-mark=TC
     passthrough=no src-address=192.168.88.0/24 dst-address=!192.168.1.0/24 

 1   chain=prerouting action=mark-routing new-routing-mark=W passthrough=no 
     src-address=192.168.1.0/24 dst-address=!192.168.88.0/25

Everything is almost fixed, my last problem is that I need to get access to my webserver via my domain internally. Externally everything works fine, but internally I can only access my webserver via it’s IP (192.168.88.15). Again I’m a novice at mikrotik routing, but I think I need some sort of Masquarade between the two networks. Am I right in thinking this?


Update:
It seems that I need to setup a hairpin NAT. I’ve tried the example on the wiki, along with some other threads on this forum, but nothing so far has worked. My remaining problem is that I cannot reach my webserver using the FQDN, nor can the webserver reach itself using its FQDN, which has caused other problems with my Mysql server and email server.

Is anyone able to help me with this routing problem? I have two WANs with two subnets, I’m unable to access my webserver on the opposite subnet using it’s FQDN. I’ve tried multiple Hairpin configurations but nothing has worked and I have been dead in the water for almost two weeks now. Please Help.

IF the Hair pin nat rules are not working

http://wiki.mikrotik.com/wiki/Hairpin_NAT

please post firewall NAT rules


****Did you use the final masquerade rule on the hairpin nat page?

*********Also, I know this is bad form but you could just put the DNS entry into the mikrotik for the local address.

Here is just about every configuration.

[b]Filter Rules[/b]
 0 X ;;; default configuration
     chain=input action=accept protocol=icmp 

 1   ;;; default configuration
     chain=input action=accept connection-state=established 

 2   ;;; default configuration
     chain=input action=drop in-interface=sfp1-gateway 

 3   ;;; default configuration
     chain=input action=drop in-interface=ether1-gateway 

 4   ;;; default configuration
     chain=input action=accept connection-state=related 

 5   ;;; default configuration
     chain=input action=drop in-interface=ether3-W-WAN 

	 
[b]NAT[/b]
 Flags: X - disabled, I - invalid, D - dynamic 
 0 X ;;; default configuration
     chain=srcnat action=masquerade out-interface=sfp1-gateway 

 1   ;;; default configuration
     chain=srcnat action=masquerade out-interface=ether1-gateway 

 2   ;;; MASQ
     chain=srcnat action=masquerade out-interface=ether3-W-WAN
	 
 3   ;;; DON'T TOUCH ME
     chain=srcnat action=masquerade dst-address=192.168.88.0/24 
     src-address-list=Internal 

 4   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=25 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=25 

 5   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=25 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=25 

 6   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=143 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=143 

 7   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=143 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=143 

 8   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=993 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=993 

 9   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=993 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=993 

10   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=110 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=110 

11   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=110 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=110 

12   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=995 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=995 

13   chain=dstnat action=dst-nat to-addresses=192.168.88.13 to-ports=995 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=995 

14   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=22 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=22 

15   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=22 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=22 

16   ;;; W -  udp
     chain=dstnat action=dst-nat to-addresses=192.168.1.2 to-ports=8080 
     protocol=udp dst-address=(External IP 2 - ether3-W-WAN) dst-port=8080 
	 
17   ;;; W -  tcp
     chain=dstnat action=dst-nat to-addresses=192.168.1.2 to-ports=8080 
     protocol=tcp dst-address=(External IP 2 - ether3-W-WAN) dst-port=8080 

18   chain=srcnat action=src-nat to-addresses=(External IP 1 - ether1-gateway) 
     src-address=192.168.88.0/24 dst-address=192.168.88.15 

19   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=80 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=80 

20   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=80 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=80 

21   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=3306 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=3306 

22   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=3306 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=3306 

23   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=1723 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=1723 

24   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=1723 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=1723 

25   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=443 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=443 

26   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=443 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=443 

27   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=123 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=123 

28   chain=dstnat action=dst-nat to-addresses=192.168.88.15 to-ports=123 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=123 

29   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=22 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=220 

30   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=22 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=220 

31 X chain=srcnat action=src-nat to-addresses=(External IP 1 - ether1-gateway) 
     src-address=192.168.88.0/24 dst-address=192.168.88.14 

32   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=25579 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=25579 

33   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=25579 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=25579 

34   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=25565 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=25565 

35   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=25565 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=25565 

36   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=8123 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=8123 

37   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=8123 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=8123 

38 X chain=srcnat action=src-nat to-addresses=(External IP 1 - ether1-gateway) 
     src-address=192.168.88.0/24 dst-address=192.168.88.36 

39   chain=dstnat action=dst-nat to-addresses=192.168.88.36 to-ports=5001 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=5001 

40   chain=dstnat action=dst-nat to-addresses=192.168.88.36 to-ports=5001 
     protocol=udp dst-address=(External IP 1 - ether1-gateway) dst-port=5001 

41   chain=dstnat action=dst-nat to-addresses=192.168.88.14 to-ports=25566 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=25575 

42   chain=dstnat action=dst-nat to-addresses=192.168.88.46 to-ports=3074 
     protocol=tcp dst-address=(External IP 1 - ether1-gateway) dst-port=3074 

43 X chain=srcnat action=src-nat to-addresses=(External IP 1 - ether1-gateway) 
     src-address=192.168.88.0/24 dst-address=192.168.88.46 

44   chain=dstnat action=dst-nat to-addresses=192.168.1.2 to-ports=8333 
     protocol=tcp dst-address=(External IP 2 - ether3-W-WAN) dst-port=8333


[b]Mangle[/b]
 0   chain=prerouting action=mark-routing new-routing-mark=TC passthrough=no src-address=192.168.88.0/24 
     dst-address=!192.168.1.0/24 

 1   chain=prerouting action=mark-routing new-routing-mark=W passthrough=no src-address=192.168.1.0/24 
     dst-address=!192.168.88.0/25

And Yes I was using the last masquerade rule on the hairpin page.

I tried adding a DNS entry for my Domain in my routers DNS. It seems to now be resolving my local IP for my domain (192.168.88.1). Is this the desired result? I also still am unable to access the server.

I’m lost on this one. I would suggest rolling a new thread labeled Hair Pin Nat.