Page 1 of 1

No routing between subnets! Why?

Posted: Thu May 21, 2015 5:09 pm
by Erlington
Hello Guys,

I'm just having and very weird issue. I have a RB2011UiAS-RM as Router of 3 Lans:
[admin@ala] > ip address print detail 
Flags: X - disabled, I - invalid, D - dynamic 
 0   ;;; default configuration
     address=192.168.88.1/24 network=192.168.88.0 interface=ether10 actual-interface=ether10 

 1   ;;; WAN Link
     address=x.x.x.x/30 network=x.x.x.x interface=ether5 actual-interface=ether5 

 3   ;;; AP PISO 1
     address=192.168.1.1/24 network=192.168.1.0 interface=ether2 actual-interface=ether2 

 4   ;;; AP PISO 2
     address=192.168.2.1/24 network=192.168.2.0 interface=ether3 actual-interface=ether3 

 5   ;;; Work Stations PISO 1
     address=192.168.3.1/24 network=192.168.3.0 interface=bridge1 actual-interface=bridge1 
I use NAT to get them to the internet:
[admin@ala] > ip firewall nat print detail 
Flags: X - disabled, I - invalid, D - dynamic 
 0 X  chain=srcnat action=masquerade src-address=192.168.1.0/24 out-interface=ether5 log=no log-prefix="" 

 1 X  chain=srcnat action=masquerade src-address=192.168.2.0/24 out-interface=ether5 log=no log-prefix="" 

 2 X  chain=srcnat action=masquerade src-address=192.168.3.0/24 out-interface=ether5 log=no log-prefix="" 
My bridge config:
 [admin@ala] > interface bridge print detail 
Flags: X - disabled, R - running 
 0  R name="bridge1" mtu=1500 l2mtu=65535 arp=enabled mac-address=D4:CA:6D:1C:96:69 protocol-mode=none priority=0x8000 auto-mac=yes admin-mac=00:00:00:00:00:00 
      max-message-age=20s forward-delay=15s transmit-hold-count=6 ageing-time=5m 
[admin@ala] > interface bridge port print detail 
Flags: X - disabled, I - inactive, D - dynamic 
 0 I  interface=ether4 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 1 I  interface=ether6 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 2    interface=ether7 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 3 I  interface=ether8 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 

 4 I  interface=ether9 bridge=bridge1 priority=0x80 path-cost=10 edge=auto point-to-point=auto external-fdb=auto horizon=none auto-isolate=no 
My DHCP Server
[admin@ala] > ip dhcp-server print detail 
Flags: X - disabled, I - invalid 
 0   name="dhcp1" interface=ether2 lease-time=10h address-pool=dhcp_pool1 bootp-support=static authoritative=after-2sec-delay lease-script="" 

 1   name="dhcp2" interface=ether3 lease-time=10h address-pool=dhcp_pool2 bootp-support=static authoritative=after-2sec-delay lease-script="" 

 2   name="dhcp3" interface=bridge1 lease-time=3d address-pool=dhcp_pool3 bootp-support=static authoritative=after-2sec-delay lease-script="" 
My Routes:
1 ADC  dst-address=192.168.1.0/24 pref-src=192.168.1.1 gateway=ether2 gateway-status=ether2 reachable distance=0 scope=10 

2 ADC  dst-address=192.168.2.0/24 pref-src=192.168.2.1 gateway=ether3 gateway-status=ether3 reachable distance=0 scope=10 

2 ADC  dst-address=192.168.3.0/24 pref-src=192.168.3.1 gateway=bridge1 gateway-status=bridge1 reachable distance=0 scope=10 
My ip settings:
[admin@ala] > ip settings print    
           ip-forward: yes
       send-redirects: yes
  accept-source-route: no
     accept-redirects: no
     secure-redirects: yes
            rp-filter: no
       tcp-syncookies: no
          arp-timeout: 30s
      icmp-rate-limit: 10
       icmp-rate-mask: 0x1818
      allow-fast-path: yes
My issue is, they can go to internet but can't comunicate among each other, i.e. A host in subnet 192.168.3.0/24 can't connect to 192.168.2.0/24 or 192.168.1.0/24 and viceversa.

Hosts can do ping others subnets gateways but just the gateways no other subnets hosts.

From Router I can ping everyone. But setting source address I can't.
[admin@ala] > ping 192.168.3.20 src-address=192.168.2.1
HOST                                     SIZE TTL TIME  STATUS                                                                                                         
192.168.3.20                                            timeout                                                                                                        
192.168.3.20                                            timeout                                                                                                        
192.168.3.20                                            timeout                                                                                                        
192.168.3.20                                            timeout                                                                                                        
    sent=4 received=0 packet-loss=100% 

[admin@ala] > ping 192.168.3.20 src-address=192.168.1.1 
HOST                                     SIZE TTL TIME  STATUS                                                                                                         
192.168.3.20                                            timeout                                                                                                        
192.168.3.20                                            timeout                                                                                                        
    sent=2 received=0 packet-loss=100% 

[admin@ala] > ping 192.168.3.20 src-address=192.168.3.1 
HOST                                     SIZE TTL TIME  STATUS                                                                                                         
192.168.3.20                               56  64 51ms 
192.168.3.20                               56  64 0ms  
    sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=25ms max-rtt=51ms

[admin@ala] > ping 192.168.3.20 
HOST                                     SIZE TTL TIME  STATUS                                                                                                         
192.168.3.20                               56  64 0ms  
192.168.3.20                               56  64 8ms  
    sent=2 received=2 packet-loss=0% min-rtt=0ms avg-rtt=4ms max-rtt=8ms 
I have no firewall rules just NAT.

Am I missing something?

Re: No routing between subnets! Why?

Posted: Thu May 21, 2015 5:19 pm
by pukkita
Can you post /ip dhcp-server network export? Maybe your DHCP server isn't setting the right gateway ip address on the clients?

Re: No routing between subnets! Why?

Posted: Thu May 21, 2015 9:12 pm
by jaykay2342
have you check your firewall rules?

Re: No routing between subnets! Why?

Posted: Thu May 21, 2015 10:17 pm
by ZeroByte
It can't be the DHCP setting, or else the hosts wouldn't be able to reach the Internet either.

Almost certainly it's the firewall filter rules, in particular - the forward chain.
The reason hosts on Lan 1 is able to ping the gateway addresses for lans 2 and 3 is because those packets don't actually go through the forward chain, but the input chain. (input = to the Mikrotik's brain, not throgh the Mikrotik to some other host)

Lan1host <---> Lan2host = forward chain

Probably you have a rule logic that says to
1: accept established,related
2: accept out-interface=ether5
3: drop all else.

If you want all lans to have full access to each other, then change this logic to:
1: accept established,related
2: drop in-interface=ether5
3: accept all
(this last rule is somewhat redundant since "allow" would be the default action anyway, but it's good practice to put an explicit rule so that it shows up and makes the behavior absolutely clear that it's intended)

Re: No routing between subnets! Why?

Posted: Thu May 21, 2015 10:43 pm
by dgnevans
Can you post your firewall rules.
Second thing on your srcnat masquerade. Why dont you drop the 192.168.*.0/24 and have a single rule as you only have one outside interface.

0 X chain=srcnat action=masquerade out-interface=ether5 log=no log-prefix=""

Re: No routing between subnets! Why?

Posted: Sat May 23, 2015 10:13 pm
by Erlington
It can't be the DHCP setting, or else the hosts wouldn't be able to reach the Internet either.

Almost certainly it's the firewall filter rules, in particular - the forward chain.
The reason hosts on Lan 1 is able to ping the gateway addresses for lans 2 and 3 is because those packets don't actually go through the forward chain, but the input chain. (input = to the Mikrotik's brain, not throgh the Mikrotik to some other host)

Lan1host <---> Lan2host = forward chain

Probably you have a rule logic that says to
1: accept established,related
2: accept out-interface=ether5
3: drop all else.

If you want all lans to have full access to each other, then change this logic to:
1: accept established,related
2: drop in-interface=ether5
3: accept all
(this last rule is somewhat redundant since "allow" would be the default action anyway, but it's good practice to put an explicit rule so that it shows up and makes the behavior absolutely clear that it's intended)
Sadly, I have no firewall rules, as I did a reset/configuration with no default configuration checked. Can you post this rules in code mode? I would definitely appreciate that.
Thanks everyone for your help!

Re: No routing between subnets! Why?

Posted: Sat May 23, 2015 10:17 pm
by Erlington
Can you post your firewall rules.
Second thing on your srcnat masquerade. Why dont you drop the 192.168.*.0/24 and have a single rule as you only have one outside interface.

0 X chain=srcnat action=masquerade out-interface=ether5 log=no log-prefix=""
I have no firewall rules besides nat, I did the NAT as your suggest, it keeps working but no subnets communication.
Thanks for your help everyone!

Re: No routing between subnets! Why?

Posted: Sat May 23, 2015 10:44 pm
by dgnevans
Do you have any rule in you IP Firewall Mangle?

Re: No routing between subnets! Why?

Posted: Sat May 23, 2015 10:48 pm
by pukkita
Can you do a traceroute from a host in a subnet to one in another subnet (check they're pingeable by the router), and check where the packet "dies" or who reports it as unreacheable? Can you check what's the default gateway in those two hosts routing table?

Do you have any mangle rules?

Re: No routing between subnets! Why?

Posted: Sun May 24, 2015 12:11 pm
by docmarius
The issue is probably not with the router, but with the clients.
e.g. Windows clients don't allow by default access to its resources from other subnets than its own, while outgoing requests are possible (in other words, 'new' connections are accepted only from its own subnet, 'established' and 'related' from any source).
So in this light, 192.168.1.0/24 machines will not be able to talk to 192.168.2.0/24 and 192.168.3.0/24 machines, unless the client's firewalls are corrected accordingly to accept /16 traffic or the subnet mask of the clients are changed to /16, so that all IPs fall into the same subnet from the clients point of view.
A src-nat to the own IP on all internal interfaces will also work, but this will prevent the proper identification of the source machine.

Re: No routing between subnets! Why?

Posted: Sun May 24, 2015 5:01 pm
by dgnevans
Hi DocMarius
There is no need to change their networks to /16 network. That is what the router is there for routing between subnets. I have used similar setups mutiple times. If they were not passing through a router they would need a flat /16 network to be able to communicate across the various subnets.

Re: No routing between subnets! Why?

Posted: Sun May 24, 2015 10:14 pm
by docmarius
I wasn't talking about routing, but about default client firewalling.
Windows by default doesn't accept connections from outside its defined subnet, especially on windows 7 and later.
So either twist the firewalls on clients, change their netmask or use src-nat from other subnets to their own.
The router may work perfectly, but windows clients will still not talk one to another unless one of those 3 mods are done.
Again, it is a client firewall issue, not a routing problem.

Re: No routing between subnets! Why?

Posted: Mon May 25, 2015 5:11 am
by lunchboxrts
I wasn't talking about routing, but about default client firewalling.
Windows by default doesn't accept connections from outside its defined subnet, especially on windows 7 and later.
So either twist the firewalls on clients, change their netmask or use src-nat from other subnets to their own.
The router may work perfectly, but windows clients will still not talk one to another unless one of those 3 mods are done.
Again, it is a client firewall issue, not a routing problem.
Windows does not block connections from outside its subnet by default. If firewall is on it will block most incoming connection regardless of source local or remote. I have multiple routed subnets with windows clients and they connect fine and do not block by default.

We need a trace route output from across the route. It has to be either the computers gateway is wrong, a firewall and or mangle rule on the router.

Re: No routing between subnets! Why?

Posted: Mon May 25, 2015 7:06 pm
by Erlington
Hello guys, first of all, thanks for your help.
Sadly I have no firewall rules besides the NAT one. No Mangle either.

I'm also suspecting about windows firewall, see my example below:

A simple host
   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : fe80::1c37:94fc:50fc:6317%8
   IPv4 Address. . . . . . . . . . . : 192.168.2.57
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 192.168.2.1
Pinging to a CCTV, no firewalls on it.
C:\Users\Agent01>ping 192.168.3.10

Pinging 192.168.3.10 with 32 bytes of data:
Reply from 192.168.3.10: bytes=32 time=10ms TTL=63
Reply from 192.168.3.10: bytes=32 time=8ms TTL=63
Reply from 192.168.3.10: bytes=32 time=8ms TTL=63
Reply from 192.168.3.10: bytes=32 time=8ms TTL=63
Pinging to a random windows computer in other subnet.
C:\Users\Agent01>tracert 192.168.3.227

Tracing route to 192.168.3.227 over a maximum of 30 hops

  1     3 ms     1 ms    17 ms  192.168.2.1
  2     *        *        *     Request timed out.
  3     *        *        *     Request timed out.
Pinging a host with windows firewall default
C:\Users\Agent01>tracert 192.168.3.244

Tracing route to ADMIN-PC [192.168.3.244]
over a maximum of 30 hops:

  1    33 ms     1 ms     3 ms  192.168.2.1
  2     4 ms     1 ms     6 ms  ADMIN-PC [192.168.3.244]
With this, we ensure there is no problem in the router at all?
Thank you all!!

Re: No routing between subnets! Why?

Posted: Mon May 25, 2015 7:14 pm
by pukkita
Can you check what's the default gateway on 192.168.3.227 routing table?

Re: No routing between subnets! Why?

Posted: Tue May 26, 2015 1:02 am
by chechito
The issue is probably not with the router, but with the clients.
e.g. Windows clients don't allow by default access to its resources from other subnets than its own, while outgoing requests are possible (in other words, 'new' connections are accepted only from its own subnet, 'established' and 'related' from any source).
So in this light, 192.168.1.0/24 machines will not be able to talk to 192.168.2.0/24 and 192.168.3.0/24 machines, unless the client's firewalls are corrected accordingly to accept /16 traffic or the subnet mask of the clients are changed to /16, so that all IPs fall into the same subnet from the clients point of view.
A src-nat to the own IP on all internal interfaces will also work, but this will prevent the proper identification of the source machine.

good advice, try disabling windows and other security software firewalls and network security services

Re: No routing between subnets! Why?

Posted: Tue May 26, 2015 1:04 am
by chechito
I wasn't talking about routing, but about default client firewalling.
Windows by default doesn't accept connections from outside its defined subnet, especially on windows 7 and later.
So either twist the firewalls on clients, change their netmask or use src-nat from other subnets to their own.
The router may work perfectly, but windows clients will still not talk one to another unless one of those 3 mods are done.
Again, it is a client firewall issue, not a routing problem.
Windows does not block connections from outside its subnet by default. If firewall is on it will block most incoming connection regardless of source local or remote. I have multiple routed subnets with windows clients and they connect fine and do not block by default.

We need a trace route output from across the route. It has to be either the computers gateway is wrong, a firewall and or mangle rule on the router.
it depends of domain or group policy

on most cases without domain, default policy do not allow inbound connections from, another subnets

with a machine enrolled on a active directory domain the behavior can be different

Re: No routing between subnets! Why?

Posted: Tue May 26, 2015 6:50 pm
by Erlington
I'll make some test by disabling the firewall on computers and I'll let you know. Thank you for your help and knowledge sharing.

Anyway I'm interested on what ZeroByte wrote regarding the firewall forward, input, etc. Does anybody knows how those rules would be in code mode?

Thank you!!

Re: No routing between subnets! Why?

Posted: Tue May 26, 2015 10:46 pm
by dgnevans
should look something like this

1: accept established,related
chain=forward action=accept connection-state=established,related protocol=tcp log=no log-prefix=""
2: accept out-interface=ether5
chain=forward action=accept out-interface=ether5 log=no log-prefix=""
3: drop all else.
chain=forward action=drop src-address="LAN IP's" dst-address=0.0.0.0/0 in-interface="enter your in interface" log=no log-prefix=""
If you want all lans to have full access to each other, then change this logic to:

2: drop in-interface=ether5
chain=forward action=drop in-interface=ether5 log=no log-prefix=""
3: accept all
chain=forward action=accept log=no log-prefix=""

Re: No routing between subnets! Why?

Posted: Wed May 27, 2015 6:24 am
by ZeroByte
should look something like this

1: accept established,related
chain=forward action=accept connection-state=established,related protocol=tcp log=no log-prefix=""
2: accept out-interface=ether5
chain=forward action=accept out-interface=ether5 log=no log-prefix=""
3: drop all else.
chain=forward action=drop src-address="LAN IP's" dst-address=0.0.0.0/0 in-interface="enter your in interface" log=no log-prefix=""
I would just make rule #3 for the first set be simple:
/ip firewall filter
add chain=forward action=accept connection-state=established,related
add chain=forward action=accept out-interface=ether5
add chain=forward action=drop
No need for the in-interface=xxxx - the more criteria you add, the more complicated the logic becomes, and the easier it is for some "unusual" packet to slip through the cracks.
If you say "allow anything that's already been established" - then you basically mean "if it's already approved, keep approving it" (I'm going to play with fast track to make this rule even FASTER)
Rule 2 - ok - so it's a new connection. If it weren't new, then you would've already stopped on rule 1 right?
Okay - well, there's also invalid, but I'm not going to worry about invalid connections because I'm going to allow everything going out to the internet anyway - I'm not protecting the net from my computers.... rule 2 = if it's going out ether5, then it's cool, so allow it. (ether5 = internet interface right?)

So now, by simple logic, the only thing that remains is new connections that will go towards one of your LAN networks. (either from the Internet or from one of the other LANs) - all of which you don't want in the first model I gave, so a simple rule that just has "action=drop" is quite enough. Don't specify interfaces. If you want to allow some other interface - say ether2 is a privileged network, add a rule between 2 and 3 which says "chain=forward action=accept in-interface=ether2"

This kind of thinking is concise, tight, and most of all, efficient. Use as few tests as possible per rule, and as few rules as possible.

Re: No routing between subnets! Why?

Posted: Wed Jul 15, 2015 3:50 am
by zeroout
Hello guys,

first of all, I'm sorry for my bad english, because it's not my native language. Is this on-topic or already off-topic? Because I can help you to route between subnets, it has nothing to do with firewall (in your case). Please give me some response.

Thanks in advance.

Kind regards,

zeroout