3LANs to 3WANs are OK, but LAN to LAN is not

Hi all, following online documentation and videos i managed to forward mi LANs to WANs.
This is my config.
ISP Router with 3 LAN ports.
The 3 LAN ports are WAN ports on the Mikrotik.
They have address:
192.168.100.10
192.168.100.20
192.168.100.30

The 3 LANs on my Mikrotik are:
192.168.10.1
192.168.20.1
192.168.30.1

I can forward everything coming rom 192.168.10.0 to 192.168.100.10 and so on using mangles.
I cannot find a rule to allow the 3 LANs to see each other.
I would like to be able to pin 192.168.20.0 and 192.168.30.0 from 192.168.10.0… and so on.
I think I need some static routes, but I am doing some mistake I cannot find.

Can someone help me?

TIA

Luca

  1. Network diagram

  2. Requirements
    a. identify users/devices groups of users/devices including the admin
    b. identify what traffic each should be able to accomplish

  3. Post config
    /export file=anynameyouwish ( minus router serial number and any public WANIP information)

1:
2: The 3 PCs must reach each other. Nothe that ther are rules to make the 3 LANs use the 3 WANs.
3: As soon as I arrive home, I will post configuration. I am at office now… I could replicate it on my VM, but it would be a little different.

So you want each subnet to get a dedicated WAN, but each users on each subnet should also be able to reach each other?
Or is it more restrictive across subnets, some users from a to b, some a to c, and some b to c, but none c to a or b ???

Once we see the config, it should be quick to fix.

I think I have to change it according to the advice you are giving to me in the other thread.
Anyway your first idea was correct: each users on each subnet should also be able to reach each other.
No restrictions.

As anav said, we need to see your config. By default all three LANs will be able to see each other unless you blocked it.

Ok, I really cannot understand.
I reduced to 2 my networks and I have:
192.168.100.81/24
192.168.100.82/24
that are the 2 WAN ports connected to my ISP router (192.168.100.1)

Now something is not working again.
As suggested in topic http://forum.mikrotik.com/t/is-hap-ax-enough-for-2wans-2lans-1gbps-each/164409/1 I deleted mangles and I m trying to use rules only.

This is my config (Note that route 0.0.0.0/0 192.168.100.81 is not valid and I cannot understand why):

feb/15/2023 22:51:40 by RouterOS 7.7

software id = XXXX

model = C53UiG+5HPaxD2HPaxD

serial number = XXXXXXXXXX

/interface ethernet
set [ find default-name=ether1 ] name=ether1-WANPC
set [ find default-name=ether2 ] name=ether2-WANSERVER
set [ find default-name=ether4 ] name=ether4-LANPC
set [ find default-name=ether5 ] name=ether5-LANSERVER
/ip pool
add name=dhcp_pool_PC ranges=192.168.10.100-192.168.10.200
add name=dhcp_pool_SERVER ranges=192.168.20.100-192.168.20.200
/ip dhcp-server
add address-pool=dhcp_pool_PC interface=ether4-LANPC lease-time=1h name=
dhcp-PC
add address-pool=dhcp_pool_SERVER interface=ether5-LANSERVER lease-time=1h
name=dhcp-SERVER
/routing table
add disabled=no fib name=PCtoWANPC
add disabled=no fib name=SERVERtoWANSERVER
/ip address
add address=192.168.100.81/24 interface=ether1-WANPC network=192.168.100.0
add address=192.168.10.1/24 interface=ether4-LANPC network=192.168.10.0
add address=192.168.20.1/24 interface=ether5-LANSERVER network=192.168.20.0
add address=192.168.100.82/24 interface=ether2-WANSERVER network=
192.168.100.0
/ip dhcp-server network
add address=192.168.10.0/24 gateway=192.168.10.1
add address=192.168.20.0/24 gateway=192.168.20.1
/ip dns
set servers=8.8.8.8,1.1.1.1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1-WANPC
add action=masquerade chain=srcnat out-interface=ether2-WANSERVER
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.81
pref-src=“” routing-table=PCtoWANPC scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.82
pref-src=“” routing-table=SERVERtoWANSERVER scope=30 suppress-hw-offload=
no target-scope=10
/routing rule
add action=lookup disabled=no dst-address=192.168.10.0/24 table=main
add action=lookup disabled=no dst-address=192.168.20.0/24 table=main
add action=lookup-only-in-table disabled=no dst-address=192.168.10.0/24
table=PCtoWANPC
add action=lookup-only-in-table disabled=no dst-address=192.168.20.0/24
table=SERVERtoWANSERVER
/system clock
set time-zone-name=Europe/Rome
/system ntp client
set enabled=yes
/system ntp client servers
add address=ntp.ien.it
add address=time.google.com
add address=ntp1.inrim.it

With this configuration:
1 - I am able to browse the Internet from both eth4 and eth5, but I exit using eth1 and not eth1 and eth2.
2 - I cannot ping 192.168.10.X from 192.168.20.x
3 - I CAN ping 192.168.20.X from 192.168.10.x

What’s completely wrong with my configuration?

Thx in advance!!!

Luca

The routing rules need work.
The first two rules are good and ensure that LAN users can see each other, and then you apply the rules forcing certain subnets out certain wans.
The problem is your use of dst-addresses applies to allowing subnet to subnet access but not to forcing out to WANs. You need source address to identify the subnets going out the WANs.

THe first set of rules identifed subnets you want to visit (destination), the second set of rules identifies which source should go out which wan
SO.

/routing rule
add action=lookup disabled=no dst-address=192.168.10.0/24 table=main
add action=lookup disabled=no dst-address=192.168.20.0/24 table=main
add action=lookup-only-in-table disabled=no src-address=192.168.10.0/24
table=PCtoWANPC
add action=lookup-only-in-table disabled=no src-address=192.168.20.0/24
table=SERVERtoWANSERVER

Hi
I can tell you some common showstoppers that might prevent a migration to ROS7:

Compatibility: ROS7 is not backward compatible with previous versions of ROS. This means that any existing ROS nodes, packages, or software developed for previous versions of ROS will need to be modified or rewritten to work with ROS7.

System requirements: ROS7 requires a more powerful hardware system than previous versions of ROS. This may require an upgrade to existing hardware or the purchase of new hardware, which could be expensive and time-consuming.

Learning curve: ROS7 introduces new features and capabilities that may require a significant amount of time and effort to learn. This may be a showstopper for organizations or individuals who are comfortable with the current ROS version and don’t have the resources to invest in learning new technologies.

Third-party package availability: The availability of third-party packages and libraries for ROS7 may be limited in the early stages after its release. This may make it difficult for developers to find and use the tools they need to build their applications.

Stability and reliability: ROS7 is a new version of ROS, and it may have bugs or stability issues that could cause problems for developers and end-users. This is a significant showstopper for mission-critical applications where reliability is of utmost importance.

In summary, migrating to ROS7 can be a challenging process, and there are various factors to consider before making the switch. It is essential to evaluate the compatibility of existing software, hardware requirements, learning curves, availability of third-party packages, and stability and reliability of the new version before deciding whether or not to migrate.

Ok, now it should work… but it does not.
I cannot browse the Internet.
Here is the configuration (note that I had to change ports since port 1 seem,s burned… if I attach anything to it the router poweroff).

These 2 routes are RED:

Here is the new configuration.
If I use 192.168.100.1 as default gateway everything works fine, but I do not have the traffic split between the 2 ports.
It looks like I cannot use 192.168.100.81 and 192.168.100.82 which are the Mikrotik WAN IP interfaces.

/interface ethernet
set [ find default-name=ether1 ] poe-out=off
set [ find default-name=ether2 ] name=ether2-WANPC
set [ find default-name=ether3 ] name=ether3-WANSERVER
set [ find default-name=ether4 ] name=ether4-LANPC
set [ find default-name=ether5 ] name=ether5-LANSERVER
/ip pool
add name=dhcp_pool_PC ranges=192.168.10.100-192.168.10.200
add name=dhcp_pool_SERVER ranges=192.168.20.100-192.168.20.200
/ip dhcp-server
add address-pool=dhcp_pool_PC interface=ether4-LANPC lease-time=1h name=
dhcp-PC
add address-pool=dhcp_pool_SERVER interface=ether5-LANSERVER lease-time=1h
name=dhcp-SERVER
/routing table
add disabled=no fib name=PCtoWANPC
add disabled=no fib name=SERVERtoWANSERVER
/ip address
add address=192.168.100.81/24 interface=ether2-WANPC network=192.168.100.0
add address=192.168.10.1/24 interface=ether4-LANPC network=192.168.10.0
add address=192.168.20.1/24 interface=ether5-LANSERVER network=192.168.20.0
add address=192.168.100.82/24 interface=ether3-WANSERVER network=
192.168.100.0
/ip dhcp-server network
add address=192.168.10.0/24 gateway=192.168.10.1
add address=192.168.20.0/24 gateway=192.168.20.1
/ip dns
set servers=8.8.8.8,1.1.1.1
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether2-WANPC
add action=masquerade chain=srcnat out-interface=ether3-WANSERVER
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.81
pref-src=“” routing-table=PCtoWANPC scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.82
pref-src=“” routing-table=SERVERtoWANSERVER scope=30 suppress-hw-offload=
no target-scope=10
/routing rule
add action=lookup disabled=no dst-address=192.168.10.0/24 table=main
add action=lookup disabled=no dst-address=192.168.20.0/24 table=main
add action=lookup-only-in-table disabled=no dst-address=“” src-address=
192.168.10.0/24 table=PCtoWANPC
add action=lookup-only-in-table disabled=no dst-address=“” src-address=
192.168.20.0/24 table=SERVERtoWANSERVER
/system clock
set time-zone-name=Europe/Rome
/system ntp client
set enabled=yes
/system ntp client servers
add address=ntp.ien.it
add address=ntp1.inrim.it
add address=time.google.com

Well your WAN setup seem very weird to me and not surprized it doesnt work yet.
Will have to think about it tomorrow.

Purely guessing, is modify the one rule…
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10

TO
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“192.168.100.81” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“192.168.100**.82**” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10

Nothing :frowning:
Tracreoute does not even arrive to the first hop.
I am wondering why I cannot choose the out interface.
This router is a rock star, it shouldn’t be so difficult: “What is coming from port 4 and has to go to 0.0.0.0/0 has to go to port 2”…
Do you think I can solve?

TIA

Luca

Assuming you are using three ports coming out of ISP router and plugging into 3 ports on the MT.
This should work…
/routing table add fib name=useWAN1
/routing table add fib name=useWAN2
/routing table add fib name=useWAN3

/ip routes
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=MAIN
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=useWAN1
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=useWAN2
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=useWAN3

/routing rules
add dst-address=192.168.10.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.20.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.30.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.100.81 src-address=192.168.10.0/24 action=lookup-only-in-table table=useWAN1
add dst-address=192.168.100.82 src-address=192.168.20.0/24 action=lookup-only-in-table table=useWAN2
add dst-address=192.168.100.83 src-address=192.168.30.0/24 action=lookup-only-in-table table=useWAN3

Dont forget the sourcenat rules…
add chain=srcnat action=src-nat out-interface=ether1 to-address=192.168.100.81
add chain=srcnat action=src-nat out-interface=ether2 to-address=192.168.100.82
add chain=srcnat action=src-nat out-interface=ether3 to-address=192.168.100.83

That’s exactly what I do

This should work…
/routing rules
add dst-address=192.168.100.81 src-address=192.168.10.0/24 action=lookup-only-in-table table=useWAN1
add dst-address=192.168.100.82 src-address=192.168.20.0/24 action=lookup-only-in-table table=useWAN2
add dst-address=192.168.100.83 src-address=192.168.30.0/24 action=lookup-only-in-table table=useWAN3

Have I to replace my rules or to add these lines to my rules?
I think they are intended to replace my ones, but I would like to be sure.

thanks again for your patience and help

Luca

Good thing you waited, a birdie informed me that my approach was not quite correct.
YES replace all with the below unless exactly the same…

Assuming the name of the ethernet interfaces, ether1, ether2 and ether2 have been left as defaults…

one set of Tables created. ( 3 in total )
one set of Routes ( 4 in total )
and one set of Routing Rules ( 6 in total )

/routing table add fib name=useWAN1
/routing table add fib name=useWAN2
/routing table add fib name=useWAN3

/ip routes
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=MAIN
add dst-address=0.0.0.0/0 gateway=192.168.100.1**%ether1** routing-table=useWAN1
add dst-address=0.0.0.0/0 gateway=192.168.100.1**%ether2** routing-table=useWAN2
add dst-address=0.0.0.0/0 gateway=192.168.100.1**%ether3** routing-table=useWAN3

/routing rules
add dst-address=192.168.10.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.20.0/24 action=lookup-only-in-table table=main
add dst-address=192.168.30.0/24 action=lookup-only-in-table table=main
add src-address=192.168.10.0/24 action=lookup-only-in-table table=useWAN1 ( edit fixed)
add src-address=192.168.20.0/24 action=lookup-only-in-table table=useWAN2 (edit fixed)
add src-address=192.168.30.0/24 action=lookup-only-in-table table=useWAN3 (edit fixed)

Dont forget the sourcenat rules…
add chain=srcnat action=src-nat out-interface=ether1 to-address=192.168.100.81
add chain=srcnat action=src-nat out-interface=ether2 to-address=192.168.100.82
add chain=srcnat action=src-nat out-interface=ether3 to-address=192.168.100.83

I cannot understand.
Everything is still going out thru the same interface.
Even if I disable the dedicated routes, everything works fine.
What am I mistaking?

/interface bridge
add name=WiFi-LAN30
/interface ethernet
set [ find default-name=ether1 ] name=ether1-LAN10 poe-out=off
set [ find default-name=ether2 ] name=ether2-LAN20
set [ find default-name=ether3 ] name=ether3-WAN10
set [ find default-name=ether4 ] name=ether4-WAN20
set [ find default-name=ether5 ] name=ether5-WAN30
/interface wifiwave2 channel
add band=2ghz-n name=2Ghz width=20/40mhz
add band=5ghz-ax name=5Ghz width=20/40/80mhz
/interface wifiwave2 security
add authentication-types=wpa2-psk,wpa3-psk encryption=tkip name=lmesecurity
/interface wifiwave2 configuration
add channel=2Ghz country=Italy mode=ap name=lmecfg2Ghz security=lmesecurity
ssid=merefy
add channel=5Ghz country=Italy mode=ap name=lmecfg5Ghz security=lmesecurity
ssid=merefy
/interface wifiwave2
set [ find default-name=wifi1 ] channel=5Ghz channel.width=20/40/80mhz
configuration=lmecfg5Ghz configuration.mode=ap disabled=no
security.authentication-types=wpa2-psk,wpa3-psk .encryption=“”
set [ find default-name=wifi2 ] channel=2Ghz configuration=lmecfg2Ghz
configuration.mode=ap disabled=no security.authentication-types=
wpa2-psk,wpa3-psk .encryption=“”
/ip pool
add name=dhcp_pool_PC ranges=192.168.10.100-192.168.10.200
add name=dhcp_pool_SERVER ranges=192.168.20.100-192.168.20.200
add name=dhcp_pool_WiFi ranges=192.168.30.100-192.168.30.200
/ip dhcp-server
add address-pool=dhcp_pool_PC interface=ether1-LAN10 lease-time=1h name=
dhcp-PC
add address-pool=dhcp_pool_SERVER interface=ether2-LAN20 lease-time=1h name=
dhcp-SERVER
add address-pool=dhcp_pool_WiFi interface=WiFi-LAN30 lease-time=1h name=
dhcp_WIFI
/routing table
add disabled=no fib name=useWAN10
add disabled=no fib name=useWAN20
add disabled=no fib name=useWAN30
/interface bridge port
add bridge=WiFi-LAN30 interface=wifi1
add bridge=WiFi-LAN30 interface=wifi2
/ip address
add address=192.168.100.81/24 interface=ether3-WAN10 network=192.168.100.0
add address=192.168.10.1/24 interface=ether1-LAN10 network=192.168.10.0
add address=192.168.20.1/24 interface=ether2-LAN20 network=192.168.20.0
add address=192.168.100.82/24 interface=ether4-WAN20 network=192.168.100.0
add address=192.168.30.1/24 interface=WiFi-LAN30 network=192.168.30.0
add address=192.168.100.83/24 interface=ether5-WAN30 network=192.168.100.0
/ip dhcp-server network
add address=192.168.10.0/24 gateway=192.168.10.1
add address=192.168.20.0/24 gateway=192.168.20.1
add address=192.168.30.0/24 gateway=192.168.30.1
/ip dns
set servers=1.1.1.1,8.8.8.8
/ip firewall nat
add action=src-nat chain=srcnat out-interface=ether3-WAN10 to-addresses=
192.168.100.81
add action=src-nat chain=srcnat out-interface=ether5-WAN30 to-addresses=
192.168.100.83
add action=src-nat chain=srcnat out-interface=ether4-WAN20 to-addresses=
192.168.100.82
/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1
pref-src=“” routing-table=main scope=30 suppress-hw-offload=no
target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.100.1%ether3-WAN10 pref-src=“” routing-table=useWAN10 scope=30
suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.100.1%ether4-WAN20 pref-src=“” routing-table=useWAN20 scope=30
suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=
192.168.100.1%ether5-WAN30 pref-src=“” routing-table=useWAN30 scope=30
suppress-hw-offload=no target-scope=10
/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.10.0/24
table=main
add action=lookup-only-in-table disabled=no dst-address=192.168.20.0/24
table=main
add action=lookup-only-in-table disabled=no dst-address=192.168.30.0/24
table=main
add action=lookup-only-in-table disabled=no dst-address=192.168.100.81/32
src-address=192.168.10.0/24 table=useWAN10
add action=lookup-only-in-table disabled=no dst-address=192.168.100.82/32
src-address=192.168.20.0/24 table=useWAN20
add action=lookup-only-in-table disabled=no dst-address=192.168.100.83/32
src-address=192.168.30.0/24 table=useWAN30
/system clock
set time-zone-name=Europe/Rome
/system ntp client
set enabled=yes
/system ntp client servers
add address=ntp.ien.it
add address=time.google.com
add address=ntp1.inrim.it

Moreover, it looks like the only rule really followed is
add dst-address=0.0.0.0/0 gateway=192.168.100.1 routing-table=MAIN
since I do not have connection on my clients if I disable that route.

Please see this topic here. Read slowly. You don’t need to implement the fail over detection unless desired.

Looks like I solved!!!

This is the final configuration:
/routing table
add disabled=no fib name=useWAN10
add disabled=no fib name=useWAN20

/ip address
add address=192.168.10.1/24 interface=ether2-LAN10 network=192.168.10.0
add address=192.168.20.1/24 interface=ether3-LAN20 network=192.168.20.0
add address=192.168.100.81/24 interface=ether4-WAN10 network=192.168.100.0
add address=192.168.100.82/24 interface=ether5-WAN20 network=192.168.100.0

/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether4-WAN10
add action=masquerade chain=srcnat out-interface=ether5-WAN20

/ip route
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1 pref-src=“” routing-table=main scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1%ether4-WAN10 pref-src=“” routing-table=useWAN10 scope=30 suppress-hw-offload=no target-scope=10
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=192.168.100.1%ether5-WAN20 pref-src=“” routing-table=useWAN20 scope=30 suppress-hw-offload=no target-scope=10

/routing rule
add action=lookup-only-in-table disabled=no dst-address=192.168.10.0/24 table=main
add action=lookup-only-in-table disabled=no dst-address=192.168.20.0/24 table=main
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 src-address=192.168.10.0/24 table=useWAN10
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 src-address=192.168.20.0/24 table=useWAN20

The problem was in the red part of anav configuration.
This is finally correct… Thanks everybody.
@pcunite: are you sure the topic you posted covers my case?
I cannot find it.

Thanks again

Are you saying the red parts are still there???
Shouldnt be??

Also in these routing rules the bits in colour can be removed!
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 src-address=192.168.10.0/24 table=useWAN10
add action=lookup-only-in-table disabled=no dst-address=0.0.0.0/0 src-address=192.168.20.0/24 table=useWAN20

Have adjusted my previous post to show that the routing rules do not require destination address of WANIP, that was in error ( non-required matching condition, unless were trying to capture incoming WAN traffic as well in the mix )

So yes, you should be good to go!
You are using masquerade rules for sourcnat still, no harm its just more accurate to use src-nat action for fixed IPs.