Routing 4 lans and 4 wans

Hi guys,

How do I go about routing WAN1 to LAN1 and WAN2 to LAN2 etc..

I tried setting up masq rules but seems like only the first one is being used.

Kind regards

/ip firewall filter
# mar/11/2020 16:44:32 by RouterOS 6.46.4
# software id = BYSX-Q3MA
#
# model = RB3011UiAS
# serial number = B8950B138191
/interface bridge
add name=bridge_default
add name=bridge_gasten
add name=bridge_lionhead
add name=bridge_voice
/interface ethernet
set [ find default-name=ether1 ] mac-address=74:4D:28:11:F8:C8 name=WAN1 speed=\
    100Mbps
set [ find default-name=ether2 ] mac-address=74:4D:28:11:F8:C9 name=WAN2 speed=\
    100Mbps
set [ find default-name=ether3 ] mac-address=74:4D:28:11:F8:CA name=WAN3 speed=\
    100Mbps
set [ find default-name=ether4 ] mac-address=74:4D:28:11:F8:CB name=WAN4 speed=\
    100Mbps
set [ find default-name=ether5 ] mac-address=74:4D:28:11:F8:CC speed=100Mbps
set [ find default-name=ether6 ] mac-address=74:4D:28:11:F8:CE speed=100Mbps
set [ find default-name=ether7 ] mac-address=74:4D:28:11:F8:CF name=\
    ether7-Lionhead speed=100Mbps
set [ find default-name=ether8 ] mac-address=74:4D:28:11:F8:D0 name=\
    ether8-RasberryPI speed=100Mbps
set [ find default-name=ether9 ] mac-address=74:4D:28:11:F8:D1 name=ether9-UNSW \
    speed=100Mbps
set [ find default-name=ether10 ] mac-address=74:4D:28:11:F8:D2 name=\
    ether10-UNSW speed=100Mbps
set [ find default-name=sfp1 ] advertise=\
    10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full mac-address=\
    74:4D:28:11:F8:CD
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp_pool0 ranges=10.0.7.20-10.0.7.254
add name=dhcp_pool1 ranges=192.168.8.2-192.168.8.254
add name=dhcp_pool2 ranges=192.168.7.2-192.168.7.254
add name=dhcp_pool3 ranges=172.16.0.2-172.16.0.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=bridge_default lease-time=1d \
    name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=bridge_gasten lease-time=\
    1d10m name=dhcp2
add address-pool=dhcp_pool2 disabled=no interface=bridge_voice lease-time=1d10m \
    name=dhcp3
add address-pool=dhcp_pool3 disabled=no interface=bridge_lionhead lease-time=1d \
    name=dhcp4
/interface pppoe-client
add disabled=no interface=WAN1 name=pppoe-out1 profile=default-encryption \
    use-peer-dns=yes user=CRPCT280363448
add disabled=no interface=WAN2 name=pppoe-out2 profile=default-encryption user=\
    CRPCT280363467
/snmp community
set [ find default=yes ] addresses=145.131.157.110/32 authentication-protocol=\
    SHA1 name=snmpv3ictsguser security=private
/interface bridge port
add bridge=bridge_default interface=ether10-UNSW
add bridge=bridge_default interface=ether9-UNSW
add bridge=bridge_default interface=ether6
add bridge=bridge_default interface=ether8-RasberryPI
add bridge=bridge_lionhead interface=ether7-Lionhead
/ip address
add address=10.0.7.1/24 interface=bridge_default network=10.0.7.0
add address=192.168.8.1/24 interface=bridge_gasten network=192.168.8.0
add address=192.168.7.1/24 interface=bridge_voice network=192.168.7.0
add address=172.16.0.1/24 interface=bridge_lionhead network=172.16.0.0
/ip dhcp-client
# DHCP client can not run on slave interface!
add add-default-route=no disabled=no interface=ether7-Lionhead
/ip dhcp-server network
add address=10.0.7.0/24 dns-server=208.91.112.53,208.91.112.52 gateway=10.0.7.1
add address=172.16.0.0/24 dns-server=208.91.112.53,208.91.112.52 gateway=\
    172.16.0.1
add address=192.168.7.0/24 dns-server=213.144.235.1,213.144.235.2 gateway=\
    192.168.7.1
add address=192.168.8.0/24 dns-server=208.91.112.53,208.91.112.52 gateway=\
    192.168.8.1

add action=drop chain=input comment="CUSTOM: WINBOX Drop Traffic to Winbox Port \
    where src-address-list!=Winbox_Admin" dst-port=8291 protocol=tcp \
    src-address-list=!Winbox_Admin
/ip firewall mangle
add action=mark-connection chain=input connection-mark=no-mark in-interface=\
    pppoe-out1 new-connection-mark=WAN1_ROS passthrough=yes
add action=mark-connection chain=input connection-mark=no-mark in-interface=\
    pppoe-out2 new-connection-mark=WAN2_ROS passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1_ROS new-routing-mark=\
    WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2_ROS new-routing-mark=\
    WAN2 passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark in-interface=\
    pppoe-out1 new-connection-mark=WAN1-LAN passthrough=no
add action=mark-connection chain=forward connection-mark=no-mark in-interface=\
    pppoe-out2 new-connection-mark=WAN2-LAN passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN1-LAN \
    new-routing-mark=WAN1 passthrough=yes src-address-list=Lan1
add action=mark-routing chain=prerouting connection-mark=WAN2-LAN \
    new-routing-mark=WAN2 passthrough=yes src-address-list=Lan2
/ip firewall nat
add action=masquerade chain=srcnat out-interface=pppoe-out1
add action=masquerade chain=srcnat out-interface=pppoe-out2
/ip firewall service-port
set sip disabled=yes
/ip route
add check-gateway=ping distance=1 gateway=pppoe-out1 routing-mark=WAN1
add check-gateway=ping distance=1 gateway=pppoe-out2 routing-mark=WAN2
add distance=1 gateway=pppoe-out1
add check-gateway=ping distance=1 gateway=pppoe-out2
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set ssh disabled=yes
set api disabled=yes
set api-ssl disabled=yes
/snmp
set enabled=yes trap-generators=interfaces,temp-exception trap-interfaces=all \
    trap-target=94.190.207.83 trap-version=3
/system clock
set time-zone-name=Europe/Bucharest

It looks like you have the routing marks setup, but you don’t define what to do with them Routing marks should lead to a separate route table for each WAN. Essentially, you need a default route on each route table for each WAN interface.

Hi Tholderbaum, thanks for your reply.

I did create the 0 routes with a routing mark, but when removing the default route from pppoe session the connection is lost with internet.

Could u maybe help me out with what needs to be added in command?

Is there a reason why you want to have each WAN tied to a LAN and not load balanced?
In other words is it not better use of available bandwidth to make all available to all users and have the router based on load provide the next connection smartly??

Also, what happens if WAN1 goes down, what then?
Are all WANIPs actually from the same cable (not four ISPs but a single ISP with one modem)?

Hi Anav, thanks for your reply.

The client wants to have 1 different PPPoE session dedicated for each different Interface / LAN like so:
WAN1>LAN1
WAN2>LAN2
WAN3>LAN3
WAN4>LAN4

Each LAN would be used by a different company, these companies shouldnt be able to reach one another. Also these companies want their own public IP.

( you only see 2 WANS in my setup right now being used )

Instead of buying 4 different routers he would like to do it on one. Giving the question, is this actually possible and how can we do this.

If connection goes offline it just goes offline, no failover whatsover

WANS are from same modem and same ISP but with different PPPoE credentials/port on modem, no vlans used and in total 1Gbps combining all 4 PPPoE session 250Mbps would be dedicated for each LAN interface. ( he would actually like to use one interface on this modem for all 4 pppoe sessions if thats possible )

Never done this kind of config, thats why asking it in here. Client is also still in testing phase, so if a reset is required then thats alright.

Might VRRP be an option?

Kind regards,

Thank for the feedback, much better understanding of the scenario and the challenge.
I am by no means equipped to apply a config, I am more requirements focussed LOL.

Interesting dilemma, I don’t think clients would every say hey I need my own PPPoE connection.
BUTY WHY are they asking for public IP? If they do not control the ROUTER, they cannot setup internal servers or anything fancy.
What are the expectations?
I need to know the why’s of everything, otherwise a design attempt is plain dumb.

In other words, don’t laugh, but I don’t see any gain from a complex approach instead of simply using onee IP address and four VLANS LOL.
There is no more throughput to distribute and if the ISP goes down, they all go down.
Load balancing/queueing can ensure that all four vlans get equal access to available bandwidth for example.

Usually a block of IPs is for a business to have a standard natted LAN, and the other IPs for more public facing SERVERS (one to one NAT) so to speak.
So this is a bit different than I am used to reading about, but could be perfectly normal.

There might be servers in the users natted lan. But take for example, this router is inside a shared patch cabinet for these 4 different clients, we dont have a subnet of public ip’s for disposal and only have the option to use PPPoE sessions. These companies don’t want to spend too much or nothing at all on switching and routing hardware, so my client is renting this out for them. But then instead of having to spend money himself and do maintenance on multiple routers he would like to move them to a single one.

The hardware that the clients do have is outdated and supports only 100Mbps, but the investment for new hardware is too high. They do bring their own servers and are required to have their own ports and stuff, but they dont want to share the ips and change excisting configs.

All in all its a hard client and instead of having to say no I would like to help him. And probably when there are port forwards required and such for the clients config he would be making them (most likely would land on my plate though, but I wouldnt be annoyed by this :smiley:)

Hope it makes sence now.

Kind regards

Did some reconfigure, but still pppoe session2 isnt used by bridge_Lan2

What am I missing?



/ip route
add check-gateway=ping distance=1 gateway=pppoe-out1 routing-mark=WAN1
add check-gateway=ping distance=1 gateway=pppoe-out2 routing-mark=WAN2
add distance=1 gateway=pppoe-out1
add check-gateway=ping distance=1 gateway=pppoe-out2

/ip firewall mangle
add action=accept chain=prerouting dst-address=86.122.48.243 in-interface=bridge_LAN2
add action=accept chain=prerouting dst-address=86.122.49.78 in-interface=bridge_LAN1
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=pppoe-out1 new-connection-mark=WAN1_ROS passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=pppoe-out2 new-connection-mark=WAN2_ROS passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN1 new-connection-mark=WAN2_ROS passthrough=no per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN2 new-connection-mark=WAN1_ROS passthrough=no per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=WAN1_ROS in-interface=bridge_LAN2 new-routing-mark=WAN1 passthrough=yes src-address-list=Lan1
add action=mark-routing chain=prerouting connection-mark=WAN2_ROS in-interface=bridge_LAN1 new-routing-mark=WAN2 passthrough=yes src-address-list=Lan2
add action=mark-routing chain=output connection-mark=WAN1_ROS new-routing-mark=WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2_ROS new-routing-mark=WAN2 passthrough=no

Your client should get out of the way and understand you are the expert on networking and design
Your client should be honest about the requirements.

You just added a potential plateful of work if there are multiple servers that will need to be programmed into the config.
What else is he not telling you?

Don’t think that some client who seems just willing to buy the cheapest hardware is going to have two clues to program an MT router.

Help yourself, by ensuring that the facts are known and that you need the whole picture, otherwise you are being taken advantage of.

Good luck on your quest!

Client is going to buy another 3 Mikrotiks but still curious on how this would work if anyone has an idea

Kind regards

Ian

For start, reverse passthrough parameters in all chain=prerouting rules. When you have passthrough=yes, it means that processing in given chain will continue. When it’s passthrough=no, it will stop with the rule that has it (if other options match).

So when you mark new incoming connections from internet, it’s ok to have passthrough=no, because you won’t do anything else with those packets. When you mark new outgoing connections from LAN, you immediatelly need to mark routing for same packets, so you need passthrough=yes. And when you mark routing, that’s the end, so it can have passthrough=no.

add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=pppoe-out1 new-connection-mark=WAN1_ROS passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark in-interface=pppoe-out2 new-connection-mark=WAN2_ROS passthrough=yes

these would be OK

add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN1 new-connection-mark=WAN2_ROS passthrough=no per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN2 new-connection-mark=WAN1_ROS passthrough=no per-connection-classifier=both-addresses:2/1

These would be No passthrough

add action=mark-routing chain=prerouting connection-mark=WAN1_ROS in-interface=bridge_LAN2 new-routing-mark=WAN1 passthrough=yes src-address-list=Lan1
add action=mark-routing chain=prerouting connection-mark=WAN2_ROS in-interface=bridge_LAN1 new-routing-mark=WAN2 passthrough=yes src-address-list=Lan2

same for this one, should be NO

add action=mark-routing chain=output connection-mark=WAN1_ROS new-routing-mark=WAN1 passthrough=no
add action=mark-routing chain=output connection-mark=WAN2_ROS new-routing-mark=WAN2 passthrough=no

and these should be YES right?

No, read my previous post again. :slight_smile:

  1. First two rules mark incoming connections and you don’t need to do anything else with packets in this direction, so you can stop here with passthrough=no (but passthrough=yes won’t break anything).
  2. Next two rules mark outgoing connections and you also need to mark routing for same packets, so here it must be passthrough=yes.
  3. Next two rules are next step after 2). And once you mark routing, you don’t need anything else, so it’s the same case as 1), passthrough=no is enough, but passthrough=yes won’t break anything:
  4. Last two rules are similar to 1) and 3), you mark routing and that’s the end.

But there’s small change, when you want static mapping, you don’t want current 2) and 3), but:

/ip firewall mangle
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN1 new-connection-mark=WAN1_ROS passthrough=yes
add action=mark-connection chain=prerouting connection-mark=no-mark dst-address-type=!local in-interface=bridge_LAN2 new-connection-mark=WAN2_ROS passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1_ROS new-routing-mark=WAN1 passthrough=no
add action=mark-routing chain=prerouting connection-mark=WAN2_ROS new-routing-mark=WAN2 passthrough=no

That’s because per-connection-classifier is for load balancing. And if each LAN has own interface, it’s enough to check that, you don’t need to look also at IP addresses.

Or you can skip all this and just do:

/ip route rule
add interface=bridge_LAN1 action=lookup-only-in-table table=WAN1
add interface=bridge_LAN2 action=lookup-only-in-table table=WAN2

Hi Sob,

Thanks for the reply, is there any downside of using this :

/ip route rule
add interface=bridge_LAN1 action=lookup-only-in-table table=WAN1
add interface=bridge_LAN2 action=lookup-only-in-table table=WAN2

Seems like its working like a charm.

Kind regards,

Ian

Not for static config like this. It would be a problem if you e.g. wanted incoming connections (forwarded ports) from WAN, which device in target LAN doesn’t use for outgoing connections. It would need other config to deal with it. It also, as is, effectively blocks access from one LAN to another.

Wait are you saying that this effectively replaces firewall rules for blocking subnets from seeing each other LOL (at L3?)

You could say that, it will make traffic from given LAN use different routing table containing only default route, i.e. it won’t be able to find other local subnets. But if you want access between LANs blocked, you probably should add firewall-level blocking anyway, for keeping good habits.

Hi Sob,

Yea the subnets wouldnt need to talk between eachothers, this solution is quiet good and isnt using performance.

Thanks!

Ian

Hi Sob,

Hopefully you are still around, seems the client wants to have traffic between 2 of the bridges. Is there I could make that happen with the route rules?

Let me know!

Kind regards,

Ian

Sure, more routing rules:

/ip route rule
add action=lookup-only-in-table dst-address=<local subnet 1> table=main
add action=lookup-only-in-table dst-address=<local subnet 2> table=main

And order matters, so these need to be first.