Community discussions

 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Routing between VLANs stopped working after PCC load balancing.

Wed Jul 18, 2018 11:41 am

Hi everybody,
First I'm sorry for there are so much topic of PPC Load Balancing, I have searched some topics but It doesn't solve my own problem.
I'm setting a network with 2 WAN using PPPOE, It worked well until I create new Vlans (First I just had one LAN). The problem is that I can't access through Vlans after PPC Load Balancing. Below is my script, is there anyone here who has had the same problems?.
I would appreciate hearing your opinion on this case.
Thanks

Code: Select all

# jul/18/2018 15:24:01 by RouterOS 6.42.4
# software id = 5RKA-6A72
#
# model = CRS326-24G-2S+
# serial number = 763C07B33D5A
/interface bridge
add fast-forward=no name=LAN
add fast-forward=no name=Trunk vlan-filtering=yes
/interface pppoe-client
add disabled=no interface=ether1 name=pppoe-out1 user=user1
add disabled=no interface=ether2 name=pppoe-out2 user=user2
/interface vlan
add interface=Trunk name=vlan10 vlan-id=10
add interface=Trunk name=vlan20 vlan-id=20
add interface=Trunk name=vlan30 vlan-id=30
add interface=Trunk name=vlan40 vlan-id=40
/ip pool
add name=dhcp_pool0 ranges=192.168.1.2-192.168.1.254
add name=dhcp_pool1 ranges=10.10.10.2-10.10.10.254
add name=dhcp_pool2 ranges=10.10.30.2-10.10.30.254
add name=dhcp_pool3 ranges=10.10.20.2-10.10.20.254
add name=dhcp_pool4 ranges=10.10.40.2-10.10.40.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=LAN name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=vlan10 name=dhcp2
add address-pool=dhcp_pool2 disabled=no interface=vlan30 name=dhcp3
add address-pool=dhcp_pool3 disabled=no interface=vlan20 name=dhcp4
add address-pool=dhcp_pool4 disabled=no interface=vlan40 name=dhcp5
/interface bridge port
add bridge=LAN interface=ether3
add bridge=Trunk interface=ether4 pvid=10
add bridge=Trunk interface=ether6 pvid=20
add bridge=Trunk interface=ether8 pvid=30
add bridge=Trunk interface=ether10 pvid=40
/interface bridge vlan
add bridge=Trunk tagged=ether6,ether8,ether10,Trunk untagged=ether4 vlan-ids=10
add bridge=Trunk tagged=ether4,ether8,ether10,Trunk untagged=ether6 vlan-ids=20
add bridge=Trunk tagged=ether4,ether6,ether10,Trunk untagged=ether8 vlan-ids=30
add bridge=Trunk tagged=ether4,ether6,ether8,Trunk untagged=ether10 vlan-ids=40
/ip address
add address=192.168.1.1/24 interface=LAN network=192.168.1.0
add address=10.10.10.1/24 interface=vlan10 network=10.10.10.0
add address=10.10.20.1/24 interface=vlan20 network=10.10.20.0
add address=10.10.30.1/24 interface=vlan30 network=10.10.30.0
add address=10.10.40.1/24 interface=vlan40 network=10.10.40.0
/ip dhcp-server network
add address=10.10.10.0/24 gateway=10.10.10.1
add address=10.10.20.0/24 gateway=10.10.20.1
add address=10.10.30.0/24 gateway=10.10.30.1
add address=10.10.40.0/24 gateway=10.10.40.1
add address=192.168.1.0/24 gateway=192.168.1.1
/ip dns
set servers=8.8.8.8
/ip firewall mangle
add action=mark-connection chain=input in-interface=pppoe-out1 \
new-connection-mark=WAN1_conn passthrough=yes
add action=mark-connection chain=input in-interface=pppoe-out2 \
new-connection-mark=WAN2_conn passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1_conn \
new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_conn \
new-routing-mark=to_WAN2 passthrough=yes
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=LAN new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=LAN new-connection-mark=WAN2_conn passthrough=yes \
per-connection-classifier=both-addresses:2/1
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan10 new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan10 new-connection-mark=WAN2_conn passthrough=yes \
per-connection-classifier=both-addresses:2/1
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan20 new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan20 new-connection-mark=WAN2_conn passthrough=yes \
per-connection-classifier=both-addresses:2/1
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan30 new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan30 new-connection-mark=WAN2_conn passthrough=yes \
per-connection-classifier=both-addresses:2/1
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan40 new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses:2/0
add action=mark-connection chain=prerouting dst-address-type=!local \
in-interface=vlan40 new-connection-mark=WAN2_conn passthrough=yes \
per-connection-classifier=both-addresses:2/1
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
in-interface=LAN new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn \
in-interface=LAN new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
in-interface=vlan10 new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn \
in-interface=vlan10 new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
in-interface=vlan20 new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn
in-interface=vlan20 new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1_conn
in-interface=vlan30 new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn
in-interface=vlan30 new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN1_conn
in-interface=vlan40 new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn
in-interface=vlan40 new-routing-mark=to_WAN2 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface=pppoe-out1
add action=masquerade chain=srcnat out-interface=pppoe-out2
/ip route
add check-gateway=ping distance=1 gateway=pppoe-out1 routing-mark=t
add check-gateway=ping distance=1 gateway=pppoe-out2 routing-mark=t
add check-gateway=ping distance=1 gateway=pppoe-out1
add check-gateway=ping distance=2 gateway=pppoe-out2
Last edited by TheLordOfTheShells on Wed Jul 18, 2018 5:41 pm, edited 2 times in total.
 
sindy
Forum Guru
Forum Guru
Posts: 3760
Joined: Mon Dec 04, 2017 9:19 pm

Re: VLANs doesn't work after PCC load balancing.  [SOLVED]

Wed Jul 18, 2018 12:30 pm

Let's translate the L2 term "VLANs" to an L3 term "connected subnets", i.e. IP subnets in which the Mikrotik has its own IP addresses.

A connected subnet also needs a route; it is added dynamically (i.e. automatically once the IP address/mask configuration is added), with distance=0, but only to the default routing table (called "main").

You seem to be another victim of wishful thinking, assuming that dst-address-type=local matches to any destination address from any connected subnet, while in reality it only matches Mikrotik's own addresses. So by using dst-address-type=!local in the mangle rules assigning connection marks, you do not prevent packets towards local subnets from being connection-marked, so these packets consequently get routing marks, and the route for them is chosen among those bearing the same routing mark.

Since a matching route with that routing mark exists, these packets are routed out via WAN 1 or WAN 2 rather than being sent to the connected LAN subnet because the route for that subnet only exists in routing table "main".

So while the topics title, "vlans do not work" suggests that VLAN tagging stopped working, actually routing between connected subnets (which in your case happen to occupy VLANs) stopped working.

But you want a solution, right? One possibility is to create an address list of all the connected subnets:
/ip firewall address-list
add list=connected-subnets address=192.168.1.1/24
add list=connected-subnets address=10.10.10.1/24
add list=connected-subnets address=10.10.20.1/24
add list=connected-subnets address=10.10.30.1/24
add list=connected-subnets address=10.10.40.1/24

and use dst-address-list=!connected-subnets instead of dst-address-type=!local in the action=mark-connection rules.

Another (simpler to configure but possibly slightly more CPU-intensive) way is to use

/ip route rule
add action=lookup table=main dst-address=192.168.1.1/24
add action=lookup table=main dst-address=10.10.10.1/24
add action=lookup table=main dst-address=10.10.20.1/24
add action=lookup table=main dst-address=10.10.30.1/24
add action=lookup table=main dst-address=10.10.40.1/24

to negate the effect of the routing-mark (route rules are evaluated just before route lookup and override or translate an eventually existing routing-mark assignment)

Unrelated to your issue, just to "structured-oriented firewalling": whichever of the two ways above you choose, I would recommend you to use an interface list:
/interface list
add name=all-lans
/interface list member
add list=all-lans interface=LAN
add list=all-lans interface=vlan10
add list=all-lans interface=vlan20
add list=all-lans interface=vlan30
add list=all-lans interface=vlan40

and replace each 5 identical mangle rules differing only in in-interface value by a single rule using in-interface-list=all-lans instead.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Re: VLANs doesn't work after PCC load balancing.

Wed Jul 18, 2018 1:59 pm

Let's translate the L2 term "VLANs" to an L3 term "connected subnets", i.e. IP subnets in which the Mikrotik has its own IP addresses.

A connected subnet also needs a route; it is added dynamically (i.e. automatically once the IP address/mask configuration is added), with distance=0, but only to the default routing table (called "main").

You seem to be another victim of wishful thinking, assuming that dst-address-type=local matches to any destination address from any connected subnet, while in reality it only matches Mikrotik's own addresses. So by using dst-address-type=!local in the mangle rules assigning connection marks, you do not prevent packets towards local subnets from being connection-marked, so these packets consequently get routing marks, and the route for them is chosen among those bearing the same routing mark.

Since a matching route with that routing mark exists, these packets are routed out via WAN 1 or WAN 2 rather than being sent to the connected LAN subnet because the route for that subnet only exists in routing table "main".

So while the topics title, "vlans do not work" suggests that VLAN tagging stopped working, actually routing between connected subnets (which in your case happen to occupy VLANs) stopped working.

But you want a solution, right? One possibility is to create an address list of all the connected subnets:
/ip firewall address-list
add list=connected-subnets address=192.168.1.1/24
add list=connected-subnets address=10.10.10.1/24
add list=connected-subnets address=10.10.20.1/24
add list=connected-subnets address=10.10.30.1/24
add list=connected-subnets address=10.10.40.1/24

and use dst-address-list=!connected-subnets instead of dst-address-type=!local in the action=mark-connection rules.

Another (simpler to configure but possibly slightly more CPU-intensive) way is to use

/ip route rule
add action=lookup table=main dst-address=192.168.1.1/24
add action=lookup table=main dst-address=10.10.10.1/24
add action=lookup table=main dst-address=10.10.20.1/24
add action=lookup table=main dst-address=10.10.30.1/24
add action=lookup table=main dst-address=10.10.40.1/24

to negate the effect of the routing-mark (route rules are evaluated just before route lookup and override or translate an eventually existing routing-mark assignment)

Unrelated to your issue, just to "structured-oriented firewalling": whichever of the two ways above you choose, I would recommend you to use an interface list:
/interface list
add name=all-lans
/interface list member
add list=all-lans interface=LAN
add list=all-lans interface=vlan10
add list=all-lans interface=vlan20
add list=all-lans interface=vlan30
add list=all-lans interface=vlan40

and replace each 5 identical mangle rules differing only in in-interface value by a single rule using in-interface-list=all-lans instead.
Thank you so much for your kindly support.
P/s: I have also changed the topic title to fit the content.
 
sindy
Forum Guru
Forum Guru
Posts: 3760
Joined: Mon Dec 04, 2017 9:19 pm

Re: VLAN tagging stopped working after PCC load balancing.

Wed Jul 18, 2018 3:57 pm

Nope :-) Now the title is definitely wrong. "Routing between VLANs stopped working ..." would be a correct one.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Re: VLAN tagging stopped working after PCC load balancing.

Wed Jul 18, 2018 5:49 pm

Nope :-) Now the title is definitely wrong. "Routing between VLANs stopped working ..." would be a correct one.
Hope everything will be fine now :lol:
P/s: Thank you once again.
 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Re: Routing between VLANs stopped working after PCC load balancing.

Mon Jul 30, 2018 1:32 pm

Hi Sindy.
Sorry to bother you again.
But after apply those configuration I feel like there is some problem with forwarding port. Inside the local i want to NAT some port to out side internet, but it seem not working.
Hope you will check my configuration again and give me some opinions.
Thanks
 
sindy
Forum Guru
Forum Guru
Posts: 3760
Joined: Mon Dec 04, 2017 9:19 pm

Re: Routing between VLANs stopped working after PCC load balancing.

Mon Jul 30, 2018 2:07 pm

Can you be more precise? In the only configuration posted above, there is nothing what would resemble a port forwarding setup, so what have you already tried and what didn't work?
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Re: Routing between VLANs stopped working after PCC load balancing.

Mon Jul 30, 2018 4:27 pm

Can you be more precise? In the only configuration posted above, there is nothing what would resemble a port forwarding setup, so what have you already tried and what didn't work?
Hi Sindy.
Here is the mangle code which i have follow your guild before.

Code: Select all

/ip firewall mangle
add action=mark-connection chain=input in-interface=pppoe-out1 \
new-connection-mark=WAN1_conn passthrough=yes
add action=mark-connection chain=input in-interface=pppoe-out2 \
new-connection-mark=WAN2_conn passthrough=yes
add action=mark-connection chain=input in-interface=ether3 new-connection-mark=\
WAN3_conn passthrough=yes
add action=mark-routing chain=output connection-mark=WAN1_conn \
new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN2_conn \
new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=output connection-mark=WAN3_conn \
new-routing-mark=to_WAN3 passthrough=yes
add action=mark-connection chain=prerouting dst-address-list=!connected-subnets \
in-interface-list=all-lans new-connection-mark=WAN1_conn passthrough=yes \
per-connection-classifier=both-addresses-and-ports:3/0
add action=mark-connection chain=prerouting dst-address-list=!connected-subnets \
dst-address-type="" in-interface-list=all-lans new-connection-mark=\
WAN2_conn passthrough=yes per-connection-classifier=\
both-addresses-and-ports:3/1
add action=mark-connection chain=prerouting dst-address-list=!connected-subnets \
dst-address-type="" in-interface-list=all-lans new-connection-mark=\
WAN3_conn passthrough=yes per-connection-classifier=\
both-addresses-and-ports:3/2
add action=mark-routing chain=prerouting connection-mark=WAN1_conn \
in-interface-list=all-lans new-routing-mark=to_WAN1 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN2_conn \
in-interface-list=all-lans new-routing-mark=to_WAN2 passthrough=yes
add action=mark-routing chain=prerouting connection-mark=WAN3_conn \
in-interface-list=all-lans new-routing-mark=to_WAN3 passthrough=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether3
add action=masquerade chain=srcnat out-interface=pppoe-out1
add action=masquerade chain=srcnat out-interface=pppoe-out2
add action=dst-nat chain=dstnat dst-port=8080 in-interface=pppoe-out1 protocol=\
tcp to-addresses=10.1.1.5 to-ports=80
/ip route
add check-gateway=ping distance=1 gateway=pppoe-out1 routing-mark=to_WAN1
add check-gateway=ping distance=1 gateway=pppoe-out2 routing-mark=to_WAN2
add check-gateway=ping distance=1 gateway=45.xxx.xxx.xxx routing-mark=to_WAN3
add distance=1 gateway=45.xxx.xxx.xxx
add distance=2 gateway=pppoe-out1
add distance=3 gateway=pppoe-out2
But when i nat port 80 and check it in ping.eu the result show that is closed.
Do you have any idea about that.
Thanks
 
sindy
Forum Guru
Forum Guru
Posts: 3760
Joined: Mon Dec 04, 2017 9:19 pm

Re: Routing between VLANs stopped working after PCC load balancing.

Mon Jul 30, 2018 4:36 pm

Yes, the idea is that while you've properly added the action=dst-nat rule to /ip fireewall nat chain=dstnat, you haven't permitted the packets from WAN(s) to LAN to be forwarded in the /ip firewall filter chain=forward. Not knowing how your rules look like, I just suggest you that there is a nice matcher, connection-nat-state, and if you put action=accept chain=forward in-interface-list=WAN connection-nat-state=dstnat to an appropriate place, you will let through anything what you dst-nat in the nat table.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
TheLordOfTheShells
just joined
Topic Author
Posts: 16
Joined: Tue Oct 03, 2017 2:48 am

Re: Routing between VLANs stopped working after PCC load balancing.

Tue Jul 31, 2018 6:13 am

Yes, the idea is that while you've properly added the action=dst-nat rule to /ip fireewall nat chain=dstnat, you haven't permitted the packets from WAN(s) to LAN to be forwarded in the /ip firewall filter chain=forward. Not knowing how your rules look like, I just suggest you that there is a nice matcher, connection-nat-state, and if you put action=accept chain=forward in-interface-list=WAN connection-nat-state=dstnat to an appropriate place, you will let through anything what you dst-nat in the nat table.
It's my mistake, the service was not running yet When I check port opening.
I also found that my server can not connect from inside Lan through Ip public, After some google I found the reason https://wiki.mikrotik.com/wiki/Hairpin_NAT.
But after change the NAT rule i still can not connect to my server inside LAN.
Here is the NAT code

Code: Select all

/ip firewall nat
add action=masquerade chain=srcnat dst-address=192.168.15.77 dst-address-type="" dst-port=8080 out-interface-list=all-lans protocol=tcp src-address-list=connected-subnets
add action=dst-nat chain=dstnat dst-address=14.xxx.xxx.xxx dst-address-type=local dst-port=8080 protocol=tcp to-addresses=192.168.15.77 to-ports=80
Thank you very much for your cooperation and time
 
sindy
Forum Guru
Forum Guru
Posts: 3760
Joined: Mon Dec 04, 2017 9:19 pm

Re: Routing between VLANs stopped working after PCC load balancing.

Tue Jul 31, 2018 12:18 pm

The srcnat chain (where the masquerade and src-nat rules live) is parsed as one of the last ones in the chain - this picture has it all. So in the masquerade rule you have posted above, just replace dst-port=8080 (which is only true until the dst-nat is made) by dst-port=80 (which is the value the packet has when it reaches the masquerade rule).
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.

Who is online

Users browsing this forum: Google [Bot] and 68 guests