No WAN connectivity on VLANs

Hello everyone.

I recently decided to give my home network a somewhat radical restyling. Thus, being the configuration I previously had on my faithful RB3011UiAS-r2 totally incompatible with the new setup, I decided to reset the box, update it to the last ROS version, and to start over from scratch.

Unfortunately, I’m currently stuck in a problem I apparently and embarassingly can’t solve by myself, so I’m here begging for your wise help trying to find a viable solution.

OVERALL SETUP DESCRIPTION: The 3011 sits at the core of my network and, for what’s relevant for the current problem, it’s connected to:

  • “WAN1/2/3/4”: 4 x different WAN providers for redundancy, of which just the first one is connected until I solve the current issue (part of “WANX” list);
  • “SRVR”: the server’s VLAN, the only one allowed to connect to the outer world (part of “LANX” list);
  • Many other VLANs, not allowed to connect to the other world via the related fw rules I’m going to add once the current issue will be solved.

Every port of the 3011 - except the emergency management one - is connected to VLAN-aware devices and tagged/filtered as such.

PROBLEM DESCRIPTION: The 3011 connects to the Internet on PPPoE without any problem and everything works well from its terminal (e.g. ping, DNS resolution, firmware updates, NTP, etc.). The “LANX” bridged VLANS have no problem at all in communicating the one with the other and with the 3011, from which they can regularly receive DNS replies. The only problem is that the “LANX” VLANs (that is, the “SRVR” VLAN) can’t exit the router using its “WANX” interface (no ping, and tracert dies at the gates of the gateway).

WHAT I TRIED TO DO: I temporarily disabled the rules on the fw, and tried to modify in many ways the masquerade rule parameters, but nothing changed.

(The following config has been intentionally censored in many sections, however objectively irrelevant for the current topic.)

--------------------------------------------------------------------------------------------------
RouterOS Version: 7.x (last update)
--------------------------------------------------------------------------------------------------

/interface bridge
add name=LANX vlan-filtering=yes
add name=WANX vlan-filtering=yes

/interface ethernet
set [ find default-name=ether5 ] disabled=yes
set [ find default-name=sfp1 ] disabled=yes

/interface vlan
[CENSORED]
add interface=LANX name=SRVR vlan-id=10
add interface=WANX name=WAN1 vlan-id=70
add interface=WANX name=WAN2 vlan-id=60
add interface=WANX name=WAN3 vlan-id=50
add interface=WANX name=WAN4 vlan-id=40

/interface pppoe-client
add add-default-route=yes disabled=no interface=WAN1 name=pppoe-out1 service-name=[CENSORED] use-peer-dns=yes user=[CENSORED]

/interface list
add name=WAN
add name=LAN
add name=OTHER

/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=[CENSORED]

/ip pool
[CENSORED]

/ip dhcp-server
[CENSORED]

/port
set 0 name=serial0

/interface bridge port
add bridge=WANX frame-types=admit-only-vlan-tagged interface=ether1 pvid=70
add bridge=WANX frame-types=admit-only-vlan-tagged interface=ether2 pvid=60
add bridge=WANX frame-types=admit-only-vlan-tagged interface=ether3 pvid=50
add bridge=WANX frame-types=admit-only-vlan-tagged interface=ether4 pvid=40
[CENSORED]
add bridge=LANX frame-types=admit-only-vlan-tagged interface=ether8 pvid=10
[CENSORED]

/ip neighbor discovery-settings
set discover-interface-list=[CENSORED]

/interface bridge vlan
add bridge=WANX tagged=WANX,ether1 vlan-ids=70
add bridge=WANX tagged=WANX,ether2 vlan-ids=60
add bridge=WANX tagged=WANX,ether3 vlan-ids=50
add bridge=WANX tagged=WANX,ether4 vlan-ids=40
[CENSORED]
add bridge=LANX tagged=LANX,ether8 vlan-ids=10
[CENSORED]

/interface list member
add interface=WANX list=WAN
add interface=SRVR list=LAN
[CENSORED]

/ip address
add address=192.168.50.1/30 interface=WAN2 network=192.168.50.0
add address=192.168.100.1/30 interface=WAN3 network=192.168.100.0
add address=192.168.150.1/29 interface=WAN4 network=192.168.150.0
[CENSORED]
[CENSORED]
[CENSORED]
add address=10.128.64.254/24 interface=SRVR network=10.128.64.0

/ip cloud
set update-time=no

/ip dhcp-server lease
[CENSORED]

/ip dhcp-server network
[CENSORED]

/ip dns
set allow-remote-requests=yes servers=8.8.8.8

/ip firewall filter
add action=drop chain=input connection-state=invalid
add action=drop chain=forward connection-state=invalid
add action=accept chain=input connection-state=established,related
add action=drop chain=input in-interface-list=WAN
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes
add action=accept chain=forward connection-state=new connection-type="" in-interface-list=LAN
add action=accept chain=forward connection-state=established,related
add action=drop chain=forward connection-nat-state=!dstnat connection-state=""

/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN

/ip service
[CENSORED]

/ip ssh
[CENSORED]

/lcd
set default-screen=stat-slideshow

/lcd pin
set pin-number=[CENSORED]

/lcd interface
set ether1 disabled=yes
set ether2 disabled=yes
set ether3 disabled=yes
set ether4 disabled=yes
set ether5 disabled=yes
set sfp1 disabled=yes
set ether6 disabled=yes
set ether7 disabled=yes
set ether8 disabled=yes
set ether9 disabled=yes
set ether10 disabled=yes
add interface=WAN1 max-speed=auto
add interface=WAN2 max-speed=auto
add interface=WAN3 max-speed=auto
add interface=WAN4 max-speed=auto
[CENSORED]
add interface=SRVR max-speed=auto
[CENSORED]

/system clock
set time-zone-name=[CENSORED]

/system identity
set name=[CENSORED]

/system note
set show-at-login=no

/system routerboard settings
set auto-upgrade=yes

/tool bandwidth-server
set enabled=no

/tool mac-server
set allowed-interface-list=[CENSORED]

/tool mac-server mac-winbox
set allowed-interface-list=[CENSORED]

Thank you in anticipation for any help.

The firewall rule list is, as far as it’s shown, a bit of a mess. Any particular reason not to stick to default as far as you can make it?

Don’t set PVID on ports with frame-types=allow-only-vlan-tagged, it doesn’t change the way ports operate, but makes reading config a bit confusing.

The pppoe-out interfaces have to be members of WAN interface list for the masquerade rule to act on traffic exiting pppoe-out interfaces (which, as far as I understand, are actual WAN interfaces, your WANX (ether 1-4) ports are only providing ethernet transport layer for the pppoE tunnels.

Completely unset the conection-type=“” property of FW rule … it doesn’t match any connection (setting property to empty value is not the same as not setting property). Ditto for connection-state=“”.

Any particular reason for using two bridges? It doesn’t make any difference on RB3011 (it doesn’t offload to switch chips when VLAN filtering is enabled), but on devices which do offload bridge to switch chip, only a single bridge gets offloaded (actual rule is “one bridge per switch chip”, but vast majority of devices only have one switch chip). Again this may be an issue in your particular use case or it may not be, it’s impossible to tell due to your extensive censoring of config.

Beware that extensive use of LCD seems to bog down CPU causing lower performance.

There could be other issues with your config, but you seem to know better as you only included the “relevant” part of it.

Hello! Many thanks for your fast reply, and for all the indications you gave me.

The messy condition you noticed in the fw rules and in other sections was due to the confusion I fell into after too many silly hours spent trying to repair what was way too messed up yet, so this morning I started over again from scratch and I solved, among others, the connectivity problem you correctly addressed as the incorrect membership of the PPPoE interface.

BTW, the usage of two distinct bridges has nothing to do with performance but it’s just somewhat a visual reminder for me: I always need to visualize the whole data flow graphically with weird lifelike analogies before translating it into a script.

Thanks again for your help - I’m pretty sure I’ll need it again in the near future!

I use separate vlans to visualize different data needs ( single bridge ).

Surely that makes more sense but, in my (however questionable) small inner world, VLANs are streets, bridges are city blocks, packets are people, rules are traffic signs, services are commercial activities, and so on. It’s a weird world, but it’s the only one I have.