Dealing with VLANs on cisco switch.

I had no success to make a PPPoE connection, when the wan interface is bridged to bridge1 the pppoe-client turns red saying it should not be on a slave interface..

/interface bridge
add fast-forward=no name=bridge1
/interface vlan
add interface=ether9 name=10.10.10.1 vlan-id=10
add interface=ether9 name=10.10.20.1 vlan-id=20
add interface=ether9 name=10.10.30.1 vlan-id=30
add interface=ether9 name=10.10.40.1 vlan-id=40
add interface=ether9 name=10.10.50.1 vlan-id=50
add interface=ether9 name=10.10.60.1 vlan-id=60
add interface=ether9 name=10.10.70.1 vlan-id=70
add interface=ether9 name=10.10.80.1 vlan-id=80
add interface=ether9 name=10.10.90.1 vlan-id=90
add interface=ether1 name=wan vlan-id=959
/interface pppoe-client
# Client is on slave interface
add disabled=no interface=wan name=pppoe-out1
/interface list
add name=WAN
add name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 horizon=3 interface=10.10.10.1
add bridge=bridge1 horizon=3 interface=10.10.20.1
add bridge=bridge1 horizon=3 interface=10.10.30.1
add bridge=bridge1 horizon=3 interface=10.10.40.1
add bridge=bridge1 horizon=3 interface=10.10.50.1
add bridge=bridge1 horizon=3 interface=10.10.60.1
add bridge=bridge1 horizon=3 interface=10.10.70.1
add bridge=bridge1 horizon=3 interface=10.10.80.1
add bridge=bridge1 horizon=3 interface=10.10.90.1
add bridge=bridge1 interface=wan
/interface list member
add interface=10.10.10.1 list=LAN
add interface=10.10.20.1 list=LAN
add interface=10.10.30.1 list=LAN
add interface=10.10.40.1 list=LAN
add interface=10.10.50.1 list=LAN
add interface=10.10.60.1 list=LAN
add interface=10.10.70.1 list=LAN
add interface=10.10.80.1 list=LAN
add interface=10.10.90.1 list=LAN
add interface=pppoe-out1 list=WAN
/ip address
add address=10.10.10.1/24 interface=10.10.10.1 network=10.10.10.0
add address=10.10.20.1/24 interface=10.10.20.1 network=10.10.20.0
add address=10.10.30.1/24 interface=10.10.30.1 network=10.10.30.0
add address=10.10.40.1/24 interface=10.10.40.1 network=10.10.40.0
add address=10.10.50.1/24 interface=10.10.50.1 network=10.10.50.0
add address=10.10.60.1/24 interface=10.10.60.1 network=10.10.60.0
add address=10.10.70.1/24 interface=10.10.70.1 network=10.10.70.0
add address=10.10.80.1/24 interface=10.10.80.1 network=10.10.80.0
add address=10.10.90.1/24 interface=10.10.90.1 network=10.10.90.0
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

But there is no reason to put the interface named wan to a bridge. The hierarchy is ether1 (physical interface) to which the tagged side of /interface vlan name=wan is connected, and to the tagless side of that /interface vlan the /interface pppoe-client name=pppoe-out1 is connected. The /interface pppoe-client gets an IP address from the ISP (check that using /ip address print) and the routing together with NAT will provide connection of devices in the VLANs to internet.

I’m currently testing the setup on a small environment, the network was shut down when i first plugged the router!!

/interface bridge
add fast-forward=no name=bridge1
/interface vlan
add interface=ether9 name=10.10.10.1 vlan-id=10
add interface=ether9 name=10.10.20.1 vlan-id=20
add interface=ether9 name=10.10.30.1 vlan-id=30
add interface=ether9 name=10.10.40.1 vlan-id=40
add interface=ether9 name=10.10.50.1 vlan-id=50
add interface=ether9 name=10.10.60.1 vlan-id=60
add interface=ether9 name=10.10.70.1 vlan-id=70
add interface=ether9 name=10.10.80.1 vlan-id=80
add interface=ether9 name=10.10.90.1 vlan-id=90
add interface=ether1 name=wan vlan-id=959
/interface pppoe-client
add interface=wan name=pppoe-out1
/interface list
add name=LAN
add name=WAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 horizon=1 interface=ether1
add bridge=bridge1 horizon=3 interface=10.10.10.1
add bridge=bridge1 horizon=3 interface=10.10.20.1
add bridge=bridge1 horizon=3 interface=10.10.30.1
add bridge=bridge1 horizon=3 interface=10.10.40.1
add bridge=bridge1 horizon=3 interface=10.10.50.1
add bridge=bridge1 horizon=3 interface=10.10.60.1
add bridge=bridge1 horizon=3 interface=10.10.70.1
add bridge=bridge1 horizon=3 interface=10.10.80.1
add bridge=bridge1 horizon=3 interface=10.10.90.1
/interface list member
add interface=10.10.10.1 list=LAN
add interface=10.10.20.1 list=LAN
add interface=10.10.30.1 list=LAN
add interface=10.10.40.1 list=LAN
add interface=10.10.50.1 list=LAN
add interface=10.10.60.1 list=LAN
add interface=10.10.70.1 list=LAN
add interface=10.10.80.1 list=LAN
add interface=10.10.90.1 list=LAN
add interface=pppoe-out1 list=WAN
/ip address
add address=10.10.10.1/24 interface=10.10.10.1 network=10.10.10.0
add address=10.10.20.1/24 interface=10.10.20.1 network=10.10.20.0
add address=10.10.30.1/24 interface=10.10.30.1 network=10.10.30.0
add address=10.10.40.1/24 interface=10.10.40.1 network=10.10.40.0
add address=10.10.50.1/24 interface=10.10.50.1 network=10.10.50.0
add address=10.10.60.1/24 interface=10.10.60.1 network=10.10.60.0
add address=10.10.70.1/24 interface=10.10.70.1 network=10.10.70.0
add address=10.10.80.1/24 interface=10.10.80.1 network=10.10.80.0
add address=10.10.90.1/24 interface=10.10.90.1 network=10.10.90.0
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

What means “was shut down”? What exactly you did? I can still see that ether1 is added as a member port of bridge1 which makes no sense in the setup.

The internet connections was terminated when i plugged the router, i had to reboot it to make it run again.
I have removed the ether1 from the bridge and still can not establish a PPPoE connection.

/interface bridge
add fast-forward=no name=bridge1
/interface vlan
add interface=ether9 name=10.10.10.1 vlan-id=10
add interface=ether9 name=10.10.20.1 vlan-id=20
add interface=ether9 name=10.10.30.1 vlan-id=30
add interface=ether9 name=10.10.40.1 vlan-id=40
add interface=ether9 name=10.10.50.1 vlan-id=50
add interface=ether9 name=10.10.60.1 vlan-id=60
add interface=ether9 name=10.10.70.1 vlan-id=70
add interface=ether9 name=10.10.80.1 vlan-id=80
add interface=ether9 name=10.10.90.1 vlan-id=90
add interface=ether1 name=wan vlan-id=959
/interface pppoe-client
add interface=wan name=pppoe-out1
/interface list
add name=LAN
add name=WAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
add bridge=bridge1 horizon=3 interface=10.10.10.1
add bridge=bridge1 horizon=3 interface=10.10.20.1
add bridge=bridge1 horizon=3 interface=10.10.30.1
add bridge=bridge1 horizon=3 interface=10.10.40.1
add bridge=bridge1 horizon=3 interface=10.10.50.1
add bridge=bridge1 horizon=3 interface=10.10.60.1
add bridge=bridge1 horizon=3 interface=10.10.70.1
add bridge=bridge1 horizon=3 interface=10.10.80.1
add bridge=bridge1 horizon=3 interface=10.10.90.1
/interface list member
add interface=10.10.10.1 list=LAN
add interface=10.10.20.1 list=LAN
add interface=10.10.30.1 list=LAN
add interface=10.10.40.1 list=LAN
add interface=10.10.50.1 list=LAN
add interface=10.10.60.1 list=LAN
add interface=10.10.70.1 list=LAN
add interface=10.10.80.1 list=LAN
add interface=10.10.90.1 list=LAN
add interface=pppoe-out1 list=WAN
/ip address
add address=10.10.10.1/24 interface=10.10.10.1 network=10.10.10.0
add address=10.10.20.1/24 interface=10.10.20.1 network=10.10.20.0
add address=10.10.30.1/24 interface=10.10.30.1 network=10.10.30.0
add address=10.10.40.1/24 interface=10.10.40.1 network=10.10.40.0
add address=10.10.50.1/24 interface=10.10.50.1 network=10.10.50.0
add address=10.10.60.1/24 interface=10.10.60.1 network=10.10.60.0
add address=10.10.70.1/24 interface=10.10.70.1 network=10.10.70.0
add address=10.10.80.1/24 interface=10.10.80.1 network=10.10.80.0
add address=10.10.90.1/24 interface=10.10.90.1 network=10.10.90.0
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN

You’ve plugged the router in parallel to existing one? Show a diagram of the existing setup, the new setup and how you interconnect them. My whole understanding so far was that you connect ether1 to the wire coming from the ISP on which the PPPoE is in vlan 959 instead of the original router. You usually cannot have several PPPoE clients using the same account simultaneously.

The diagram that i posted earlier is what the setup should look like, i tested the setup via a switch (not router) connected to the ISP as you said in parallel, the reason for this is i didn’t want any unnecessary down time for the network. So, i understand from you that i should test it directly to ISP.

I have tested the setup directly, but there is no avail so far :frowning:

I don’t know how to explain that if I don’t understand the current setup, I cannot suggest one for the Mikrotik replacing/extending the current setup.

If you connect the Cisco to the cable from the ISP box directly, instead of putting Mikrotik in between, does it work? Where is the PPPoE client in the current setup? On the Cisco, in the ISP box, in one of the PCs connected to Cisco?

This is the detailed diagram of the setup that i’m looking for.
Untitled Diagram.jpg

If i connect a PC directly to the cable from the ISP the PPPoE will work, if i plug the cable into a switch and connect a PC to that switch the PPPoE works, if i separate internal networks via physical ports like how i did here>> Connecting multiple networks. - #26 by digitalinee the PPPoE will work again. The problems starts when VLANS involve in the equation.

So is VLAN 959 tagged towards the MikroTik router from the ISP Cisco Distribution Switch or not?

Can you validate tagged or untagged by using the MikroTik to perform a packet capture using /tool sniffer?

The cisco switch if plugged directly to the isp distribution switch will work okay and provide service as well but it will keep showing notifications every minute in the CLI of native vlan mismatch 959, it will gone if i switchport all the ports to vlan 959.
So does it mean tagged or untagged?

As all the PPPoE clients run on the PCs directly, all the PCs must be connected directly into VLAN959. This was not clear from the initial information, so I was expecting that the PCs have normal static or DHCP IP configuration and there is a router between the private IP subnets each living on one of the VLANs and the uplink subnet.

More than that, I’m not sure how it could actually work in the original state (without the Mikrotik) and what do you want to achieve by inserting the Mikrotik into the scheme. Please post the output of show running-config from the central Cisco in the picture and any single one of those to which the PC client is connected, and separately give the names/numbers of the ports used (to which port of the satellite Cisco the PC is connected, which ports of the sattellite Cisco and the central one are used for their interconnection, which port of the central Cisco is connected to the ISP Cisco).

It seems to me that the VLANs on the Ciscos are actually unused and everything actually lives because the default VLAN is the same on all the Ciscos, so the PPPoE communication runs tagless on all cables.

It is my bad that i didn’t told you that what i’m asking here is for upgrading the current network, it is not what the actual current network setup is.
The current setup is a simple layer 2 switch environment for all clients on default VLANs i didn’t even switch them to VLAN 959, and they work just fine to connect to PPPoE server.
All what i was posting here was happening in a test environment with totally different physical devices for testing purposes only away from the actual network devices, so as soon as it works i will transfer the setup to the actual network.

This is the current actual setup:
Note that the clients are over 100 but for demonstration and simplicity i use 9 only.
Untitled Diagram (4).jpg

OK, so we’re getting somewhere. The purpose of ID-based VLANs is to isolate several L2 address spaces using the same physical infrastructure from each other, but all the PPPoE clients and their PPPoE server must all share the same L2 address space. So in the current configuration, all the PPPoE clients must be connected to VLAN 959 directly, or some of the elements would have to translate between VLAN ID 959 on one side and some other VLAN ID on the other side. But you can not use any kind of VLAN translation to make all the PCs see the PPPoE server but not see each other.

As most Windows network card drivers do not understand the idea of VLAN tagging, you also cannot let the PPPoE clients on the Windows to use one VLAN ID and attach some other IP address to another VLAN ID on the same Ethernet interface of the PC, allowing the internal communication between the PCs to use that other VLAN ID (or more VLAN IDs if you would like to set some restrictions on which PCs can talk to which other ones).

So one possibility would be to set the Mikrotik to run one PPPoE server in each of the 10 VLANs, let the PCs from that VLAN to connect to that Mikrotik using PPPoE, and let the Mikrotik use its local PPPoE clients to connect to the ISP’s server in the name of each of the PCs. Another possibility could be to let only the PPPoE frames leak between the VLANs using some kind of bridge filters, but it is only a theoretical idea I haven’t thought through thoroughly.

So please provide the description regarding the application layer purpose of partitioning your network for which you wanted to use the VLANs so that we could find an optimal method of achieving that goal.

Hi Sindy!!
Please consider this setup that works perfectly, it uses physical interfaces so it is limited to the number of ports on the router .
My requirement is a duplicate setup that uses VLANs instead of physical interfaces.

/interface bridge
add name=bridge1
/interface bridge port
add bridge=bridge1 horizon=1 interface=ether1
add bridge=bridge1 horizon=2 interface=ether2
add bridge=bridge1 horizon=3 interface=ether3
add bridge=bridge1 horizon=3 interface=ether4
add bridge=bridge1 horizon=3 interface=ether5
add bridge=bridge1 horizon=3 interface=ether6
add bridge=bridge1 horizon=3 interface=ether7
add bridge=bridge1 horizon=3 interface=ether8
add bridge=bridge1 horizon=3 interface=ether9
/ip address
add address=172.19.59.250/24 interface=ether1 network=172.19.59.0
add address=10.10.20.1/24 interface=ether2 network=10.10.20.0
add address=10.10.30.1/24 interface=ether3 network=10.10.30.0
add address=10.10.40.1/24 interface=ether4 network=10.10.40.0
add address=10.10.50.1/24 interface=ether5 network=10.10.50.0
add address=10.10.60.1/24 interface=ether6 network=10.10.60.0
add address=10.10.70.1/24 interface=ether7 network=10.10.70.0
add address=10.10.80.1/24 interface=ether8 network=10.10.80.0
add address=10.10.90.1/24 interface=ether9 network=10.10.90.0
/ip firewall nat
add action=masquerade chain=srcnat
/ip route
add distance=1 gateway=172.19.59.254

That’s nice, as it confirms what I’ve expected - the PPPoE clients are all in the same VLAN like the PPPoE server. But this setup would work even without the IP addresses. So the question is what is the role of the IP addresses you’ve assigned to interfaces given that the PCs get their IP addresses from the PPPoE server. Leaving aside that IP addresses assigned to interfaces which are member ports of bridges do not work, or rather half-work.

The IP addresses are needed in order to communicate via layer 3 between these networks, ether1 (WAN) horizon is 1, ether2 horizon is 2, others are 3, ether2 is for admin for now so he could access all networks.
The PPPoE IPs are received from the ISP PPPoE server and they are not concerning me, all i want is to clear the way between the client and the ISP PPPoE server.

So the goal is to maintain communication between the PCs at L3 on the subnet addresses and additionaly isolate these subnets at L2 using VLANs, but at the same time preserve the visibility between all the PPPoE clients and the PPPoE server, which would mean to use another VLAN ID for the PPPoE traffic than for the L3 one, while the PCs send both the frames carrying L3 packets the frames carrying PPPoE untagged.

I could imagine how to do this at the switch ports to which the PCs are connected, where the ingress frames could be tagged to the correct VLAN depending on their ethertype, i.e. frames with one of the PPPoE ethertypes would be tagged with VLAN 959 and frames with IP ethertype would be tagged with e.g. VLAN 10. On egress, frames with either VID would be untagged. This would of course require that such L2 translation capability would be supported by the switch.

The same could still be achieved at the ports of the central Cisco if each satellite Cisco would serve only one VLAN, as the translation rules could be individual for each port.

But I have no idea how to achieve this at the central element where already several VLANs come in through the same port. In the PC->ISP direction, you could still remove or rewrite the VLAN tag for PPPoE packets using _/interface ethernet switch rule_s if your Mikrotik model has a switch chip which supports them; but in the ISP->PC direction, no information is available which would allow to choose the correct VID so that the central Cisco switch could forward the frame to the correct satellite Cisco. At L2, there is no equivalent of L3 connection tracking which would allow to dynamically create a mapping context between the MAC address and the VID.

So the only solution I can imagine is the one described earlier, to cut each PPPoE session into two, one between the PC and the Mikrotik and another one between the Mikrotik and the ISP’s PPPoE server, but in this case the Mikrotik would have to “know” all the PPPoE usernames and passwords.