End customer vpls Internet connection from ISP

vnic0 · June 15, 2023, 10:52pm

Hello All,

I have a L2 vpls from my ISP to connect 5 locations and access Internet. The ISP has provided a single 1Gbps connection. They provided the below information, but I do not seem to be able to reach the internet from the CCR2116. The ISP says they have tagged all the vlan's and the assigned subnet is routing from the ISP core to the CE IP, and all I need to do is configure a static route from my assigned network pointing to the ISP PE IP. I have not been able to get this working, there is a lot of information about vpls for mikrotik but not from the end customer perspective.

Would anyone have suggestions or hints for what I am missing or doing incorrectly to get this working?

ISP PE IP: 261.555.164.153/30
ISP CE IP: 261.555.164.154/30
Assigned Network Address: 261.444.194.96/28
Assigned customer inside Address: 261.444.194.97/28
Assigned useable IP's: 261.444.194.98...110/28

Config I have so far

create one bridge, set VLAN mode off while we configure

/interface bridge add name=primarybridge comment="Primary Bridge" protocol-mode=none vlan-filtering=no

LAN facing IP address on a MGMT_VLAN

/interface vlan add interface=primarybridge name=40-MGMT vlan-id=40
/ip address add address=10.101.40.254/24 interface=40-MGMT

Set local DNS

/ip dns set servers=8.8.8.8

#######################################

IP Addressing & Routing

#######################################

/interface vlan add interface=ether1 name=101 vlan-id=101
/ip address add interface=101 address=172.16.101.1/30

/interface vlan add interface=ether1 name=102 vlan-id=102
/ip address add interface=102 address=172.16.102.1/30

/interface vlan add interface=ether1 name=103 vlan-id=103
/ip address add interface=103 address=172.16.103.1/30

/interface vlan add interface=ether1 name=104 vlan-id=104
/ip address add interface=104 address=172.16.104.1/30

/interface vlan add interface=ether1 name=105 vlan-id=105
/ip address add interface=105 address=172.16.105.1/30

/interface vlan add interface=ether1 name=106 vlan-id=106
/ip address add interface=106 address=172.16.106.1/30

/interface vlan add interface=ether1 name=990-VPLSNET vlan-id=990
/ip address add interface=990-VPLSNET address=216.444.194.97/28

/ip address add interface=ether1 address=261.555.164.154/30

/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=990-VPLSNET
add disabled=no dst-address=261.444.194.96/28 gateway=261.555.164.153
add disabled=no dst-address=10.102.0.0/16 gateway=172.16.102.2
add disabled=no dst-address=10.103.0.0/16 gateway=172.16.103.2
add disabled=no dst-address=10.104.0.0/16 gateway=172.16.104.2
add disabled=no dst-address=10.105.0.0/16 gateway=172.16.105.2
add disabled=no dst-address=10.106.0.0/16 gateway=172.16.106.2

#######################################

IP Services

#######################################

20-SERVICES VLAN interface creation, IP assignment, and DHCP service

/interface vlan add interface=primarybridge name=20-SERVICES vlan-id=20
/ip address add interface=20-SERVICES address=10.101.20.1/24
/ip pool add name=SERVICES_POOL ranges=10.101.20.2-10.101.20.254
/ip dhcp-server add address-pool=SERVICES_POOL interface=20-SERVICES name=SERVICES_DHCP disabled=no
/ip dhcp-server network add address=10.101.20.0/24 dns-server=10.101.20.1 gateway=10.101.20.1

30-DEVICES VLAN interface creation, IP assignment, and DHCP service

/interface vlan add interface=primarybridge name=30-DEVICES vlan-id=30
/ip address add interface=30-DEVICES address=10.101.30.1/24
/ip pool add name=DEVICES_POOL ranges=10.101.30.2-10.101.30.254
/ip dhcp-server add address-pool=DEVICES_POOL interface=30-DEVICES name=DEVICES_DHCP disabled=no
/ip dhcp-server network add address=10.101.30.0/24 dns-server=10.101.30.1 gateway=10.101.30.1

40-MGMT VLAN interface creation, IP assignment, and DHCP service

#/interface vlan add interface=primarybridge name=40-MGMT vlan-id=40
/ip address add interface=40-MGMT address=10.101.40.1/24
/ip pool add name=MGMT_POOL ranges=10.101.40.2-10.101.40.100
/ip dhcp-server add address-pool=MGMT_POOL interface=40-MGMT name=MGMT_DHCP disabled=no
/ip dhcp-server network add address=10.101.40.0/24 dns-server=10.101.40.1 gateway=10.101.40.1

#######################################

-- Limits and L3 HW Offloading --

#######################################

Enable L3 hardware offloading on switch1

/interface/ethernet/switch set 0 l3-hw-offloading=yes

Set limits and enable L3 hardware offloading on all ports

:foreach i in=[/interface/ethernet/switch/port/find where switch=switch1] do={ /interface/ethernet/switch/port set $i limit-broadcasts=yes limit-unknown-multicasts=yes limit-unknown-unicasts=yes storm-rate=1 l3-hw-offloading=yes }

Remove l3-hw-offloading on ether1 so we can use IP Firewall across VPLS/WAN

/interface/ethernet/switch/port set ether1 l3-hw-offloading=no

#######################################

-- Trunk Ports --

#######################################

ingress behavior

All SFP/QSFP ports, leave pvid to default of 1, enable L2 hardware offloading

VLAN Security - Only allow ingress packets WITH tags on Trunk Ports

/interface bridge port
add bridge=primarybridge interface=ether2 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether3 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether4 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether5 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether6 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether7 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether8 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether9 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether10 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=ether11 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=sfp-sfpplus1 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=sfp-sfpplus2 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=sfp-sfpplus3 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged
add bridge=primarybridge interface=sfp-sfpplus4 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged

egress behavior

/interface bridge vlan
add bridge=primarybridge tagged=primarybridge,ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 vlan-ids=20
add bridge=primarybridge tagged=primarybridge,ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 vlan-ids=30
add bridge=primarybridge tagged=primarybridge,ether2,ether3,ether4,ether5,ether6,ether7,ether8,ether9,ether10,ether11,sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 vlan-ids=40

#######################################

-- Access Ports --

#######################################

ether12 is MGMT port with pvid 40

/interface bridge port
add bridge=primarybridge interface=ether12 hw=yes ingress-filtering=yes pvid=40

egress behavior, handled automatically

#######################################

-- Firewall Rules --

#######################################

Block SYN flood

/ip settings set tcp-syncookies=yes

/interface list
add comment=defconf name=LAN
add comment=defconf name=WAN

/interface list member
add interface=primarybridge list=LAN
add interface=990-VPLSNET list=WAN

/ip firewall address-list
add address=10.101.40.0/24 comment="Management VLAN and Port" list=mgmt-ports

/ip firewall filter
add action=drop chain=input dst-address-list=mgmt-ports dst-port=21,22,23,80,443,8291 protocol=tcp src-address-list=!mgmt-ports
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related

/ip firewall nat
add chain=srcnat action=masquerade out-interface-list=WAN comment="Default Internet masquerade" disabled=no

/interface bridge set primarybridge protocol-mode=rstp vlan-filtering=yes dhcp-snooping=no add-dhcp-option82=no

wiseroute · June 18, 2023, 2:38pm

hello

ip firewall nat
add chain=srcnat action=masquerade out-interface-list=WAN comment=“Default Internet masquerade” disabled=no

hmm, how about trying to ping the internet via that setting on the router.

ping dst-ip src-addr=your-wan-ip.

Don’t forget to put that src-addr.

ip route list?

vnic0 · June 20, 2023, 3:21pm

Hi, this is the ping results and route table:

[admin@VPLS-TEST] > ping 8.8.8.8 src-addr=261.444.194.97
SEQ HOST SIZE TTL TIME STATUS
0 8.8.8.8 timeout
1 8.8.8.8 timeout
2 8.8.8.8 timeout
3 261.444.194.97 84 64 129ms187us host unreachable
4 8.8.8.8 timeout
sent=5 received=0 packet-loss=100%

[admin@VPLS-TEST] > ip route print
Flags: D - DYNAMIC; A - ACTIVE; c, s, y - BGP-MPLS-VPN; H - HW-OFFLOADED; + - ECMP
Columns: DST-ADDRESS, GATEWAY, DISTANCE

DST-ADDRESS GATEWAY DISTANCE

0 As 0.0.0.0/0 990 1
DAc 10.101.20.0/24 20 0
DAc 10.101.30.0/24 30 0
DAcH+ 10.101.40.0/24 40 0
DAcH+ 10.101.40.0/24 40 0
1 As 10.102.0.0/16 172.16.102.2 1
2 As 10.103.0.0/16 172.16.103.2 1
3 As 10.104.0.0/16 172.16.104.2 1
4 As 10.105.0.0/16 172.16.105.2 1
5 As 10.106.0.0/16 172.16.106.2 1
DAc 172.16.101.0/30 101 0
DAc 172.16.102.0/30 102 0
DAc 172.16.103.0/30 103 0
DAc 172.16.104.0/30 104 0
DAc 172.16.105.0/30 105 0
DAc 172.16.106.0/30 106 0
DAc 261.555.164.152/30 ether1 0
DAc 261.444.194.96/28 990-VPLSNET 0

wiseroute · June 21, 2023, 2:23am

hello

ok.

vpls is a professional services from your isp, they should be the first one who you call to assist you with the problem. while this forum members we don’t have any idea you and your isp setup.

The ISP says they have tagged all the vlan’s and the assigned subnet is routing from the ISP core to the CE IP, and all I need to do is configure a static route from my assigned network pointing to the ISP PE IP.

did your isp say that their gateway were vlan tagged or plain ether? this interface should be matched with your masquerade.

afaik, i don’t think that your isp gateway was on tagged vlan - they should be plain ether (untagged one) because you did your own vlan trunk and intervlan routing inside your vpls link.

As 0.0.0.0/0 990 1
…
DAc 261.444.194.96/28 990-VPLSNET 0

hmm… what kind of default routing table is that? is that 990 a vlan interface to your isp gateway? you didn’t point to your isp gateway ip?

and those 261.444 are they real? or just for the forum?

vnic0 · June 26, 2023, 5:47pm

Hi, yes the ISP said they are tagging all VLANs. So 100-110 are office locations and 990 is for Internet traffic. What I need to do is route 100-110 to the IP subnets I created (on both ends) and send all Internet to main HQ for routing Internet out VLAN 990. This is why 0.0.0.0/0 is showing 990. I am sending WAN traffic out 990-VPLSNET and masquerade that traffic as external IP. It is an L2 service which, usually I would simply trunk the port for all VLAN’s and input the ISP gateway…but because Internet needs to also traverse the same link, I think I may have the config wrong.

ISP does not know Mikrotik configuration so they are limited in helping to get the connection setup on my side. Hoping forum members might be able to assist.

yes, all IP’s are fake just for the forum.