Page 1 of 1

Basic CHR config - problems with ESXi, VLANs, CAPsMAN (not yet installed)

Posted: Fri Oct 04, 2019 10:23 pm
by Lorsk
Hello,
I have a problem with the correct configuration of a Cloud Hosted Router (ROS 6.45.6).
Platform: ESXI 6.7 current patch level
OVA package
1 Ethernet Interface (connected to ESXi virtual switch vlan ID 4095, promiscuous mode enabled, MAC spoofing enabled)

I will write down the CHR's config at the end of this post.

The problem is, that bridge doesn't work when vlan-filtering is enabled. I can test this with interface "vlan-test". It will get a ip-address when bridge vlan-filtering is disabled and will perfectly change ip-addresses according to configured vlan-tag.
As soon as turning on vlan-filtering at bridge1 it will stop working and no ip-packets will be transported.

I want to use the CHR as CAPsMAN so I need a bridge with properly vlan-filtering.

How should I set up the CHR? What will be a basic setup with 1 ethernet interface (tagged vlans) and multiple virtual interfaces in router os (for different wlan-interfaces with different vlans managed by CAPsMAN)

(BTW: This configuration works like a charme on a CRS-328...)

Thanks a lot


My test CHR-Config:
/interface ethernet
set [ find default-name=ether1 ] advertise=1000M-full,10000M-full arp=enabled arp-timeout=auto auto-negotiation=yes \
cable-settings=default disable-running-check=no disabled=no full-duplex=yes loop-protect=default \
loop-protect-disable-time=5m loop-protect-send-interval=5s mac-address=00:0C:29:42:AF:31 mtu=1500 name=ether1 \
orig-mac-address=00:0C:29:42:AF:31 speed=10Gbps
/interface bridge
add ageing-time=5m arp=enabled arp-timeout=auto auto-mac=yes dhcp-snooping=no disabled=no ether-type=0x8100 fast-forward=yes \
forward-delay=15s frame-types=admit-all igmp-snooping=no ingress-filtering=no max-message-age=20s mtu=auto name=bridge1 \
priority=0x8000 protocol-mode=rstp pvid=100 transmit-hold-count=6 vlan-filtering=yes
/interface vlan
add arp=enabled arp-timeout=auto disabled=no interface=ether1 loop-protect=default loop-protect-disable-time=5m \
loop-protect-send-interval=5s mtu=1500 name=Management_VLAN_IFace use-service-tag=no vlan-id=100
add arp=enabled arp-timeout=auto disabled=no interface=bridge1 loop-protect=default loop-protect-disable-time=5m \
loop-protect-send-interval=5s mtu=1500 name=vlan1-test use-service-tag=no vlan-id=80

/interface bridge port
add auto-isolate=no bpdu-guard=no bridge=bridge1 broadcast-flood=yes disabled=no edge=auto fast-leave=no frame-types=\
admit-all horizon=none hw=yes ingress-filtering=no interface=ether1 internal-path-cost=10 learn=auto multicast-router=\
temporary-query path-cost=10 point-to-point=auto priority=0x80 pvid=100 restricted-role=no restricted-tcn=no \
tag-stacking=no trusted=no unknown-multicast-flood=yes unknown-unicast-flood=yes
/interface bridge settings
set allow-fast-path=no use-ip-firewall=no use-ip-firewall-for-pppoe=no use-ip-firewall-for-vlan=no
/interface bridge vlan
add bridge=bridge1 comment=Management disabled=no tagged="" untagged="" vlan-ids=100
add bridge=bridge1 comment=intern disabled=no tagged="" untagged="" vlan-ids=30
add bridge=bridge1 comment=Gast disabled=no tagged="" untagged="" vlan-ids=60
add bridge=bridge1 comment=Medien_TV disabled=no tagged="" untagged="" vlan-ids=80
/ip address
add address=10.28.100.14/24 disabled=no interface=Management_VLAN_IFace network=10.28.100.0
/ip dhcp-client
add add-default-route=yes default-route-distance=1 dhcp-options=hostname,clientid disabled=no interface=vlan1-test use-peer-dns=\
yes use-peer-ntp=yes
/ip route
add !bgp-as-path !bgp-atomic-aggregate !bgp-communities !bgp-local-pref !bgp-med !bgp-origin !bgp-prepend !check-gateway \
disabled=no distance=1 dst-address=0.0.0.0/0 gateway=10.28.100.1 !route-tag !routing-mark scope=30 target-scope=10

Re: Basic CHR config - problems with ESXi, VLANs, CAPsMAN (not yet installed)

Posted: Tue Oct 08, 2019 4:39 pm
by IPANetEngineer
What does the vswitch config look like in ESXI?

Re: Basic CHR config - problems with ESXi, VLANs, CAPsMAN (not yet installed)

Posted: Tue Oct 08, 2019 7:03 pm
by Steveocee
Looking at your config, you haven't stated which interfaces VLAN traffic will be tagged or not tagged on. Turning filtering on would pretty much remove these interfaces from use as all VLANs are neither tagged or untagged on any interfaces.
Yours:
/interface bridge vlan
add bridge=bridge1 comment=Management disabled=no tagged="" untagged="" vlan-ids=100
add bridge=bridge1 comment=intern disabled=no tagged="" untagged="" vlan-ids=30
add bridge=bridge1 comment=Gast disabled=no tagged="" untagged="" vlan-ids=60
add bridge=bridge1 comment=Medien_TV disabled=no tagged="" untagged="" vlan-ids=80

Should be for example:
/interface bridge vlan
add bridge=bridge1 comment=Management disabled=no tagged="etherx" untagged="ethery" vlan-ids=100
add bridge=bridge1 comment=intern disabled=no tagged="etherx" untagged="ethery" vlan-ids=30
add bridge=bridge1 comment=Gast disabled=no tagged="etherx" untagged="ethery" vlan-ids=60
add bridge=bridge1 comment=Medien_TV disabled=no tagged="etherx" untagged="ethery" vlan-ids=80


You can test the ESXi exposure to VLAN by leaving filtering off, adding an IP to one of your VLAN interfaces and pinging. If it works then ESXi is set fine and the problem is RouterOS config (likely as I've pointed out above).

Re: Basic CHR config - problems with ESXi, VLANs, CAPsMAN (not yet installed)

Posted: Wed Oct 09, 2019 9:52 pm
by Lorsk
I have only 1 "physical" ethernet adapter called ether1.

In ESXi this is a tagged interface.

I added ether1 to bridge1 -> in an oldschool-config now it should be the "master"-interface and I don't need to configure it because all traffic is forwarded to the bridge. Perhaps this is wrong - I don't know.

bridge 1 is configured in the vlan section as a bridge -> it is neither tagged nor untagged because it is the bridge.

I also tried to configure ether1 as tagges for all vlans - this doesn't change the behavior.

When everything works, I will add CAPs MAN and therefor I will have a lot of virtual interfaces.

In my oppinion I don't have to configure the vlan-tags because CAPsMAN as well as Router-OS will do it automatically (see VLAN-Config in CRS-328 wiki - but I can't compare because I have only 1 port).

ESXi looks like this (I think this is okay, because the second machine (OPNSense) works like a charm with a lot of virtual interfaces filtering vlan-traffic): https://ibb.co/1QNdGBq

Perhaps somebody could post a small standard-config for CHR RouterOS? I need:

ether1 connected to ESXi (VMXNET3 driver) -> only tagged packets
bridge1 which will connect ether1 as well as all other virtual adapters coming from CAPsMAN after configuration
virtual VLAN Adapter (pvid 100) which will be management-interface in RouterOS

VLAN filtering turned on, so that all adapters only get the right traffic.

Thanks a lot

Re: Basic CHR config - problems with ESXi, VLANs, CAPsMAN (not yet installed)

Posted: Sat Oct 19, 2019 7:10 pm
by Lorsk
Hello,

does anybody have an idea or a standard-config for me?

Thanks a lot