Community discussions

 
User avatar
vecernik87
Long time Member
Long time Member
Topic Author
Posts: 642
Joined: Fri Nov 10, 2017 8:19 am

CHR does not transmit frames with VLAN tags from bridge

Thu Jan 31, 2019 2:34 am

I suspect this is not version related. I am experiencing similar behavior on 6.42.7 and 6.44beta61 (and anything i tried between) running in HyperV

I actually did some digging and hopefully I narrowed it down to a simple statement:
receiving and transmitting untagged data between bridge and hypervisor's virtual ethernet interface work
receiving and transmitting VLAN tagged data between VLAN interface and hypervisor's virtual ethernet interface work
receiving and transmitting VLAN tagged data between bridge and hypervisor's virtual ethernet interface DOES NOT work
(there is no VLAN filtering on bridge enabled, even though it seems like obvious cause)

Now let me prove it.
In HyperV, I have properly configured vlan trunking:
PS C:\WINDOWS\system32> Get-VMNetworkAdapterVlan

VMName          VMNetworkAdapterName Mode  VlanList
------          -------------------- ----  --------
dut             eth1             Trunk 0,1-4094
tester          eth1             Trunk 0,1-4094
(dut is the Device Under Test where I will prove the bug, tester is doing all pings (arp requests) and is configured practically same way, only difference is that tester does not have any bridge)
Similarly, I have enabled MAC spoofing on both adapters.

Then I have CHR2 (dut) with following config:
/interface bridge
   add fast-forward=no name=bridge1
/interface ethernet
   set [ find default-name=ether1 ] disable-running-check=no
   set [ find default-name=ether2 ] disable-running-check=no name=ether2-management
/interface vlan
   add interface=ether1 name=vlan1 vlan-id=98
/interface wireless security-profiles
   set [ find default=yes ] supplicant-identity=MikroTik
/interface bridge port
   add bridge=bridge1 hw=no interface=ether1 trusted=yes
/ip neighbor discovery-settings
   set discover-interface-list=none
/ip address
   add address=10.99.0.3/24 interface=ether1 network=10.99.0.0
   add address=10.99.1.3/24 interface=bridge1 network=10.99.1.0
   add address=10.99.98.3/24 interface=vlan1 network=10.99.98.0
/ip dhcp-client
   add add-default-route=no dhcp-options=hostname,clientid disabled=no interface=ether2-management use-peer-dns=no use-peer-ntp=no
/system identity
   set name=dut
Pretty simple, right? (I promise, I didn't skip anything, this is literary blank CHR just for this test) Lets completely ignore Ether2 (it is just for management purpose) and focus on ether1, bridge1 and vlan1

With this config, I can easily ping any address. All of them are responding fine (even though ether1 shouldn't have address as it is slave...) I am actually using ARP ping to make sure that no ARP shenanigans will affect my test:
[admin@tester] > ping 10.99.0.3 arp-ping=yes interface=ether1
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 00:15:5D:C9:30:0B                                 1ms 
    1 00:15:5D:C9:30:0B                                 0ms  
    2 00:15:5D:C9:30:0B                                 0ms  

[admin@tester] > ping 10.99.1.3 arp-ping=yes interface=ether1
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 00:15:5D:C9:30:0B                                 0ms  
    1 00:15:5D:C9:30:0B                                 0ms  
    2 00:15:5D:C9:30:0B                                 2ms  

[admin@tester] > ping 10.99.98.3 arp-ping=yes interface=vlan1 
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 00:15:5D:C9:30:0B                                 0ms  
    1 00:15:5D:C9:30:0B                                 1ms  
    2 00:15:5D:C9:30:0B                                 3ms  
With this, I confirmed that my HyperV settings is right and does not limit VLANs.
Now, lets change config to create the bug:
[admin@dut] > /interface vlan set vlan1 interface=bridge1 
Thats all folks. All I did is moving my vlan1 from ether1 to bridge1. Shouldn't make difference, right? Actually - the previous configuration was practically a misconfiguration. (This suggest that packet flow in CHR does not behave same way as in normal routers. According to the description, VLAN on slave interface should not work at all because all traffic should be forwarded straight to the bridge)

Anyway, lets look what results I get now:
[admin@tester] > ping 10.99.0.3 arp-ping=yes interface=ether1 
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 00:15:5D:C9:30:0B                                 4ms  
    1 00:15:5D:C9:30:0B                                 0ms  
    2 00:15:5D:C9:30:0B                                 4ms  

[admin@tester] > ping 10.99.1.3 arp-ping=yes interface=ether1  
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 00:15:5D:C9:30:0B                                 2ms 
    1 00:15:5D:C9:30:0B                                 7ms  
    2 00:15:5D:C9:30:0B                                 4ms  

[admin@tester] > ping 10.99.98.3 arp-ping=yes interface=vlan1
  SEQ HOST                                     SIZE TTL TIME  STATUS                             
    0 10.99.98.3                                              timeout                            
    1 10.99.98.3                                              timeout                            
    2 10.99.98.3                                              timeout                            
ohhh, where is my VLAN? Why is it not responding? There is no VLAN filtering on the bridge and it should work (at least it works on any physical router which I tested around)
Lets look at packet sniffers. We have 4 points of interest: vlan1 in dut, bridge1 in dut, ether1 in dut and finally ether1 in tester:
[admin@dut] > tool sniffer quick interface=vlan1
INTERFACE     TIME    NUM DI SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                        
vlan1        10.71      1 <- 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF        10.99.98.2: who has 10.99.98.3?    
vlan1        10.71      2 -> 00:15:5D:C9:30:0B 00:15:5D:C7:30:0D        10.99.98.3: at 00:15:5D:C9:30:0B   

[admin@dut] > tool sniffer quick interface=bridge1 
INTERFACE     TIME    NUM DI SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                        
bridge1      7.237      1 <- 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF 98     10.99.98.2: who has 10.99.98.3?    
bridge1      7.237      2 -> 00:15:5D:C9:30:0B 00:15:5D:C7:30:0D 98     10.99.98.3: at 00:15:5D:C9:30:0B   

[admin@dut] > tool sniffer quick interface=ether1
INTERFACE     TIME    NUM DI SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                        
ether1       5.187      4 <- 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF 98     10.99.98.2: who has 10.99.98.3?    
ether1       5.187      5 -> 00:15:5D:C9:30:0B 00:15:5D:C7:30:0D 98     10.99.98.3: at 00:15:5D:C9:30:0B   

[admin@tester] > tool sniffer quick interface=ether1
INTERFACE                     TIME    NUM DI SRC-MAC           DST-MAC           VLAN   SRC-ADDRESS                        
eth2-jcw                     0.991      1 -> 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF 98     10.99.98.2: who has 10.99.98.3?    
eth2-jcw                     2.051      2 -> 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF 98     10.99.98.2: who has 10.99.98.3?    
eth2-jcw                     3.081      3 -> 00:15:5D:C7:30:0D FF:FF:FF:FF:FF:FF 98     10.99.98.2: who has 10.99.98.3?    
With this, I just confirmed that dut can see both ARP-Request and ARP-Reply on ALL interfaces in, but those ARP-Replies do not really leave the CHR (it just pretends to be TXed from Ether1)

Now, why do I think it is fault of CHR? Elementary, my dear Watson - there is no difference in frame, whether it goes vlan1->ether1 or vlan1->bridge1->ether1. Due to that, anything else except dut (for example HyperV, tester, host OS...) can't possibly know, there was such change in topology.
Same thing happens when (instead of vlan1) communication flows for example through bridged EoIP from some remote device. As long as it is not tagged, it works. Once it gets tagged and goes through a bridge in CHR to virtual ethernet, it never leaves...
It also does not matter if the "tester" device is virtual or physical.

As always, when I find some reportable bug, I would like to ask you - clever members of this community - to look at it and point out any possible issue which I missed.
This time, situation is really simple and easily replicable - can anyone confirm/deny this bug?
 
anav
Forum Guru
Forum Guru
Posts: 2936
Joined: Sun Feb 18, 2018 11:28 pm
Location: Nova Scotia, Canada

Re: CHR does not transmit frames with VLAN tags from bridge

Tue Mar 19, 2019 6:38 pm

Not to worry its a virtual bug, not real! ;-)

Any configuration with routerOS and vlans that I have worked with has bridge vlan-filtering=yes???
I'd rather manage rats than software. Follow my advice at your own risk! (Sob & mkx forced me to write that!)
 
User avatar
Steveocee
Forum Guru
Forum Guru
Posts: 1092
Joined: Tue Jul 21, 2015 10:09 pm
Location: UK
Contact:

Re: CHR does not transmit frames with VLAN tags from bridge

Sat Apr 06, 2019 10:50 pm

What happens if you change the vlan ID? Try using anything other than 1.

Sounds mad but in my mind I had a problem similar and it was caused by this and we’re talking about MikroTik.
Steve "Steveocee" Carter
PC Gamer, Airsofter, MikroTik Nerd
My Website - My MikroTik Tutorials
 
User avatar
vecernik87
Long time Member
Long time Member
Topic Author
Posts: 642
Joined: Fri Nov 10, 2017 8:19 am

Re: CHR does not transmit frames with VLAN tags from bridge

Sun Apr 07, 2019 4:23 am

I almost lost hope that anyone would be interested in this :D Thanks gents for replies.

Any configuration with routerOS and vlans that I have worked with has bridge vlan-filtering=yes???
That applies if you want to do vlan filtering (i.e. you want to tag/untag stuff). In my case, I have vlan-filtering=no on purpose, because all ports should behave as trunks. This is similar to normal dumb-switch behavior, which just forwards all frames based on MAC, no matter what VLAN ID is set.
In addition, keep in mind that same configuration works on physical routerboards.

What happens if you change the vlan ID? Try using anything other than 1.
I never use 1. Its just default name of the interface. All my tests were conducted on several different ID's. To be specific 1016, 250, 100, 99 and finally 98 which i used purely for this test with fresh installation of CHR

Sounds mad but in my mind I had a problem similar
Could you please describe it bit more?
- What hypervisor?
- was your VLAN interface bound to bridge or directly on Ethernet interface?
It might have something similar but I am pretty sure that this is very particular and easily replicable bug.

All I need to know is, whether someone else can replicate it or if it works for everyone except me.
Simply said - if VLAN interfaces work when their master-interface is Ethernet but don't work when their master-interface is bridge, then you most likely replicated the bug (but please, share your config so we can compare and make sure)
 
User avatar
vecernik87
Long time Member
Long time Member
Topic Author
Posts: 642
Joined: Fri Nov 10, 2017 8:19 am

Re: CHR does not transmit frames with VLAN tags from bridge

Sun May 12, 2019 5:33 am

update: I just got chance to test this config on ESXi 5.5 and surprise-surprise, it works! (obviously, vlans and promiscuous mode must be enabled on virtual switch)

Who is online

Users browsing this forum: No registered users and 8 guests