CCR2004-16G-2S and CRS354-48P-4S+2Q+, packet drop and weird behavior

RouterOS Beginner Basics
1

Hello,

I have a CRS354 switch with several vlan (1,10,20,30).

This switch is connected to CCR2004-16G-2S thought a trunk on the SFP+ port.

All the port are on a bridge with vlan filtering. The goal, is to perform intervlan routing with the CCR.

The CCR has only two SFP+, one to the WAN on vlan 100 and the other one is the trunk.

Again, both SFP+ port are on a bridge with vlan filtering.

I created vlan interface for each vlan.

I noticed some weird behavior, packet dropping, some ip are not pinguable and other yes.

I think, I have a misconfiguration, but I stuck to find where is my error. Maybe, I should not have CCR SFP+ interface in the same bridge ?

Thanks a lot, I am quite desesperate :frowning:

Here, in an extract of the configuration :

CCR

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] comment=WAN
set [ find default-name=sfp-sfpplus2 ] comment="Trunk fibre"
/interface vlan
add comment=Admin interface=bridge1 name=vlan1 vlan-id=1
add comment=Guest interface=bridge1 name=vlan10 vlan-id=10
add comment=Net interface=bridge1 name=vlan20 vlan-id=20
add comment=Dante interface=bridge1 name=vlan30 vlan-id=30
add comment=WAN interface=bridge1 name=vlan100 vlan-id=100
/interface list
add name=Trunk
add name=WAN
/ip pool
add comment=Admin name=pool-1 ranges=192.168.1.50-192.168.1.250
add comment=Guest name=pool-10 ranges=192.168.12.50-192.168.13.200
add comment=Net name=pool-20 ranges=192.168.16.50-192.168.16.200
add comment=Dante name=pool-30 ranges=192.168.20.50-192.168.20.200
/ip dhcp-server
add address-pool=pool-1 comment=Admin interface=vlan1 name=server-1
add address-pool=pool-20 comment=Net interface=vlan20 name=server-20
add address-pool=pool-10 comment=Guest interface=vlan10 name=server-10
add address-pool=pool-30 comment=Dante interface=vlan30 name=server-30
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge1 comment=vlan1 disabled=yes interface=ether1
add bridge=bridge1 comment=vlan20 disabled=yes interface=ether2 pvid=20
add bridge=bridge1 comment=vlan100 disabled=yes interface=ether3 pvid=100
add bridge=bridge1 comment=trunk interface=sfp-sfpplus2
add bridge=bridge1 interface=WAN pvid=100
/interface bridge vlan
add bridge=bridge1 comment=Guest tagged=Trunk,bridge1 vlan-ids=10
add bridge=bridge1 comment=Net tagged=Trunk,bridge1 vlan-ids=20
add bridge=bridge1 comment=Dante tagged=Trunk,bridge1 vlan-ids=30
add bridge=bridge1 comment=admin tagged=Trunk,bridge1 vlan-ids=1
add bridge=bridge1 comment=Internet tagged=bridge1 untagged=WAN vlan-ids=100
/interface list member
add interface=sfp-sfpplus2 list=Trunk
add interface=sfp-sfpplus1 list=WAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether15 network=192.168.88.0
add address=192.168.20.1/24 comment=Dante interface=vlan30 network=192.168.20.0
add address=192.168.16.1/22 comment=Net interface=vlan20 network=192.168.16.0
add address=192.168.12.1/22 comment=Guest interface=vlan10 network=192.168.12.0
add address=192.168.1.1/24 interface=vlan1 network=192.168.1.0
/ip dhcp-client
add interface=vlan100 use-peer-ntp=no
/ip dhcp-server config
set store-leases-disk=1h
/ip dhcp-server network
add address=192.168.1.0/24 comment=Admin dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.1.1 ntp-server=192.168.1.1
add address=192.168.12.0/22 comment=Guest dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.12.1 ntp-server=192.168.12.1
add address=192.168.16.0/22 comment=Net dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.16.1 ntp-server=192.168.16.1
add address=192.168.20.0/24 comment=Dante dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.20.1 ntp-server=192.168.20.1
/ip dns
set allow-remote-requests=yes servers=9.9.9.9,1.1.1.1,8.8.8.8
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked disabled=yes
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid disabled=yes
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid disabled=yes
add action=accept chain=input comment="defconf: accept ICMP" disabled=yes protocol=icmp
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related disabled=yes hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked disabled=yes
/ip firewall nat
add action=masquerade chain=srcnat comment="nat admin" out-interface=vlan100 src-address=192.168.1.0/24
add action=masquerade chain=srcnat comment="nat guest" out-interface=vlan100 src-address=192.168.12.0/22
add action=masquerade chain=srcnat comment="nat dante" out-interface=vlan100 src-address=192.168.20.0/22
add action=masquerade chain=srcnat comment="nat Net" out-interface=vlan100 src-address=192.168.16.0/24
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
add address=::224.0.0.0/100 comment="defconf: other" list=bad_ipv6
add address=::127.0.0.0/104 comment="defconf: other" list=bad_ipv6
add address=::/104 comment="defconf: other" list=bad_ipv6
add address=::255.0.0.0/104 comment="defconf: other" list=bad_ipv6
/ipv6 firewall filter
add action=fasttrack-connection chain=forward comment="ipv6 fastrack" connection-state=established
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=log chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 log=yes log-prefix=fezfef protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward
/system clock
set time-zone-name=Europe/Paris
/system ntp client
set enabled=yes
/system ntp server
set enabled=yes
/system ntp client servers
add address=145.238.80.80
add address=145.238.80.83
add address=pool.ntp.org
/system routerboard settings
set enter-setup-on=delete-key

CRS354

/interface bridge
add admin-mac=DC:2C:6E:B6:6D:E7 auto-mac=no comment=defconf name=bridge port-cost-mode=short vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus4 ] comment=trunk
/interface vlan
add comment=Admin interface=bridge name=vlan1 vlan-id=1
add comment=Guest interface=bridge name=vlan10 vlan-id=10
add interface=bridge name=vlan20 vlan-id=20
add comment=Dante interface=bridge name=vlan30 vlan-id=30
/interface ethernet switch qos port
set qsfpplus1-1 trust-l3=keep
set qsfpplus1-2 trust-l3=keep
set qsfpplus1-3 trust-l3=keep
set qsfpplus1-4 trust-l3=keep
set qsfpplus2-1 trust-l3=keep
set qsfpplus2-2 trust-l3=keep
set qsfpplus2-3 trust-l3=keep
set qsfpplus2-4 trust-l3=keep
set sfp-sfpplus1 trust-l3=keep
set sfp-sfpplus2 trust-l3=keep
set sfp-sfpplus3 trust-l3=keep
set sfp-sfpplus4 trust-l3=keep
set ether1 trust-l3=keep
set ether2 trust-l3=keep
set ether3 trust-l3=keep
set ether4 trust-l3=keep
set ether5 trust-l3=keep
set ether6 trust-l3=keep
set ether7 trust-l3=keep
set ether8 trust-l3=keep
set ether9 trust-l3=keep
set ether10 trust-l3=keep
set ether11 trust-l3=keep
set ether12 trust-l3=keep
set ether13 trust-l3=keep
set ether14 trust-l3=keep
set ether15 trust-l3=keep
set ether16 trust-l3=keep
set ether17 trust-l3=keep
set ether18 trust-l3=keep
set ether19 trust-l3=keep
set ether20 trust-l3=keep
set ether21 trust-l3=keep
set ether22 trust-l3=keep
set ether23 trust-l3=keep
set ether24 trust-l3=keep
set ether25 trust-l3=keep
set ether26 trust-l3=keep
set ether27 trust-l3=keep
set ether28 trust-l3=keep
set ether29 trust-l3=keep
set ether30 trust-l3=keep
set ether31 trust-l3=keep
set ether32 trust-l3=keep
set ether33 trust-l3=keep
set ether34 trust-l3=keep
set ether35 trust-l3=keep
set ether36 trust-l3=keep
set ether37 trust-l3=keep
set ether38 trust-l3=keep
set ether39 trust-l3=keep
set ether40 trust-l3=keep
set ether41 trust-l3=keep
set ether42 trust-l3=keep
set ether43 trust-l3=keep
set ether44 trust-l3=keep
set ether45 trust-l3=keep
set ether46 trust-l3=keep
set ether47 trust-l3=keep
set ether48 trust-l3=keep
/interface ethernet switch qos profile
add dscp=56 name=dante-ptp pcp=7 traffic-class=7
add dscp=46 name=dante-audio pcp=5 traffic-class=5
add dscp=8 name=dante-low pcp=1 traffic-class=0
/interface list
add name=WAN
add name=LAN
add name=Dante
add name=Trunk
add comment="Bornes WIFI" name=Bornes
add comment="reseau network1" name=network1
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether32 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether39 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4 internal-path-cost=10 path-cost=10
add bridge=bridge interface=ether15
add bridge=bridge interface=ether18
add bridge=bridge interface=Dante pvid=30
add bridge=bridge interface=network1 pvid=20
add bridge=bridge interface=Bornes
/interface ethernet switch l3hw-settings
set ipv6-hw=yes
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set lldp-mac-phy-config=yes lldp-med-net-policy-vlan=1
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge comment=Guest tagged=Trunk,bridge,Bornes,ether39 vlan-ids=10
add bridge=bridge comment=network1 tagged=Trunk,bridge,Bornes,ether39 untagged=network1 vlan-ids=20
add bridge=bridge comment=Dante tagged=Trunk,bridge untagged=Dante vlan-ids=30
add bridge=bridge tagged=Trunk,bridge untagged=LAN,WAN,Bornes,ether39,ether18,ether15 vlan-ids=1
/interface detect-internet
set detect-interface-list=WAN internet-interface-list=WAN lan-interface-list=LAN wan-interface-list=WAN
/interface ethernet switch
set 0 l3-hw-offloading=yes qos-hw-offloading=yes
/interface ethernet switch qos map ip
add dscp=56 profile=dante-ptp
add dscp=46 profile=dante-audio
add dscp=8 profile=dante-low
/interface ethernet switch qos tx-manager queue
set 1 weight=1
set 2 schedule=strict-priority
set 3 schedule=strict-priority
set 4 schedule=strict-priority
set 5 schedule=strict-priority
/interface list member
add interface=ether49 list=LAN
add interface=ether1 list=network1
add interface=ether2 list=network1
add interface=ether3 list=network1
add interface=ether4 list=network1
add interface=ether5 list=network1
add interface=ether6 list=network1
add interface=ether7 list=network1
add interface=ether8 list=network1
add interface=ether9 list=network1
add interface=ether10 list=network1
add interface=ether11 list=network1
add interface=ether12 list=network1
add interface=ether13 list=Bornes
add interface=ether14 list=network1
add interface=ether16 list=Bornes
add interface=ether17 list=Bornes
add interface=ether18 list=LAN
add interface=ether19 list=network1
add interface=ether20 list=network1
add interface=ether26 list=Bornes
add interface=ether27 list=network1
add interface=ether28 list=network1
add interface=ether29 list=Dante
add interface=ether30 list=Dante
add interface=ether31 list=Dante
add interface=ether33 list=network1
add interface=ether34 list=network1
add interface=ether35 list=network1
add interface=ether37 list=network1
add interface=ether38 list=network1
add interface=ether40 list=network1
add interface=ether41 list=network1
add interface=ether42 list=Bornes
add interface=ether43 list=network1
add interface=ether44 list=network1
add interface=ether45 list=network1
add interface=ether46 list=network1
add interface=ether47 list=network1
add interface=ether48 list=Bornes
add interface=sfp-sfpplus1 list=WAN
add interface=sfp-sfpplus4 list=Trunk
/ip address
add address=192.168.1.2/24 interface=vlan1 network=192.168.1.0
add address=192.168.20.2/24 comment=Dante interface=vlan30 network=192.168.20.0
add address=192.168.16.2/22 comment=network1 interface=vlan20 network=192.168.16.0
add address=192.168.12.2/22 comment=Guest interface=vlan10 network=192.168.12.0
add address=192.168.88.1/24 interface=ether49 network=192.168.88.0
/ip dhcp-server config
set store-leases-disk=never
/ip dns
set servers=9.9.9.9,1.1.1.1,8.8.8.8
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=192.168.1.1 routing-table=main suppress-hw-offload=no
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/routing bfd configuration
add disabled=no
/system clock
set time-zone-name=Europe/Paris

/system identity
set name=MikroTik-CRS
2

I changed the configuration on the CCR in order to not have both SFP+ in the same bridge.

Thus, I removed the wan interface from the bridge and changed nat rules.

I don’t have any drop on the CCR. Drop packets are on the CRS which only perform vlan filtering.

3

Maybe related, maybe not, but JFYI, the Rules of Mikrotik Club:

4

I understand, but we used the CRS has a basic switch without vlan filtering.

Then, we had to create vlan. However, we use unifi WIFI ap. I read they discourage using another vlan for administration. That’s why I kept vlan 1 for these devices.

5

I forgot to mention both devices are up-to-date with latest vesion 7.20.4

6

Which also may, or may not, be an aggravating factor (that would be Rule #10).
7.20.4 (actually 7.20.x) is - generally speaking - a troublesome version, there have been lots of reports of issues with it, even if it may be unrelated, if I were you I would downgrade to 7.19.4 or 7.19.6.

7

Thanks, I will consider this option, and thanks for your link. I disabled “detect internet”.

I also noticed something weird. In Winbox, I have many drop packets on several interfaces. But I can’t see this counter from the terminal.

Is it easy to downgrade version, I am afraid I will lose configuration?

8

You should anyway make an export before changing RoS version (and also a backup, better be safe than sorry) but downgrading should be as safe as upgrading (at least with near enough versions):

9

I understand I can downgrade to factory, then apply update before 7.20 ? I will try, thanks

10

No, you can downgrade directly to any version as long as it is the same or more recent than factory version.

You can choose between 7.19.4 or 7.19.6 (both relatively recent and both known to be usually working) upload the .npk(s) for the appropriate platform of that chosen version and downgrade.

1 Like
11

Thanks, I will try this approach. I will also repost cleaned configuration. I tried to identify any misconfiguration from Layer2 misconfiguration - RouterOS - MikroTik Documentation
But I did not notice any error.

My main issue, is I can ony ping 192.168.1.250 from vlan1.
All equipements from all networks are able to ping each other. Even the IP 192.168.1.104 in the same vlan.

The only difference is 1.250 and 1.104 are on the same edge switch (there is no configuration on this switch) on port ethernet15

I tried to simplify my configuration without firewall rules (except masquerade)

Here is the schema of the network

reseau
reseau1123Ă—794 64.2 KB

Here is the configuration on the router CCR :

/interface bridge
add name=bridge1 vlan-filtering=yes
/interface ethernet
set [ find default-name=ether1 ] disabled=yes
set [ find default-name=ether2 ] disabled=yes
set [ find default-name=ether3 ] disabled=yes
set [ find default-name=ether4 ] disabled=yes
set [ find default-name=ether5 ] disabled=yes
set [ find default-name=ether6 ] disabled=yes
set [ find default-name=ether7 ] disabled=yes
set [ find default-name=ether8 ] disabled=yes
set [ find default-name=ether9 ] disabled=yes
set [ find default-name=ether10 ] disabled=yes
set [ find default-name=ether11 ] disabled=yes
set [ find default-name=ether12 ] disabled=yes
set [ find default-name=ether13 ] disabled=yes
set [ find default-name=ether14 ] disabled=yes
set [ find default-name=ether15 ] disabled=yes
set [ find default-name=ether16 ] disabled=yes
set [ find default-name=sfp-sfpplus1 ] comment=WAN
set [ find default-name=sfp-sfpplus2 ] comment="Trunk fibre"
/interface vlan
add comment=Admin interface=bridge1 name=vlan1 vlan-id=1
add comment=Guest interface=bridge1 name=vlan10 vlan-id=10
add comment=Mineral interface=bridge1 name=vlan20 vlan-id=20
add comment=Dante interface=bridge1 name=vlan30 vlan-id=30
/interface list
add name=Trunk
add name=WAN
/ip pool
add comment=Admin name=pool-1 ranges=192.168.1.50-192.168.1.250
add comment=Guest name=pool-10 ranges=192.168.12.50-192.168.13.200
add comment=Mineral name=pool-20 ranges=192.168.16.50-192.168.16.200
add comment=Dante name=pool-30 ranges=192.168.20.50-192.168.20.200
/ip dhcp-server
add address-pool=pool-1 comment=Admin interface=vlan1 name=server-1
add address-pool=pool-20 comment=Mineral interface=vlan20 name=server-20
add address-pool=pool-10 comment=Guest interface=vlan10 name=server-10
add address-pool=pool-30 comment=Dante interface=vlan30 name=server-30
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge1 comment=vlan1 disabled=yes interface=ether1
add bridge=bridge1 comment=vlan20 disabled=yes interface=ether2 pvid=20
add bridge=bridge1 comment=trunk interface=sfp-sfpplus2
/interface bridge vlan
add bridge=bridge1 comment=Guest tagged=Trunk,bridge1 vlan-ids=10
add bridge=bridge1 comment=Mineral tagged=Trunk,bridge1 vlan-ids=20
add bridge=bridge1 comment=Dante tagged=Trunk,bridge1 vlan-ids=30
add bridge=bridge1 comment=admin tagged=Trunk,bridge1 vlan-ids=1
/interface list member
add interface=sfp-sfpplus2 list=Trunk
add interface=sfp-sfpplus1 list=WAN
/ip address
add address=192.168.88.1/24 comment=defconf interface=ether15 network=192.168.88.0
add address=192.168.20.1/24 comment=Dante interface=vlan30 network=192.168.20.0
add address=192.168.16.1/22 comment=Mineral interface=vlan20 network=192.168.16.0
add address=192.168.12.1/22 comment=Guest interface=vlan10 network=192.168.12.0
add address=192.168.1.1/24 interface=vlan1 network=192.168.1.0
add address=192.168.100.1/24 comment=wireguard1 interface=wg1 network=192.168.100.0
/ip dhcp-client
add interface=sfp-sfpplus1 use-peer-ntp=no
/ip dhcp-server config
set store-leases-disk=1h
/ip dhcp-server network
add address=192.168.1.0/24 comment=Admin dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.1.1 ntp-server=192.168.1.1
add address=192.168.12.0/22 comment=Guest dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.12.1 ntp-server=192.168.12.1
add address=192.168.16.0/22 comment=Mineral dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.16.1 ntp-server=192.168.16.1
add address=192.168.20.0/24 comment=Dante dns-server=1.1.1.1,9.9.9.9,8.8.8.8 gateway=192.168.20.1 ntp-server=192.168.20.1
/ip dns
set allow-remote-requests=yes servers=9.9.9.9,1.1.1.1,8.8.8.8
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related hw-offload=yes
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
/ip firewall nat
add action=masquerade chain=srcnat comment="nat admin" out-interface=sfp-sfpplus1 src-address=192.168.1.0/24
add action=masquerade chain=srcnat comment="nat guest" out-interface=sfp-sfpplus1 src-address=192.168.12.0/22
add action=masquerade chain=srcnat comment="nat dante" out-interface=sfp-sfpplus1 src-address=192.168.20.0/22
add action=masquerade chain=srcnat comment="nat mineral" out-interface=sfp-sfpplus1 src-address=192.168.16.0/24
/ipv6 firewall address-list
add address=::/128 comment="defconf: unspecified address" list=bad_ipv6
add address=::1/128 comment="defconf: lo" list=bad_ipv6
add address=fec0::/10 comment="defconf: site-local" list=bad_ipv6
add address=::ffff:0.0.0.0/96 comment="defconf: ipv4-mapped" list=bad_ipv6
add address=::/96 comment="defconf: ipv4 compat" list=bad_ipv6
add address=100::/64 comment="defconf: discard only " list=bad_ipv6
add address=2001:db8::/32 comment="defconf: documentation" list=bad_ipv6
add address=2001:10::/28 comment="defconf: ORCHID" list=bad_ipv6
add address=3ffe::/16 comment="defconf: 6bone" list=bad_ipv6
add address=::224.0.0.0/100 comment="defconf: other" list=bad_ipv6
add address=::127.0.0.0/104 comment="defconf: other" list=bad_ipv6
add address=::/104 comment="defconf: other" list=bad_ipv6
add address=::255.0.0.0/104 comment="defconf: other" list=bad_ipv6
/ipv6 firewall filter
add action=fasttrack-connection chain=forward comment="ipv6 fastrack" connection-state=established
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf: drop packets with bad src ipv6" src-address-list=bad_ipv6
add action=drop chain=forward comment="defconf: drop packets with bad dst ipv6" dst-address-list=bad_ipv6
add action=log chain=forward comment="defconf: rfc4890 drop hop-limit=1" hop-limit=equal:1 log=yes log-prefix=fezfef protocol=icmpv6
add action=accept chain=forward comment="defconf: accept ICMPv6" protocol=icmpv6
add action=accept chain=forward
/system clock
set time-zone-name=Europe/Paris
/system identity
set name=MikroTik-CCR
/system ntp client
set enabled=yes
/system ntp server
set enabled=yes
/system ntp client servers
add address=145.238.80.80
add address=145.238.80.83
add address=pool.ntp.org
/system routerboard settings
set enter-setup-on=delete-key
/tool sniffer
set file-name=test filter-interface=vlan1 filter-ip-protocol=icmp filter-src-ip-address=192.168.16.200/32

And here is the switch

/interface bridge
add admin-mac=DC:2C:6E:B6:6D:E7 auto-mac=no comment=defconf name=bridge port-cost-mode=short vlan-filtering=yes
/interface ethernet
set [ find default-name=ether13 ] 
set [ find default-name=ether14 ] auto-negotiation=no speed=100M-baseT-full
set [ find default-name=ether16 ] 
set [ find default-name=ether17 ] 
set [ find default-name=ether18 ] 
set [ find default-name=ether19 ] 
set [ find default-name=ether21 ] disabled=yes
set [ find default-name=ether22 ] disabled=yes
set [ find default-name=ether23 ] disabled=yes
set [ find default-name=ether24 ] disabled=yes
set [ find default-name=ether25 ] disabled=yes
set [ find default-name=ether26 ] 
set [ find default-name=ether29 ] 
set [ find default-name=ether30 ] 
set [ find default-name=ether31 ] 
set [ find default-name=ether32 ] 
set [ find default-name=ether34 ] 
set [ find default-name=ether35 ] 
set [ find default-name=ether36 ] disabled=yes
set [ find default-name=ether37 ]
set [ find default-name=ether38 ] 
set [ find default-name=ether39 ] 
set [ find default-name=ether42 ] 
set [ find default-name=ether48 ] 
set [ find default-name=ether49 ] mac-address=DC:2C:6E:B6:6D:17
set [ find default-name=sfp-sfpplus1 ] disabled=yes
set [ find default-name=sfp-sfpplus2 ] disabled=yes
set [ find default-name=sfp-sfpplus3 ] disabled=yes
/interface vlan
add comment=Admin interface=bridge name=vlan1 vlan-id=1
add comment=Guest interface=bridge name=vlan10 vlan-id=10
add comment=Mineral interface=bridge name=vlan20 vlan-id=20
add comment=Dante interface=bridge name=vlan30 vlan-id=30
/interface ethernet switch qos port
set qsfpplus1-1 trust-l3=keep
set qsfpplus1-2 trust-l3=keep
set qsfpplus1-3 trust-l3=keep
set qsfpplus1-4 trust-l3=keep
set qsfpplus2-1 trust-l3=keep
set qsfpplus2-2 trust-l3=keep
set qsfpplus2-3 trust-l3=keep
set qsfpplus2-4 trust-l3=keep
set sfp-sfpplus1 trust-l3=keep
set sfp-sfpplus2 trust-l3=keep
set sfp-sfpplus3 trust-l3=keep
set sfp-sfpplus4 trust-l3=keep
set ether1 trust-l3=keep
set ether2 trust-l3=keep
set ether3 trust-l3=keep
set ether4 trust-l3=keep
set ether5 trust-l3=keep
set ether6 trust-l3=keep
set ether7 trust-l3=keep
set ether8 trust-l3=keep
set ether9 trust-l3=keep
set ether10 trust-l3=keep
set ether11 trust-l3=keep
set ether12 trust-l3=keep
set ether13 trust-l3=keep
set ether14 trust-l3=keep
set ether15 trust-l3=keep
set ether16 trust-l3=keep
set ether17 trust-l3=keep
set ether18 trust-l3=keep
set ether19 trust-l3=keep
set ether20 trust-l3=keep
set ether21 trust-l3=keep
set ether22 trust-l3=keep
set ether23 trust-l3=keep
set ether24 trust-l3=keep
set ether25 trust-l3=keep
set ether26 trust-l3=keep
set ether27 trust-l3=keep
set ether28 trust-l3=keep
set ether29 trust-l3=keep
set ether30 trust-l3=keep
set ether31 trust-l3=keep
set ether32 trust-l3=keep
set ether33 trust-l3=keep
set ether34 trust-l3=keep
set ether35 trust-l3=keep
set ether36 trust-l3=keep
set ether37 trust-l3=keep
set ether38 trust-l3=keep
set ether39 trust-l3=keep
set ether40 trust-l3=keep
set ether41 trust-l3=keep
set ether42 trust-l3=keep
set ether43 trust-l3=keep
set ether44 trust-l3=keep
set ether45 trust-l3=keep
set ether46 trust-l3=keep
set ether47 trust-l3=keep
set ether48 trust-l3=keep
/interface ethernet switch qos profile
add disabled=yes dscp=56 name=dante-ptp pcp=7 traffic-class=7
add disabled=yes dscp=46 name=dante-audio pcp=5 traffic-class=5
add disabled=yes dscp=8 name=dante-low pcp=1 traffic-class=0
/interface list
add name=WAN
add name=LAN
add name=Dante
add name=Trunk
add name=Bornes
add  name=Mineral
/interface lte apn
set [ find default=yes ] ip-type=ipv4 use-network-apn=no
/ip smb users
set [ find default=yes ] disabled=yes
/port
set 0 name=serial0
/interface bridge port
add bridge=bridge comment=defconf ingress-filtering=no interface=ether7 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether32 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=ether39 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus1 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus2 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus3 internal-path-cost=10 path-cost=10
add bridge=bridge comment=defconf ingress-filtering=no interface=sfp-sfpplus4 internal-path-cost=10 path-cost=10
add bridge=bridge interface=ether15
add bridge=bridge interface=ether18
add bridge=bridge interface=Dante pvid=30
add bridge=bridge interface=Mineral pvid=20
add bridge=bridge interface=Bornes
/interface ethernet switch l3hw-settings
set ipv6-hw=yes
/ip firewall connection tracking
set udp-timeout=10s
/ip neighbor discovery-settings
set lldp-mac-phy-config=yes lldp-med-net-policy-vlan=1
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes max-neighbor-entries=8192
/interface bridge vlan
add bridge=bridge comment=Guest tagged=Trunk,bridge,Bornes,ether39 vlan-ids=10
add bridge=bridge comment=Mineral tagged=Trunk,bridge,Bornes,ether39 untagged=Mineral vlan-ids=20
add bridge=bridge comment=Dante tagged=Trunk,bridge untagged=Dante vlan-ids=30
add bridge=bridge tagged=Trunk,bridge untagged=LAN,WAN,Bornes,ether39,ether18,ether15 vlan-ids=1
/interface ethernet switch
set 0 l3-hw-offloading=yes qos-hw-offloading=yes
/interface ethernet switch qos map ip
add dscp=56 profile=dante-ptp
add dscp=46 profile=dante-audio
add dscp=8 profile=dante-low
/interface ethernet switch qos tx-manager queue
set 1 weight=1
set 2 schedule=strict-priority
set 3 schedule=strict-priority
set 4 schedule=strict-priority
set 5 schedule=strict-priority
/interface list member
add interface=ether49 list=LAN
add interface=ether1 list=Mineral
add interface=ether2 list=Mineral
add interface=ether3 list=Mineral
add interface=ether4 list=Mineral
add interface=ether5 list=Mineral
add interface=ether6 list=Mineral
add interface=ether7 list=Mineral
add interface=ether8 list=Mineral
add interface=ether9 list=Mineral
add interface=ether10 list=Mineral
add interface=ether11 list=Mineral
add interface=ether12 list=Mineral
add interface=ether13 list=Bornes
add interface=ether14 list=Mineral
add interface=ether16 list=Bornes
add interface=ether17 list=Bornes
add interface=ether18 list=LAN
add interface=ether19 list=Mineral
add interface=ether20 list=Mineral
add interface=ether26 list=Bornes
add interface=ether27 list=Mineral
add interface=ether28 list=Mineral
add interface=ether29 list=Dante
add interface=ether30 list=Dante
add interface=ether31 list=Dante
add interface=ether33 list=Mineral
add interface=ether34 list=Mineral
add interface=ether35 list=Mineral
add interface=ether37 list=Mineral
add interface=ether38 list=Mineral
add interface=ether40 list=Mineral
add interface=ether41 list=Mineral
add interface=ether42 list=Bornes
add interface=ether43 list=Mineral
add interface=ether44 list=Mineral
add interface=ether45 list=Mineral
add interface=ether46 list=Mineral
add interface=ether47 list=Mineral
add interface=ether48 list=Bornes
add interface=sfp-sfpplus1 list=WAN
add interface=sfp-sfpplus4 list=Trunk
/ip address
add address=192.168.1.2/24 interface=vlan1 network=192.168.1.0
add address=192.168.20.2/24 comment=Dante interface=vlan30 network=192.168.20.0
add address=192.168.16.2/22 comment=Mineral interface=vlan20 network=192.168.16.0
add address=192.168.12.2/22 comment=Guest interface=vlan10 network=192.168.12.0
add address=192.168.88.1/24 interface=ether49 network=192.168.88.0
/ip dhcp-server config
set store-leases-disk=never
/ip dns
set servers=9.9.9.9,1.1.1.1,8.8.8.8
/ip ipsec profile
set [ find default=yes ] dpd-interval=2m dpd-maximum-failures=5
/ip route
add disabled=no dst-address=0.0.0.0/0 gateway=192.168.1.1 routing-table=main suppress-hw-offload=no
/ip smb shares
set [ find default=yes ] directory=/flash/pub
/routing bfd configuration
add disabled=no
/system clock
set time-zone-name=Europe/Paris
/system health settings
set fan-full-speed-temp=58C fan-min-speed-percent=30% fan-target-temp=55C
/system identity
set name=MikroTik-CRS
/system ntp client
set enabled=yes
/system ntp client servers
add address=145.238.80.80
add address=145.238.80.83
add address=pool.ntp.org
/tool sniffer
set file-name=test filter-interface=bridge
12

I have some news. I disabled hardware offload on ether15.
I played with packet sniffer on ether15 and I see ICMP ping and response for 192.168.1.104. But only ping request for 192.168.1.250.
That's eliminate issue on CCR.
However, I was able to connect to the small switch behind ether15, it is an ubiquiti edge switch 8 with a minimal interface.
It allowed me to inspect the interface.
The surprise is I can see ping and request for 192.168.1.250 ! Unfortunately, the interface is very limited. It is hard to see the flow direction.
A device is lying, but I can't tell which one.

13

I spend time to investigate. I noticed, the CRS keep the vlan tag=1 on ether15 for outgoing packet.
It should remove the vlan tag has it is untagged interface ?

14

You can try that, really cannot say if it will work.

Part of the issue might be in the unmanaged (and VLAN unaware) switch you have connected, it seems like some of these "dumb" switches remove any VLAN tag while some others pass the packets exactly as they receive them.

15

No, I don't think the issue is on the small switch. I think it is not normal the Mikrotik leave vlan id on untagged port.

16

But isn't ether15 inside the bridge?

17

Indeed, I am not alone :

18

Yes, but untagged for vlan 1. The switch should remove vlan header.

19

I may found why the device did not respond to ping :

https://www.getdante.com/support/faq/dante-device-enters-failsafe-mode-due-to-vlan-configuration/

Some Dante devices can enter Failsafe mode if they receive traffic from more than one subnet.

I was furious when I read this :face_with_symbols_on_mouth:
I added a source nat to hide sender behind the gateway and it worked :partying_face:

That’s explained why the device respond ping from the same subnet, but not the other one. It is completely silly…
I will confirm tomorrow it is working as expected.

20

TL;DR : I thought there was an L2 misconfiguration, but the issue was not on Mikrotik devices.

It was the end devices (Yamaha Matrix with Dante network) which had a weird behavior. It does not respond to ping if it is coming from another network, even if routing and bridging is OK.

Many thanks @jaclaz, I learned a lot !