CCR2116+ROS7.16b4 - Crazy arp on tagged interfaces

RouterOS General
1

APR rec.png
hello. I have a very weird problem with CCR2116 device.

  • CCR2116: Port 13 has 2 VLAN interfaces, each VLAN interface assigned to separate bridge, an IP address assigned to each bridge. Two bridges, two subnets.
  • External switch with 2 trunk ports (21,22), one port (21) connected to CCR2116 port 13, another port (22) connected to “network”.
    Switch has 2 VLANS, each VLAN has IP interface from same subnet as CCR2116 Bridges. Each VLAN has only two (21,22) ports in it, tagged. Switch VLAN = CCR2116 Bridge.
    VLAN IP forwarding disabled for both vlans.

So, ping from CCR2116 to switch VLAN1 ip interface pass ok. ping from CCR2116 to switch vlan2 pass ok. And ping from switch to CCR2116 to both bridges ip’s are ok. So, L2 working. L3 are fine.
But, somewhere in the network i have another devices, what are reachable via ping from switch, but not from CCR2116. And this is crazy part.
Basically i have VLAN from MT via switch via “network”, all devices has ip from one subnet, but ping pass ok only from switch.
Please look at screenshot. Why ARP records for interface “BR_PZZ5” for one subnet looks so different? They are result from same ping command.
All 5 ip does not respond to ping. With second VLAN | subnet situation exactly the same.

2

There are many pitfalls with this approach https://help.mikrotik.com/docs/spaces/ROS/pages/19136718/Layer2+misconfiguration, use a single VLAN-aware bridge.

3

on CCR2116 ether13 interface must be used only for management purposes, do not include it on any bridge

4

Adding static arp records to both CCR and host solve issue. ping works. But why network does not pass arp from/to CCR?

5

I checked CCR2116 diagram, port 13 are directly connected to CPU, so it differs from ports 1-12.
I am not a guru here, but i want to note, RB4011 sfp+ also has direct cpu link, so is it also management only port?

I change ports on CCR to port 8. This does not help at all. I tested vlan if on ethernet port and vlan if under bridge port, same result, ping not work.
After this I added static arp on CCR and on remote host and ping pass.
So, for unknown to me reason, network does not pass arp request/responce between CCR and host, but pass arp request between switch and host.

6

MT tag.png
CCR remove payload padding, when sending packet out from tagged port.

Ether 12 / VLAN_2005 - Incoming - 64/60
Ether 12 / VLAN_2003 - Outgoing - 60/56

So, when packet became less than 64 bytes it will be dropped by opposite side of ethernet link, IMHO. How to fix this?

7

Problem solved. All routers interfaces, connected to external network (SDH) was set as tagged.
So, anything entered SDH as tagged packet must go through and out as tagged packet.
With this set-up, small ip packets go through ok. ARP works. :very-very-happy-smile

8

Thanks for the update.