I was already thinking it was going into this direction …
There is some internal joke between more seasoned users about some basic rules when using MT gear.
It goes like this:
You do not use VLAN1
You DO NOT use VLAN1
You do not use Quickset
You do not use detect internet
5)…
Background reason:
Mikrotik handles VLAN1 a bit different then how most other suppliers do and when not set up properly, it may break things.
And this might be what you are seeing.
So either make sure VLAN1 is handled as a proper VLAN, or do not use it at all ! (I don’t at home).
This also involves getting rid of all bridges except the one for Docker (and even there, some question if it is needed. You can perfectly do without. But since it’s in MT’s documentation, it’s tolerated …).
Base line: If you need multiple bridges, your VLAN setup is not optimal.
Some switches try to be smart (too smart as it turns out) … and try to detect anomalous traffic. I have a Dlink manged switch and it was messing with NTP traffic (UDP to/from port 123) inside LAN. After disabling that “feature” I have zero problems (since more than a year ago). So check your smart switches to see if they are trying to be too smart.
As to the dumb switch: mostly it’s fine to use dumb switch even in environment with VLANs. Such a switch obviously won’t do any ingress/egress filtering (so it might upset some windows host, where NIC drivers silently strip VLAN headers if they are not configured to be VLAN aware, and IP/IPv6 stack then starts to receive traffic which should not be there, such as IPv6 Router Advertisiements making such NIC to assume IPv6 address from wrong VLAN). But it has to support frames with size of at least 1508 bytes (baby-jumbo frames, standard ethernet is 1500 bytes). If they don’t, then they might drop (or corrupt) frames with full size payload. Shorter frames should survive just fine though. If it’s actually true that your CAPsMAN to CAP communication flows over untagged (default) “VLAN”, then this should not be an issue at all.
Another thing to think about: packet drop. While it’s not really desirable to have packet dropped in a LAN, all TCP-based communication is somehow immune to it (due to TCP’s retransmission mechanisms). UDP-based communication is not immune. And my feeling is that CAPsMAN protocol is particularly fragile (even short communication loss means CAP device to deprovision). Cause for packet loss in LAN environment might be marginal cables[*] and some receivers are slightly less fragile than others (which would explain why some switches don’t cause problems while others do). 80 meters of UTP is quite a long stretch …
[*] my own experience: my UTP cable installer (a knowledgeable electrical technician, but obviously didn’t have much experience with UTP cables) did termination in wall sockets (I did patching in patch panel myself). Static tests were fine (all pins properly connected), but I had a problem with one computer connecting to one of longest cables … and it was less than 40m cable stretch. Then I borrowed a professional UTP cable tester (some Fluke monster of tester) and it came out that none of cables was up to 1Gbps requirements … everything was wrong. When then I inspected the wall sockets, I found out that the guy untwisted all wires in length of around 10 cm (I guess it was easier for him to connect wires to socket). After I twisted those wires again, all Fluke tests came out perfect and I never had any problems with connected devices ever again.
VLAN 3999 for IPTV … are you, by any chance, “victim” of Telekom Slovenije?
I wouldn’t realy call myself a victim Yes it is from Telekom, but my main internet is on Telemach. I was testing if i can get Multicast over EOIP and use their TV box on my internet connection and it worked without problem. I don’t use this in practice but i just leave it configured and added UDPXY as a docker container on 5009. So now if i open M3U playlist, 5009 pulls Multicast over EOIP, UDPXY changes it to unicast and then it doesn’t causes problems in network anymore (Multicast traffic on all ports and similar things). I made all LAN cables myself, previusly i had 120 meters of CAT6 lan cable and it worked between Mikrotiks but not between Mikrotik and some dumb POE switch. Now i shortened it quite a bit.
The problem is, that there realy isn’t any disconnection, just Caps connection becomes suddenly “interrupted” and after few seconds starts working again.
Now with TpLink switch inbetween for now, no problems at all. This realy is strange. I also made a factoy reset on Zyxell switch. I will check if that helped in anyway later on.
Well i don’t use detect internet and quickset, so i can check two boxes. I’m simply not experienced enough for advanced VLAN configurations and since more people and also my friend over wireless bridge uses this internet connection i cannot realy simply use trial and error system to separate everything into VLANs better because it would take days for me, to make this work.
Separate bridge for traffic that comes over EOIP would probably still be needed.
I will report my findings later on, but for now it seems that the only problem is in switches, that are too smart. I already thinking of buying CRS326 instead of Zyxell but there isnt any real guarantee, that this would help and that it would work without problem.
When frame loss happens, links don’t disconnect … in worst case they may renegotiate to lower speed (e.g. 100Mbps instead of 1Gbps) but even for this frame loss has to be extensive.
But as I wrote, 100m is the “standard” maximum length of ethernet cable (check random internet article on ethernet cat6 cables, e.g. wikipedia article). And, again, cable has to be perfectly made for flawless operation at the extreme length (and both link partners have to be good at it, I’d expect some low-end switches to be less than perfect).
Yesterday i made a factory reset on the Zyxel switch. Then i enabled IGMP snooping on it, connected all CAPs back on Zyxel and for now, it’s all good. I will wait few days, to see if it’s ok.
Also i moved all IOT device back on my main network so i will probably remove IOT bridge and VLAN, because it’s not needed anymore.
I deleted those rules because it also works without them. I agree, some cleening would be a good idea, i will try my best.
After i made a factory reset on Zyxel swithch everything was ok for quite some time but now the same problems have returned. Guess i will have to replace this switch and also 5009 in some time, because one of the ports is only working with 100 Mbps without any apparent reason.
This is still happening. Now i’m not even sure if switch even has any influence on this behavior. For example, i got connection interrupted messages at 20:19:34. Then i went and check what is happening with cameras all around the house, that stream to NVR and this stream even comes in the switch on the same port where CAP is connected. No disruptions and no disconnections at all. I simply have no idea where should i look for error, because error discription in log isn’t the best.
Now i even tried and leave cameras and CAPs on switch and connected NVR on Mikrotik. At 21:29:54 again all CAPs got connection interrupted but cameras show no disconnection at all and stream is fluent. Out of ideas.
I had the issue once maybe 3 weeks ago. Rebooted 5009 (Capsman and main router), nothing. Rebooted 24 port Zyxel switch, nothing, then i started rebooting all APs on network and it stopped doing this when i rebooted HapAX2 (it happened also before AX2 was put on that place, so it's not device specific). It's like some kind of loop that doesn't stop until you reboot device, that causes problems. AX2 in that place is also switch for 4 more CAPs behind him.
It just throws all CAPs off the network causing connection lost for all Wifi devices every minute or maybe a little bit more.
Yes it is from Telekom, but my main internet is on Telemach. I was testing if i can get Multicast over EOIP and use their TV box on my internet connection and it worked without problem. I don’t use this in practice but i just leave it configured and added UDPXY as a docker container on 5009. So now if i open M3U playlist, 5009 pulls Multicast over EOIP, UDPXY changes it to unicast and then it doesn’t causes problems in network anymore (Multicast traffic on all ports and similar things). I made all LAN cables myself, previusly i had 120 meters of CAT6 lan cable and it worked between Mikrotiks but not between Mikrotik and some dumb POE switch. Now i shortened it quite a bit.
