Community discussions

MikroTik App
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Port flapping RB5009 + vodafone modem

Tue Nov 02, 2021 10:21 pm

I have a pretty standard network config:
Vodafone modem (Arris TG3442) -> RB5009

I'm using port8 as the wan port as the 2.5gbe port is best used elsewhere. Apart from that it's basically the default config from 7.1rc5. The problem is that the port keeps disconnecting, it's a bit random but I guess on average it's every 5-10mins.
20:59:06 dhcp,info dhcp-client on ether8-wan lost IP address 188.194.xxx.xx - lease stopped locally 
20:59:45 interface,info ether8-wan link up (speed 1G, full duplex) 
20:59:46 dhcp,info dhcp-client on ether8-wan got IP address 188.194.xxx.xx
A pcap of the dhcp request seems perfectly normal, it really seems the ethernet connection goes down, I can even see the LEDs on port8 going down:
0.000000	0.0.0.0	255.255.255.255	DHCP	342	DHCP Discover - Transaction ID 0x5fb96bfd
0.028581	88.134.xxx.xxx	255.255.255.255	DHCP	342	DHCP Offer    - Transaction ID 0x5fb96bfd
0.028753	0.0.0.0	255.255.255.255	DHCP	342	DHCP Request  - Transaction ID 0x5fb96bfd
0.050977	88.134.xxx.xxx	255.255.255.255	DHCP	342	DHCP ACK      - Transaction ID 0x5fb96bfd
Reading a bit shows people seem to have similar issues with the RB3011 and 4011 with no obvious solution. I tried a few of the tips, mostly forcing the ethernet auto negociation off and into 1gb/s mode, switched all cables, used different ports etc... Also everything on the mikrotik is on 1000Base-T, so the switch is not in any kind of mixed mode.

Then I put a cheap gig switch in between the modem and the RB5009 and voila it's now working perfectly. Now my network is:
Vodafone modem (Arris TG3442) -> el cheapo switch -> RB5009

I'd put this on the vodafone modem but I had a very similar issue with an RB2011 and a fritzbox, and when I connect a server directly with an i350 and intel network card it's working perfectly. So it really seems like the network ports on mikrotik behave a little weird no?
Last edited by arfoll on Tue Nov 02, 2021 11:02 pm, edited 2 times in total.
 
User avatar
Hominidae
Member Candidate
Member Candidate
Posts: 195
Joined: Thu Oct 19, 2017 12:50 am

Re: Port flapping RB5009 + vodafone modem

Tue Nov 02, 2021 10:59 pm

I had the same issue with my RB4011 and FritzCable....and yes, I purely blame the Fritz as I had it stable for years, before the fritz got pushed a FW-update (it started with FW7.21).
Now I am on a Technicolor Modem in Bridge mode and uptime is several months.
 
fragtion
Frequent Visitor
Frequent Visitor
Posts: 84
Joined: Fri Nov 13, 2009 10:08 pm
Location: Johannesburg, South Africa

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 7:49 pm

Hi , are you still seeing this issue with your RB5009 ? If so , have you figured out what may be causing it ? I have two RB5009's and both do the exact same thing. Occasional patterns of port flapping which comes and goes but is very disruptive
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 9:57 pm

Hi , are you still seeing this issue with your RB5009 ? If so , have you figured out what may be causing it ? I have two RB5009's and both do the exact same thing. Occasional patterns of port flapping which comes and goes but is very disruptive
I have a netgear GS105GE between the rb5009 and the modem, works completely fine with no drops. I havent re-tried recently with the latest updates.
 
Zacharias
Forum Guru
Forum Guru
Posts: 2972
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 10:09 pm

If you test after the latest updates let us know if the problem persists ...
Also, do the link downs follow a specific pattern ? Do they happen for example exactly every 2 or 3 etc minutes or they are totally random ?
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 10:50 pm

If you test after the latest updates let us know if the problem persists ...
Also, do the link downs follow a specific pattern ? Do they happen for example exactly every 2 or 3 etc minutes or they are totally random ?

I had a quick go, updated to 7.1.1 (packages & routerboard). Didn't take long:

20:36:50 system,info router rebooted
20:37:03 interface,info ether8-wan link up (speed 1G, full duplex)
20:37:04 dhcp,info dhcp-client on ether8-wan got IP address 188.194.166.65
...
21:42:30 interface,info ether8-wan link down
21:42:30 dhcp,info dhcp-client on ether8-wan lost IP address 188.194.166.65 - lease stopped locally
21:42:33 interface,info ether8-wan link up (speed 1G, full duplex)
21:42:34 dhcp,info dhcp-client on ether8-wan got IP address 188.194.166.65
21:47:48 interface,info ether8-wan link down
21:47:48 dhcp,info dhcp-client on ether8-wan lost IP address 188.194.166.65 - lease stopped locally
21:47:52 interface,info ether8-wan link up (speed 1G, full duplex)
21:47:52 dhcp,info dhcp-client on ether8-wan got IP address 188.194.166.65
It happens pretty randomly, there doesn't seem to be an obvious pattern and the link comes back really quickly. It seems to happen more when you have a load on the connection, but it's difficult to really say, as you can see above it worked fine for ~1hr then had an issue then as soon as I ran a speedtest on the connection I got another one at 21:47, ran another and nothing.
 
Zacharias
Forum Guru
Forum Guru
Posts: 2972
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 11:06 pm

Did you try with auto negotiation disabled? And set fixed speed at 1Gbps full duplex and then 100 Mbps Full duplex just for testing...
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Re: Port flapping RB5009 + vodafone modem

Wed Jan 12, 2022 11:29 pm

Did you try with auto negotiation disabled? And set fixed speed at 1Gbps full duplex and then 100 Mbps Full duplex just for testing...
Autonegociation is already off, and loop protect is off - I had tried going down to 100Mbps in the past and it made no difference, I can try again later but I really don't see why it would make any difference...
 7 R  name="ether8-wan" default-name="ether8" mtu=1500 l2mtu=1514 mac-address=2C:C8:1B:FF:8F:64 
      orig-mac-address=2C:C8:1B:FF:8F:64 arp=enabled arp-timeout=auto loop-protect=off loop-protect-status=off 
      loop-protect-send-interval=5s loop-protect-disable-time=5m auto-negotiation=no 
      advertise=10M-half,10M-full,100M-half,100M-full,1000M-half,1000M-full full-duplex=yes tx-flow-control=off 
      rx-flow-control=off speed=1Gbps bandwidth=unlimited/unlimited switch=switch1 
I've also got all brand new cables and done quite alot of cable swapping, the wan connection is on premade earthed cat6a of what I assume is reasonable quality (i admit i do not have a tester :)). The whole rack is earthed and the cable modem is also earthed via the cable F plug.

Note its the same cable that went to the netgear switch and for the whole time (~2 months) i didnt get a single port down.
 
User avatar
rushlife
Member Candidate
Member Candidate
Posts: 197
Joined: Thu Nov 05, 2015 12:30 pm
Location: czech republic

Re: Port flapping RB5009 + vodafone modem

Thu Jan 13, 2022 12:32 am

you must use 2.5Gbe port, than it works without any error
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Re: Port flapping RB5009 + vodafone modem

Thu Jan 13, 2022 8:28 am

you must use 2.5Gbe port, than it works without any error
I'm not going to change my whole config for this attempt. But I don't understand why this would change anything, all ports on the RB5009 are connected to the 88E6393X switch? Running the switch in mixed mode I could understand it cause some odd issues but in my case they're all fixed to 1Gbps.
 
fragtion
Frequent Visitor
Frequent Visitor
Posts: 84
Joined: Fri Nov 13, 2009 10:08 pm
Location: Johannesburg, South Africa

Re: Port flapping RB5009 + vodafone modem

Thu Jan 13, 2022 9:00 pm

@arfoll if you check your logs, do all of your ethernet ports go down or only some? For me, if there's a cable plugged into a port (active link on the port), it will go down with the rest whenever the fault triggers. They go down (and come back up) in numerical order, starting with port 1 upwards

The fact that all of the ethernet ports go down and up together at the same time, and in numerical sequence, leads me to believe that it's an issue with the switch chip, or communication to the switch chip. Whether that is hardware or software is obviously going to be a mystery to us until MikroTik support can advise more on the matter. I have an open ticket [Sup #SUP-71476] and am eagerly waiting for some form of acknowledgement

For the record, ether1 is my WAN port and it goes down/up just like the others, so I don't think the 2.5Gbps port exception holds any water ?

Given how seemingly random these occurrences are, and yet how they seem to come in bursts which could last for a while, I've started to wonder if it couldn't maybe be some kind of external/environmental factors which are provoking it. I'm trying to improve cooling of my units (I'm sitting between 41-43 degrees Celsius most of the time) to see if that has any impact. How are your temperatures looking ?
Last edited by fragtion on Thu Jan 13, 2022 9:13 pm, edited 1 time in total.
 
Zacharias
Forum Guru
Forum Guru
Posts: 2972
Joined: Tue Dec 12, 2017 12:58 am
Location: Greece

Re: Port flapping RB5009 + vodafone modem

Thu Jan 13, 2022 9:05 pm

@fragtion the OP refers to a single port flapping and not all together...
 
fragtion
Frequent Visitor
Frequent Visitor
Posts: 84
Joined: Fri Nov 13, 2009 10:08 pm
Location: Johannesburg, South Africa

Re: Port flapping RB5009 + vodafone modem

Thu Jan 13, 2022 9:16 pm

@fragtion the OP refers to a single port flapping and not all together...
Yes that seems to be the case. But yet OP's issue description seems almost identical to what I am experiencing, and there are visibly no other threads around here that I could find discussing this type of issue. On both of my RB5009 devices (I mean what are the odds???) purchased from different parts of the globe, I experience this issue and every time it causes all *connected* ethernet ports to flap simultaneously. So I think it's critical to distinguish for certain if OP is still only utilizing one port, or what happens if they have more than one port connected, etc. For instance, if it's a software issue, then why would it affect some devices with all ports and another device with just one port ? Even if it's a hardware issue, that discrepancy seems obscure wouldn't you agree? That means we're dealing with at least 3 faulty devices between us, two of which have a DIFFERENT fault to the other while showing almost identical (but not the same) symptoms ?
 
arfoll
just joined
Topic Author
Posts: 18
Joined: Mon Sep 24, 2012 8:24 pm

Re: Port flapping RB5009 + vodafone modem

Fri Jan 14, 2022 7:55 am

@arfoll if you check your logs, do all of your ethernet ports go down or only some? For me, if there's a cable plugged into a port (active link on the port), it will go down with the rest whenever the fault triggers. They go down (and come back up) in numerical order, starting with port 1 upwards
No only the port with the cable connected to my cable modem shows any signs of issues for me, the only effect visible on mikrotik is that log line. I was using a different port at one point just to try see if this helped (though never port1), I think it was port2 but it made no difference.

Fyi I'm using more than one port, all on 1gbps and none of them show any issues.

The fact that all of the ethernet ports go down and up together at the same time, and in numerical sequence, leads me to believe that it's an issue with the switch chip, or communication to the switch chip. Whether that is hardware or software is obviously going to be a mystery to us until MikroTik support can advise more on the matter. I have an open ticket [Sup #SUP-71476] and am eagerly waiting for some form of acknowledgement
To be honest this sounds like a different issue. Is yours also fixed by having a dumb switch in between your rb5009 and the flapping port?


Given how seemingly random these occurrences are, and yet how they seem to come in bursts which could last for a while, I've started to wonder if it couldn't maybe be some kind of external/environmental factors which are provoking it. I'm trying to improve cooling of my units (I'm sitting between 41-43 degrees Celsius most of the time) to see if that has any impact. How are your temperatures looking ?
My rack is fully open atm since I still haven't finished it properly, I've never been super careful with temp monitoring the mikrotik but it's sitting at 38C which I consider pretty cool, it's never shown itself to be super warm. I did also hook up a different stronger power supply to just check this wasn't a weird voltage drop but seemed to make no difference.
 
fragtion
Frequent Visitor
Frequent Visitor
Posts: 84
Joined: Fri Nov 13, 2009 10:08 pm
Location: Johannesburg, South Africa

Re: Port flapping RB5009 + vodafone modem

Fri Jan 14, 2022 9:38 pm

To be honest this sounds like a different issue. Is yours also fixed by having a dumb switch in between your rb5009 and the flapping port?
Interesting. Well the symptoms are similar in that it seems sporadic and unprovoked and affects ethernet up/down state. But you're right, probably different issues. I'm starting to wonder if my issue isn't config-related, as it affects both my RB5009's but nobody else seems to be facing this "variant" of the problem.

For what it's worth, this person seems to be facing exactly the same fault as you: https://twitter.com/g_yotuya/status/1475666222096486403
 
aTOMico
just joined
Posts: 1
Joined: Sun Jan 16, 2022 5:31 pm

Re: Port flapping RB5009 + vodafone modem

Sun Jan 16, 2022 5:51 pm

Hi,
had pretty the same issue with my RB4011 and my Arris TG3442.
I found nothing to fix this with my config, so i placed a dump switch between the RB4011 and the Arris and it worked.
I decided to buy a SFP Copper 1G Module for the RB4011 and connected the Artis Modem to that and it worked perfect.
After doing so i never had any port flapping problems. All of my Ethernet ports are working with other devices, so i think its some Kind of incompatibility.
Last edited by aTOMico on Sun Jan 16, 2022 6:13 pm, edited 2 times in total.
 
User avatar
jbl42
newbie
Posts: 48
Joined: Sun Jun 21, 2020 12:58 pm

Re: Port flapping RB5009 + vodafone modem

Mon Jan 17, 2022 12:27 am

But I don't understand why this would change anything, all ports on the RB5009 are connected to the 88E6393X switch?
The 88E6393X is a 11port switch chip: 3x 10GB raw MI and 8x 1GB with integrated PHYs.
One 10GB goes to the CPU, one to the SFP+ and the 3rd is connected to a 2.5GB PHY chip (QCA8081) going to ether1. Ether2-ether8 are connected to 7 of the 8 1GB ports. The 8th 1GB port is unused and hidden in ROS (and also missing in the RB5009 block diagram).
This is also why the ether1 activity/Link LEDs behave slightly different compared to ether2-ether8, as LEDs are driven by the PHYs.

On RB5009 it is possible to see differences in link stability on port ether1 vs. ports ether2-ether8

Who is online

Users browsing this forum: No registered users and 9 guests