Community discussions

MikroTik App
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Weird issue SwOS 2.16 on CRS312

Sat Mar 09, 2024 3:01 am

Yes, I'm bit lazy and use SwOS on that nice home central switch. Earlier fw 2.13 works great about year without any issue but once upgraded to lastest it will hang somehow on 10-20h ..
Every time I will notice that my wifi ap's that are connected to this sw are unavailable and when I log in from my desktop that are directly connected to it it look good. But when login my router (rb4011, with quite basic cfg) I notice that all dhcp leases are gone, only hosts that are with static ip are working.

Once I reboot crs312, everything start working again and all clients get new leases. Tomorrow I need to drop second powerline out and put clockswitch to remain power cable to mitigate issue. But this sound ...

At same time I upgraded ros 7.12.x to 7.14 on rb4011, if it has some bug among this sw.. (yes, my mistake, a) upgrade something that works b) upgrade both on same night)
crs312 is connected to rb4011 with aoc fiber dac (10G).
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Sun Mar 10, 2024 12:35 am

It seems that I need reboot crs312 now every 24h to keep clients get ip addr with dhcp. Just rebooted sw and all clients are now back in dhcp server lease table.
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Sun Mar 10, 2024 11:42 pm

yup .. again same state... need to reboot crs312 to fix all...

Does anyone know where older versions are available? not found from https://mikrotik.com/download or https://mikrotik.com/download/archive
 
kevinds
Long time Member
Long time Member
Posts: 651
Joined: Wed Jan 14, 2015 8:41 am

Re: Weird issue SwOS 2.16 on CRS312

Mon Mar 11, 2024 8:28 am

yup .. again same state... need to reboot crs312 to fix all...
Might be a silly question, but have you tried updating to the latest version?

Nevermind.. It is "Lite" that has 2.18
to keep clients get ip addr with dhcp
Curious.. If you give the hosts static IPs, do they keep working? What is the lease time set to? What happens if you change it?
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Mon Mar 18, 2024 10:46 pm

yup .. again same state... need to reboot crs312 to fix all...
Might be a silly question, but have you tried updating to the latest version?

Nevermind.. It is "Lite" that has 2.18
Question is not silly ... All works nicely on previous version 2.13 and went broken after updated to lastest 2.16.
Tried also reload/reupload lastest second time .. I'm trying figure out is older version available anywhere.. (not found from Mikrotik DL or archive site)

to keep clients get ip addr with dhcp
Curious.. If you give the hosts static IPs, do they keep working? What is the lease time set to? What happens if you change it?
Yes, have now done some debugging .. Hosts that has static ip works ok.
Lease time is "mikrotik standard 10min" ... issue cause always near after 24h uptime .. But, this might be good to test.

I have also CSS326 and in that box latest fw works ok.. but it of course has different switch chip. Also I think that most of CRS312 users may run ROS instead SwOS.

Some cfg:
- DHCP snooping is off
- no ACL
- Mikrotik discovery protocol is on
- snmp enabled
- 5 vlans with port isolation and learning enabled
- some ports has only tagged vlan, some only untagged
- no LAG ports
- all other settings are basically "default" state
- if there is SW behind port of crs312, there is same issue with hosts that are connected to those sw. Most of connected sw are mikrotik, one is zyxel
- my router (rb4011) was connected to crs312 .. first with aoc fiber dac and then tested with rj45 cat6 also .. same issue. Now router is connected to another sw instead this crs312.
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Tue Mar 19, 2024 1:04 pm

sw has now 6d uptime and I set my desktop pc from static to dhcp and it drop nw access as it will not get ip from dhcp server. After change back to static it work as normal.

Added dhcp logging to rb4011 and I don't see any (dhcp) requests from my desktop pc.
It look that after 24h from crs312 reboot, all dhcp traffic are dropped/filtered out until next reboot. With static ip all works ok.
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Tue Mar 19, 2024 3:11 pm

Haaa. Might be resolved now!

in System tab, there is DHCP Snooping and port that provide dhcp service to this area (in my case RB4011 is connected to CSS326 that are connected with dac to CRS312) need to be selected AND next parameter under portlist of snooping ports, "Add Information Option" must be unchecked.

I have tested this now with 2 hosts that are connected to crs312 and current situation look promising.


Let's test it now with this configuration on few days.
 
kevinds
Long time Member
Long time Member
Posts: 651
Joined: Wed Jan 14, 2015 8:41 am

Re: Weird issue SwOS 2.16 on CRS312

Tue Mar 19, 2024 4:53 pm

Promising update.
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Wed Mar 20, 2024 11:41 am

Still working ... just added more hosts to crs312 as well another sw and wifi ap ..

Summary of solutions: with SwOS 2.16
- CRS312: in system tab under DHCP Snooping "Add Information Option" need to be unchecked.
- CSS326: in system tab under DHCP Snooping "Add Information Option" need to be checked.
(- RB4011: no changes any configs)

Next step: if all works properly tomorrow, 24h after this update, I will reboot both switches to ensure that all works properly.
 
sakke42
just joined
Topic Author
Posts: 8
Joined: Tue Aug 01, 2023 2:26 pm

Re: Weird issue SwOS 2.16 on CRS312

Thu Mar 21, 2024 9:52 am

near all setups was identical as begining of case, only changes was "Add additional.." checkbox.

Today morning near nothing works. Only hosts that are connect directly crs312 works ok, they got ip addr from dhcp etc ... BUT clients that are behind any connected box between crs312 and host, like wifi box have loss their ip after yesterday..
-> checked that direct connected host get ip after nw cable was small time unconnected
-> moved wifi ap to behind css326, not getting ip from rb4011 that are connected direct to crs312
-> after above finding, I moved rb4011 to css326 and then all wifi clients get ip
-> after router move to css326, then crs312 clients no more get ip

... sad as last night everything works ok.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 3:23 am

I'm seeing this too. Had to roll back to 2.13.

I thought Mikrotik would fix this quickly, because it's a show-stopping bug. But it looks like we're going to have to shout about it to get it patched.
 
holvoetn
Forum Guru
Forum Guru
Posts: 5670
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 8:59 am

Has everyone experiencing this problem created a support ticket ?
Otherwise changes are high they don't even know about it ... some MT staff uses this forum but not all. And those that do, do not read all posts..
 
EdPa
MikroTik Support
MikroTik Support
Posts: 305
Joined: Fri Sep 15, 2017 10:05 am
Location: Riga
Contact:

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 4:41 pm

Hi sakke42 and TantalizingEmu,

Thank you for sharing your troubleshooting steps so far. However, I couldn't replicate the same behavior in our labs.

Could you provide your network diagram and specify the ports/VLANs where the DHCP client and server are located (mark the clients that work and that do not)? Additionally, share screenshots from the SwOS VLAN, VLANs, and System pages. If you have LAG configured, include a screenshot of the SwOS LAG page as well.

If your setup includes multiple SwOS devices, try disabling the System "Add Information Option" setting on all switches and let us know the results.

If your network includes other devices that might add Option 82, ensure the SwOS ports connected to those devices are marked as "Trusted," as described on the SwOS System page.
Ports that receive DHCP client packets with already added Option-82 must also be trusted, otherwise these packets are dropped.
 
User avatar
k6ccc
Forum Guru
Forum Guru
Posts: 1528
Joined: Fri May 13, 2016 12:01 am
Location: Glendora, CA, USA (near Los Angeles)
Contact:

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 5:08 pm

For whatever it's worth, My RB4011 is directly connected to two different CSS326 switches and then there is one more CSS326 in the house. All the switches have been running 2.16 since within a day or two of 2.16 coming out. No problems at all.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 10:54 pm

For whatever it's worth, My RB4011 is directly connected to two different CSS326 switches and then there is one more CSS326 in the house. All the switches have been running 2.16 since within a day or two of 2.16 coming out. No problems at all.

Good to know it's working on other models. Both sakke42 and I have the CRS312.

Has everyone experiencing this problem created a support ticket?

No, and that's entirely my bad. I was being lazy and just thought a serious bug like this would get identified and addressed quickly.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Fri May 24, 2024 11:25 pm

Could you provide your network diagram and specify the ports/VLANs where the DHCP client and server are located (mark the clients that work and that do not)? Additionally, share screenshots from the SwOS VLAN, VLANs, and System pages. If you have LAG configured, include a screenshot of the SwOS LAG page as well.

I've attached screenshots of the GUI's Link and System tabs on 2.13. The Link tab shows all the currently connected devices and the negotiated speeds. The DHCP port will be COMBO4, which is connected via 10GbE copper to a Netgear XS724EM switch (firmware latest at 1.0.2.8 ). The XS724EM switch is connected via SFP to an Asus AX89X router.

Not sure if this matters, but there is a Wi-Fi node connected to the CRS312 (labeled AX88U). I first became aware of the problem in 2.16 because all my wireless clients were dropping off that node. That's when I saw that every device connected to the CRS312 was no longer reachable.

No VLANS are in use on my network (all "disabled" in the CRS312 GUI), and no LAG is configured on the CRS312.

This switch is "mission critical" in the sense that it links key devices in our home network, which is why I rolled back to 2.13 at the first sign of trouble. I'm happy to upgrade to 2.16 and then capture and submit logs, but I don't know how to do it in the SwOS GUI.
You do not have the required permissions to view the files attached to this post.
 
lukasj
just joined
Posts: 2
Joined: Mon May 27, 2024 8:41 am

Re: Weird issue SwOS 2.16 on CRS312

Mon May 27, 2024 8:44 am

Same problem here. Need to restart CSS326 for clients to get IP.
 
EdPa
MikroTik Support
MikroTik Support
Posts: 305
Joined: Fri Sep 15, 2017 10:05 am
Location: Riga
Contact:

Re: Weird issue SwOS 2.16 on CRS312

Mon May 27, 2024 2:49 pm

TantalizingEmu, I was still unable to replicate the same behavior. Try updating to version 2.16, take screenshots of all the SwOS pages, and report this to our support system or through e-mail: support@mikrotik.com.

If possible, capture DHCP packets on both client and server devices and share the pcap files as well.

While you are at v2.16, check if any of the ports on the SwOS RSTP page show State "discarding", while they should be "forwarding" (ports that are up and running and not creating a L2 loop). Also, try disabling "Add Information Option" on the System page, and see if that makes any difference.

lukasj, please make a report as well.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Wed May 29, 2024 6:19 pm

While you are at v2.16, check if any of the ports on the SwOS RSTP page show State "discarding", while they should be "forwarding" (ports that are up and running and not creating a L2 loop). Also, try disabling "Add Information Option" on the System page, and see if that makes any difference.

Okay, I upgraded to 2.16 via the GUI this morning. I have a busy day and don't have time to dig into the steps required for DHCP packet capture. I am seeing "discarding" on a number of ports on the RSTP page, but each of these links is currently inactive/unused.

RSTP.png

"Add information option" is unchecked on the 'System' page.

I've grabbed screenshots of each GUI page and will be sending that to the support e-mail.

Thanks for the speedy response and the willingness to help. I would love to know this is fixed for the upcoming 2.17 release.
You do not have the required permissions to view the files attached to this post.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Fri May 31, 2024 7:11 pm

Well, we're now at two full days of uptime, which is more than I had previously on 2.16. It looks un-checking the "Add information option" was important to the stability I'm now seeing.

I don't want to get my hopes up, as sakke42 briefly saw an improvement before the issue reappeared. But I did want to offer a 48-hour status update.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Mon Jun 03, 2024 8:00 pm

Okay, after 5 days of uptime, I just had my first post-2.16 upgrade error. My NAS was unreachable on any port, which meant that port 1 (Ethernet), Combo port 2 (Ethernet) and Combo port 3 (SFP) were all inaccessible. However, the switch itself was reachable and showed no errors in the GUI.

Rebooting the switch immediately restored connectivity, which shows that it wasn't a problem with the NAS.

I'd like to delve into things to see if I can pinpoint what happened. How does one download and examine system logs in SwOS?
 
lukasj
just joined
Posts: 2
Joined: Mon May 27, 2024 8:41 am

Re: Weird issue SwOS 2.16 on CRS312

Tue Jun 04, 2024 8:46 am

Okay, after 5 days of uptime, I just had my first post-2.16 upgrade error. My NAS was unreachable on any port, which meant that port 1 (Ethernet), Combo port 2 (Ethernet) and Combo port 3 (SFP) were all inaccessible. However, the switch itself was reachable and showed no errors in the GUI.

Rebooting the switch immediately restored connectivity, which shows that it wasn't a problem with the NAS.

I'd like to delve into things to see if I can pinpoint what happened. How does one download and examine system logs in SwOS?
In my case it is just a DHCP packet problem. Ports and connectivity work when I manually configure IP. So try plugging in a statically addressed device before rebooting the switch and see if connectivity works.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Tue Jun 04, 2024 6:46 pm

In my case it is just a DHCP packet problem. Ports and connectivity work when I manually configure IP. So try plugging in a statically addressed device before rebooting the switch and see if connectivity works.

These devices are static IP'd. One port is the NAS' SMC GUI, which is manually assigned a static IP on both the client side and the router side (as opposed to, say, just the router side). The other two ports are part of a NAS-side bond (not LAG on the switch side) that also has a static IP configured in the same way,

This suggestion is also a workaround, not a fix. I can't and don't want to assign every single device that's connected to this switch a static IP.

Also, the drop happened again sometime yesterday evening. Around 11pm I noticed the NAS was inaccessible and I had to reboot the switch.
 
TantalizingEmu
just joined
Posts: 8
Joined: Fri May 24, 2024 3:21 am

Re: Weird issue SwOS 2.16 on CRS312

Thu Jun 06, 2024 6:58 pm

Okay, the dropouts are now happening once per day with the 2.16 firmware. Static IPs and unchecking "Add information option" have had no effect.

I don't WTF to do, as there's no guidance on how to download and submit logs, and the support team has not responded to my issue report. Even trying to access the support portal results in the error below.
Mikrotik_503_sm.png

Rolling back to 2.13.
You do not have the required permissions to view the files attached to this post.
 
Yowa
just joined
Posts: 1
Joined: Sat Jun 08, 2024 9:06 am

Re: Weird issue SwOS 2.16 on CRS312

Sat Jun 08, 2024 9:44 am

Hi,

got the same error with my crs312, i plan to do a factory reset and will report back If that helped next weekend.
I will do a full screenshot of every tab before and after to Log the differences. If the reset should do the trick i will share the differences i have found with you.

Edit: Had my third crash tonight, so factory reset doesn't seem to fix this. The factory settings also doesn't seem to do the trick as i didn't change any settings apart from password and static Switch ip.
I will update my Support Ticket accordingly.

Greetings
Yowa
Last edited by Yowa on Sat Jun 08, 2024 10:09 am, edited 1 time in total.

Who is online

Users browsing this forum: No registered users and 0 guests