v7.16.2 [stable] is released!

bbs2web · Tue Oct 08, 2024 9:27 am

Herewith the MLAG configuration:
switch1a:

/interface bridge
add admin-mac=DE:AD:BE:EF:DE:A0 auto-mac=no name=bridge port-cost-mode=short priority=0x5000 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus4 ] comment="Partner Switch sfp-sfpplus4:" l2mtu=10218
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond-peer slaves=sfp-sfpplus4 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=39 mode=802.3ad name=bonde39 slaves=ether39 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=40 mode=802.3ad name=bonde40 slaves=ether40 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=41 mode=802.3ad name=bonde41 slaves=ether41 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=42 mode=802.3ad name=bonde42 slaves=ether42 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=43 mode=802.3ad name=bonde43 slaves=ether43 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=44 mode=802.3ad name=bonde44 slaves=ether44 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=45 mode=802.3ad name=bonde45 slaves=ether45 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=46 mode=802.3ad name=bonde46 slaves=ether46 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=47 mode=802.3ad name=bonde47 slaves=ether47 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=48 mode=802.3ad name=bonde48 slaves=ether48 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=101 mode=802.3ad name=bondsfp1 slaves=sfp-sfpplus1 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=102 mode=802.3ad name=bondsfp2 slaves=sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=103 mode=802.3ad name=bondsfp3 slaves=sfp-sfpplus3 transmit-hash-policy=layer-3-and-4
/interface bridge mlag
set bridge=bridge peer-port=bond-peer
/interface bridge port
add bridge=bridge comment="MLAG Peer:" interface=bond-peer pvid=99 trusted=yes
add bridge=bridge interface=bonde38 restricted-role=yes
add bridge=bridge interface=bonde39 restricted-role=yes
add bridge=bridge interface=bonde40 restricted-role=yes
add bridge=bridge interface=bonde41 restricted-role=yes
add bridge=bridge interface=bonde42 restricted-role=yes
add bridge=bridge interface=bonde43 restricted-role=yes
add bridge=bridge interface=bonde44 restricted-role=yes
add bridge=bridge interface=bonde45 restricted-role=yes
add bridge=bridge interface=bonde46 restricted-role=yes
add bridge=bridge interface=bonde47 restricted-role=yes
add bridge=bridge interface=bonde48 restricted-role=yes
add bridge=bridge edge=no-discover interface=bondsfp1 restricted-role=yes trusted=yes
add bridge=bridge edge=no-discover interface=bondsfp2 restricted-role=yes trusted=yes
add bridge=bridge interface=bondsfp3 restricted-role=yes
/interface bridge vlan
add bridge=bridge tagged=bond-peer vlan-ids=1
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=10
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=11
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=21
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=22
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=23
add bridge=bridge tagged="bridge,bond-peer,bondsfp1,bondsfp2,bondsfp3,bonde38,bonde39,bonde40,bonde41,bonde42,bonde43,bonde44,bonde45,\
    bonde46,bonde47,bonde48" vlan-ids=50
add bridge=bridge tagged="bridge,bond-peer,bondsfp1,bondsfp2,bondsfp3,bonde38,bonde39,bonde40,bonde41,bonde42,bonde43,bonde44,bonde45,\
    bonde46,bonde47,bonde48" vlan-ids=52

switch1b:

/interface bridge
add admin-mac=DE:AD:BE:EF:DE:A1 auto-mac=no name=bridge port-cost-mode=short priority=0x5000 vlan-filtering=yes
/interface ethernet
set [ find default-name=sfp-sfpplus4 ] comment="Partner Switch sfp-sfpplus4:" l2mtu=10218
/interface bonding
add lacp-rate=1sec mode=802.3ad name=bond-peer slaves=sfp-sfpplus4 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=39 mode=802.3ad name=bonde39 slaves=ether39 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=40 mode=802.3ad name=bonde40 slaves=ether40 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=41 mode=802.3ad name=bonde41 slaves=ether41 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=42 mode=802.3ad name=bonde42 slaves=ether42 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=43 mode=802.3ad name=bonde43 slaves=ether43 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=44 mode=802.3ad name=bonde44 slaves=ether44 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=45 mode=802.3ad name=bonde45 slaves=ether45 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=46 mode=802.3ad name=bonde46 slaves=ether46 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=47 mode=802.3ad name=bonde47 slaves=ether47 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=48 mode=802.3ad name=bonde48 slaves=ether48 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=101 mode=802.3ad name=bondsfp1 slaves=sfp-sfpplus1 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=102 mode=802.3ad name=bondsfp2 slaves=sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
add lacp-rate=1sec mlag-id=103 mode=802.3ad name=bondsfp3 slaves=sfp-sfpplus3 transmit-hash-policy=layer-3-and-4
/interface bridge mlag
set bridge=bridge peer-port=bond-peer
/interface bridge port
add bridge=bridge comment="MLAG Peer:" interface=bond-peer pvid=99 trusted=yes
add bridge=bridge interface=bonde39 restricted-role=yes
add bridge=bridge interface=bonde40 restricted-role=yes
add bridge=bridge interface=bonde41 restricted-role=yes
add bridge=bridge interface=bonde42 restricted-role=yes
add bridge=bridge interface=bonde43 restricted-role=yes
add bridge=bridge interface=bonde44 restricted-role=yes
add bridge=bridge interface=bonde45 restricted-role=yes
add bridge=bridge interface=bonde46 restricted-role=yes
add bridge=bridge interface=bonde47 restricted-role=yes
add bridge=bridge interface=bonde48 restricted-role=yes
add bridge=bridge edge=no-discover interface=bondsfp1 restricted-role=yes trusted=yes
add bridge=bridge edge=no-discover interface=bondsfp2 restricted-role=yes trusted=yes
add bridge=bridge interface=bondsfp3 restricted-role=yes
/interface bridge vlan
add bridge=bridge tagged=bond-peer vlan-ids=1
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=10
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=11
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=21
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=22
add bridge=bridge tagged=bridge,bond-peer,bondsfp1,bondsfp2 vlan-ids=23
add bridge=bridge tagged="bridge,bond-peer,bondsfp1,bondsfp2,bondsfp3,bonde38,bonde39,bonde40,bonde41,bonde42,bonde43,bonde44,bonde45,\
    bonde46,bonde47,bonde48" vlan-ids=50
add bridge=bridge tagged="bridge,bond-peer,bondsfp1,bondsfp2,bondsfp3,bonde38,bonde39,bonde40,bonde41,bonde42,bonde43,bonde44,bonde45,\
    bonde46,bonde47,bonde48" vlan-ids=52

spippan · Tue Oct 08, 2024 12:10 pm

MLAG on 2 x CRS354-48G-2S+2Q+ switches continues to be a problem. MLAG peer link is a 802.3ad (LACP) bond interface utilising two Q+ (40G) ports with MikroTik DACs. We've tried swapping the DACs which made no difference.

Bond and slave interface status shows zero 'link down' events but MLAG peer status is reported as flapping regularly:
Code: Select all
 09:28:26 bridge,warning "bridge" peer disconnected
 09:28:26 bridge,warning "bridge" peer link down
 09:28:26 bridge,info "bridge" peer link up
 09:28:26 bridge,info "bridge" peer connected
 09:28:26 bridge,info "bridge" peer becomes secondary DC:2C:6E:D2:AF:4B
 
I've sent numerous supout.rif files, for all 7.15 versions of the firmware and now 7.16; never hear anything back from support@mikrotik.com. Also no response or acknowledgement when I log a support case in the support portal.

PS: Works perfectly 99.9% of the time, but these frequent peer link flaps interrupt Teams calls for 2-3 seconds each time they occur and the result is that we can't recommend this for client networks as a result.

PS: Also makes no difference when I temporarily use a 10G DAC and reconfigure the bond-peer interface to use sfp-sfpplus4 instead of the 2 x QSFP+ interfaces (not a QSFP+ (40G) issue).

NB: Only happens during office hours, never over weekends or evenings. Network utilisation is actually higher at night when backups run.

a quick thought ... is auto-negotiation enabled on those 40G links? is FEC configured on the 40G links on both sides?
i'd try to set the link speeds to fixed on both ends and configure FEC accordingly ("fec74" on 40G)

bbs2web · Tue Oct 08, 2024 11:16 pm

a quick thought ... is auto-negotiation enabled on those 40G links? is FEC configured on the 40G links on both sides?
i'd try to set the link speeds to fixed on both ends and configure FEC accordingly ("fec74" on 40G)

It's using direct attach copper cables so there, IMHO, wouldn't be forwarding error correction capabilities as this would need to be provided by a transceiver. The problem unfortunately continues when we swap the QSFP (40G) ports for a standard 10 Gbps SFP+ DAC and move bond-peer to utilise eg sfp-sfpplus4.

PS: Thanks for your feedback though!

Edit: I see FEC74 doesn't have anything to do with forward error correction. Will read up about it and experiment, the interface itself nor the bond register drops or breaks in communication though. Only the MLAG peer link reports disconnecting and then re-establishing connectivity.

spippan · Wed Oct 09, 2024 12:09 am

a quick thought ... is auto-negotiation enabled on those 40G links? is FEC configured on the 40G links on both sides?
i'd try to set the link speeds to fixed on both ends and configure FEC accordingly ("fec74" on 40G)

Edit: I see FEC74 doesn't have anything to do with forward error correction. Will read up about it and experiment, the interface itself nor the bond register drops or breaks in communication though. Only the MLAG peer link reports disconnecting and then re-establishing connectivity.

hm maybe i misinterpreted something on the FEC part. but maybe you get some results on that end.
as for auto-negotiation - tried to set both ends to fixed link speeds and duplex?

3dfx · Wed Oct 09, 2024 8:52 pm

I think that the default config of v7.16 is missing the correct settings for the LEDs on the RB951Ui-2HnD.
Can anyone confirm that?

coddy · Thu Oct 10, 2024 7:37 am

@bbs2web MLAG Config:

Just some quick observations, may or may not help:
I see you are tagging the bridge with every vlan, unless you are creating vlans under the bridge to add IPs for routing (not supported with MLAG) I am not sure why you are doing that. My MLAG switch is purely L2 with the exception of a management IP which is added to the bridge itself, with the bridge pvid being my management network. You should remove the tagged="bridge,bond-peer,..." and replace with tagged="bond-peer,..." otherwise at a guess you are potentially driving additional traffic to the CPU. This may or may not help, it is just an observation. I do not tag the bridge under /interface/bridge/vlan in any entries (it ends up being dynamically tagged in the vlan I set the bridge pvid to).

Looks like you are correctly setting aside a vlan for the peerlink, and have untagged that vlan on the peerlink, and tagged vlan 1 on the peerlink, so that looks good.

I don't think a port like spf-sfpplus4 (a straight 10G interface) would have any FEC settings (looks like that is for 25G+ ports), I have not set any for my interface an it works.

I would also wonder (like you have in the past), if this is something to do with the chipset in the CRS358, it does work on the CRS317. You have removed qsfp from the equation, and it is still failing, and that eliminates the port, and DAC cable, since you had to use another port and DAC. Carefully remove your bridge from being tagged under interface/bridge/vlan (have a console cable at hand if you drop L3 to the switch). I am assuming you are not trying to use this for L3 (Since you can't with MLAG anyway).

ps. Only other difference I can see is you have set a static mac address, in mine the bridge has an automatic mac address.

pps. Just looked at your bridge definition and you are not specifying a spanning tree protocol. Spanning tree is a requirement for MLAG I thought, mine is set to rstp.

Hope you find something in this that helps.

Milecus · Thu Oct 10, 2024 6:20 pm

@3dfx:
Install two packages: routeros & wireless & reset the configurations

3dfx · Thu Oct 10, 2024 6:51 pm

@3dfx:
Install two packages: routeros & wireless & reset the configurations

I just found out that I missed to install the wireless package and came to delete my post...
Thanks!

4L3xN3t · Fri Oct 11, 2024 12:49 pm

Hi All!
I have a RB5009, a CRS309 (connected via SFP+ to the RB5009) and a cAPax (connected via ethernet to the RB5009).
They were on 7.15.3 working perfectly till yesterday when I decided to upgrade to 7.16 all of them.
First of all something strange happened to the CRS309. Downloaded and installed the fw, rebooted, upgraded from routerboard window, rebooted again => DEAD. I had to flash fw back with netinstall.
After this something strange happened: I had more or less no issues with the RB5009 and cAPax, but the devices connected to the CRS309 were no more able to get a IP (DHCP server is on RB5009, it assigns IPs to the switch and AP too, this two have DHCP client set on bridge interface).
So looking under DHCP client on the CRS309 I found two entries, both of them dynamic, one from bridge and one from sfp-sfpplus1 (it goes to the RB5009), with the same IP, one of them was marked as "invalid". Can't delete, disable or modify those 2 entries and clearly there was a mess in the route list (everything duplicate).
After downgrading everything to 7.15.3 and applying the old settings (just leaving as it is was not enough) it worked again.

At this point it might just be something I had previously misconfigured that didn't was important on previous fw, but can't imagine what.

I had a RB750v2 too that I use for some testing and wanted to replicate the procedure (I'm now a bit afraid of doing random things on my daily devices), despite beeing a router is configured to act just like a normal switch (a very simple config I'd say).
So I exported the config and did a screenshot of what I think is important before and after upgrading it to 7.16 and I experienced the same behaviour: two DHCP client entries, everything duplicates, conflicts, etc.

I'll post them here.

Can you spot something really wrong I did that could cause the issue?
thank you

infabo · Fri Oct 11, 2024 1:01 pm

/interface detect-internet
set detect-interface-list=all

That's where your dynamic DHCP clients come from. And your mess in the routes. Please do yourself a favor and disable detect-internet.

/interface/detect-internet/set detect-interface-list=none

4L3xN3t · Fri Oct 11, 2024 1:34 pm

Please do yourself a favor and disable detect-internet.

I remember that I enabled it because otherwise something didn't work, but cannot remember exactly what...
Anyway just did it on the testing devices and it worked...now I'm a bit afraid of repeating the process on the "production" devices :D
I'm going to keep you updated

PS: I only now red the notice on the documentation: "Note that Detect Internet can install DHCP clients, default routes, DNS servers and affect other facilities. Use with precaution, and after enabling the service, check how it interferes with your other configuration." :D :D :D

Question: It shouldn't cause issues if I keep it enabled on the main router, right?

thank you

Fri Oct 11, 2024 2:06 pm

Most here will keep it disabled.

HB1 · Fri Oct 11, 2024 2:13 pm

Hi All,

I use my hAP ac3 as a Home gw and I use PPPOE client to connect to Internet. From my side there is a default VRF only ID 0 to all interface and not use BGP.

Based on previous comments I guess the BGP and /or custom VRF config may cause issue to connect Internet (via PPPOE ?)

Should I expect any problem with Internet access if I upgrade my router to 7.16?

Currently I'm running 7.15.3.

Thank you for your help in advance

CGGXANNX · Fri Oct 11, 2024 2:24 pm

Please do yourself a favor and disable detect-internet.
I remember that I enabled it because otherwise something didn't work, but cannot remember exactly what...

I think many "accidentally" enabled it because the mobile MikroTik app requires "Detect Internet" in order to show the traffic graphs on its home screen.

Fri Oct 11, 2024 2:25 pm

What's new in 7.16.1 (2024-Oct-10 17:03):

*) defconf - changed wireless installation from "indoor" to "any";
*) defconf - disable 5GHz secondary channel on RB4011;
*) dns - do not look up local cache when executing ":resolve" command with specified "server" parameter (introduced in v7.16);
*) sfp - improved initialization for certain SFP modules on CRS309 and CRS317 devices ("/system routerboard upgrade" required);

4L3xN3t · Fri Oct 11, 2024 2:34 pm

I remember that I enabled it because otherwise something didn't work, but cannot remember exactly what...

I think many "accidentally" enabled it because the mobile MikroTik app requires "Detect Internet" in order to show the traffic graphs on its home screen.

mh no, it was more something related to the ability of finding and downloading updates or that had to do with DDNS or time update...but I tested these 3 things and seems to work even if detect internet is disabled. maybe a old bug?
anyway I can't find an explanation to why it became a problem in 7.16. it's years I have that option enabled and did tons of fw updates

infabo · Fri Oct 11, 2024 2:45 pm

I remember that I enabled it because otherwise something didn't work, but cannot remember exactly what...

I think many "accidentally" enabled it because the mobile MikroTik app requires "Detect Internet" in order to show the traffic graphs on its home screen.

🤯🤯🤯🤯🤯🤯

infabo · Fri Oct 11, 2024 2:50 pm

What's new in 7.16.1 (2024-Oct-10 17:03):

*) defconf - changed wireless installation from "indoor" to "any";

How can this be a change in 7.16.1?

"configuration.installation" was introduced in 7.17beta2:

What's new in 7.17beta2 (2024-Sep-27 10:07):

*) wifi - added configuration.installation property to limit use of indoor-only channels;

Fri Oct 11, 2024 3:33 pm

The change is for "wireless" not "wifi".

infabo · Fri Oct 11, 2024 3:37 pm

Thanks for clarification!👍

Already updated my devices. up and running.

bratislav · Sat Oct 12, 2024 12:13 pm

Is there a list of supported SFP modules and has it been updated?

I presume you could have find this yourself, the link is listed in all devices specs that have SFP interfaces:
https://help.mikrotik.com/docs/display/ ... patibility
And it is updated from time to time...

Unless you are talking about 3rd parties modules in which case I just don't see why would Mikrotik waste time to check some other company product, that should be done by the OEM...

mszru · Sat Oct 12, 2024 7:15 pm

Did anyone notice any problems with attached USB sticks?
Mine got removed and added again. All containers show up as "running" but not longer responsive.

The USB stick (Kingston DTKN/64) and the 4G modem (Alcatel IK41VE1) are connected to my hAP ax3 via USB 3 hub (Gembird UHB-U3P4-03). The router is powered by the mini-UPS that delivers up to 42W (12V, 3.5A) which should be sufficient given the max power consumption of hAP ax3 (38W).

 10:37:28 disk,info remove usb1
 10:37:29 disk,info add usb1 model:Kingston DataTraveler 3.0 interface:USB 3.20 5000Mbps size:62.0G fs:ext4
 . . .
 10:45:42 interface,info lte1 detect WAN
 10:45:44 disk,info remove usb1
 10:45:46 disk,info add usb1 model:Kingston DataTraveler 3.0 interface:USB 3.20 5000Mbps size:62.0G fs:ext4
 10:45:49 interface,info lte1 detect INTERNET

I think it started since the upgrade to 7.16 as I don't remember having such problems at 7.15.3
Shall I try enabling logging for "disk,debug" topics to find out the reason or submit the SUP ticket? :)

UPDATE:
The filesystem on the USB stick got corrupted and that was likely the cause of the problem. After fixing filesystem errors (and connecting the USB stick to a different port of the hub) the disk is no longer removed and re-added.

Sat Oct 12, 2024 7:21 pm

Is it startup script which does this ?
Could be to assure usb3 devices are properly recognized as such after startup.
Long standing issue with some brands especially on rb5009.

Curious too where this comes from otherwise.

mszru · Sat Oct 12, 2024 8:06 pm

Is it startup script which does this ?

No, the only startup script I have is the one that turns on/off the leds depending on the time of the day.
And just before the disk removal there was some abnormal activity of the Chinese air monitor device. I don't think these events are related, but who knows...

 07:00:00 system,info led settings changed by scheduler:leds-on (/system leds settings set all-leds-off=never)
 09:37:52 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -45
 09:38:05 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -46
 09:38:05 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:38:06 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:39:40 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -46
 09:39:41 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -46
 09:39:41 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:39:41 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:48:46 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -45
 09:48:49 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -46
 09:48:50 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:48:50 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:50:05 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -45
 09:50:20 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -46
 09:50:20 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:50:21 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:53:01 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -41
 09:53:16 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -41
 09:53:16 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:53:19 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:59:21 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -46
 09:59:22 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -47
 09:59:22 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 09:59:22 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 10:37:15 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -45
 10:37:28 disk,info remove usb1
 10:37:29 disk,info add usb1 model:Kingston DataTraveler 3.0 interface:USB 3.20 5000Mbps size:62.0G fs:ext4
 10:38:01 wireless,info 54:5A:A6:**:**:**@wifi4 connected, signal strength -46
 10:38:02 dhcp,info iot-dhcp4 deassigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 10:38:02 dhcp,info iot-dhcp4 assigned 192.168.22.249 for 54:5A:A6:**:**:** cgllc-airmonitor-b1_miio32732741
 10:45:37 wireless,info 54:5A:A6:**:**:**@wifi4 disconnected, connection lost, signal strength -46
 10:45:42 interface,info lte1 detect WAN
 10:45:44 disk,info remove usb1
 10:45:46 disk,info add usb1 model:Kingston DataTraveler 3.0 interface:USB 3.20 5000Mbps size:62.0G fs:ext4
 10:45:49 interface,info lte1 detect INTERNET

usx · Sat Oct 12, 2024 11:41 pm

No immediate problems noticed after upgrading the following from 7.15.3:

- RBmAP2n ("mAP 2n") x2
- RBD25G-5HPacQD2HPnD ("Audience")
- 2011UiAS-2HnD
- hAP RB951Ui-2nD ("hAP")
- RB4011iGS
- hEX 750G r3 ("hEX")

I also made a firmware upgrade on all these devices, all the firmwares were still at 7.2.

Have been running for a couple of hours without problems.

actck · Sun Oct 13, 2024 7:01 am

Counter is wrong ? Do you know why ?

llag · Mon Oct 14, 2024 4:43 am

What's new in 7.16.1 (2024-Oct-10 17:03):

*) sfp - improved initialization for certain SFP modules on CRS309 and CRS317 devices ("/system routerboard upgrade" required);

Works for my Zaram XGSPON module. Thanks for the update

starlingus · Mon Oct 14, 2024 3:04 pm

I recently upgraded my hAP ax2, RB5009UG+S+IN, and mAP Lite from version 7.12.1 to 7.16.1. The RB5009UG+S+IN is handling OVPN, WireGuard, DNS, VLANs, CapsMan (for two hAP ax2 devices), and Containers. => After 24 hours, everything is running smoothly with no issues.

I have a suggestion regarding the v7 stable releases: could it be beneficial to adopt an odd-even release cycle, similar to how the Linux kernel used to be released?

For example:

Odd-numbered releases could focus on introducing new features, even if they might bring heavy breaking changes. This requires extensive testing and broad customer feedback.

Even-numbered releases could prioritize stability and refinements, addressing feature reliability and minimizing impact.

This approach might be a good compromise for customers who are currently missing a long-term v7 release.

pe1chl · Mon Oct 14, 2024 5:05 pm

*) bgp - fixed corrupted as-path when received update with empty AS_PATH attribute (introduced in v7.15);

Are you sure this problem has been completely fixed?
I am still seeing corrupted AS-PATH in a setup with mixed iBGP and eBGP... probably will try to switch to exclusive use of eBGP.

bratislav · Mon Oct 14, 2024 5:47 pm

Odd-numbered releases could focus on introducing new features, even if they might bring heavy breaking changes. This requires extensive testing and broad customer feedback.

Even-numbered releases could prioritize stability and refinements, addressing feature reliability and minimizing impact.
This approach might be a good compromise for customers who are currently missing a long-term v7 release.

How would that be different than having STABLE and LONG-TERM releases... except it would probably be more confusing having 2 stable releases with different version numbers...

The issue is probably not the naming but the lack of resources for maintaining separate 2 code trees...

K0NCTANT1N · Mon Oct 14, 2024 6:30 pm

Believe that v7.12 should be moved to long-term support, as it is stable for devices with limited resources, such as hAP ac2, hAP lite, SXTsq and others

Many users in our situation face similar issues with updates

MikroTik web indicates that these devices receive update support, but their functionality in the long-term will be limited, highlighting the need to use a more stable version

Amm0 · Mon Oct 14, 2024 8:44 pm

The issue is probably not the naming but the lack of resources for maintaining separate 2 code trees...

It's as folks think Mikrotik just declaring something makes it so. A bug-free version is not solved by nomenclature. There thousands of fix from say 7.12.x to 7.16.1 - each one of those made something "unstable"/"buggy"/etc for someone. Nothing stops anyone from stopping at 7.12 - I do in fact, until 16MB can be replace or netinstall'ed. I'd imagine when the length gets shorter may be best indicate when a long-term is coming. Now I do think they should add a new channel, "previous-stable" the just floats.

Believe that v7.12 should be moved to long-term support [...]
Many users [...] face similar issues with updates

As I've long advocated, some new channels like "previous-major" and "previous-minor" would really help to deal with the "uncertainty" in new releases. Bridging the "gap" between all the work that goes into "declaring long-term" - while still allow folks to rollback just one version prior as a "pseudo-long-term" solution. Without all the overloaded meaning in "stable" and "long-term".

turan · Tue Oct 15, 2024 9:02 am

anyone having mikrotik routing policy issue like this: viewtopic.php?t=211706

starlingus · Tue Oct 15, 2024 1:20 pm

The suggestion to alternate odd/even release numbers with specific purposes is based on user feedback from forum comments on each release. Simplifying these comments, there seem to be two main groups of users:

Users requesting more features as quickly as possible to cover a broader range of business scenarios.

Users seeking a well-tested, stable release suitable for production deployment to cutomers with minimal risk.

In one thread, I read a helpful explanation about why v7 cannot yet be promoted to long-term status. It involves feature parity (or at least some compatibility) with v6 features, among other reasons I can’t fully recall.

I'm suggesting that this odd/even approach could serve as a temporary solution until v7 reaches long-term status, addressing both user groups' needs without requiring changes to development processes, as it would be controled by product management.

Even releases would focus on stability, reliability, and performance, while

odd releases would emphasize new features and potentially introduce breaking changes.

jbl42 · Tue Oct 15, 2024 6:04 pm

I'm suggesting that this odd/even approach could serve as a temporary solution until v7 reaches long-term status, addressing both user groups' needs without requiring changes to development processes, as it would be controled by product management.
- even releases would focus on stability, reliability, and performance, while
- odd releases would emphasize new features and potentially introduce breaking changes.

If you search through the forum history, you will find many discussions like this. But Mikrotik version numbers are just usable to distinguish releases, but have not much further meaning.
They introduce new feature from RC to the next RC release (including new bugs), new features in .1 releases and just turn RC builds overnight into official releases despite known bugs. All of this with release notes full of "fixed something with something" entries without further details.

We had MT officials here claiming several time it's to much effort for them to improve on this. So that's how it is and it will hardly change.

infabo · Tue Oct 15, 2024 6:38 pm

Pretty accurate summary. 👏

pe1chl · Tue Oct 15, 2024 7:37 pm

Nothing stops anyone from stopping at 7.12 - I do in fact, until 16MB can be replace or netinstall'ed.

On existing devices, yes. But when you buy new stuff, at some point it will come with a higher version and not allow downgrade.
Of course you should no longer buy devices that have only 16MB, it will be a dead-end.

K0NCTANT1N · Wed Oct 16, 2024 1:46 am

I bought and I have the same "dead end" (shipping Latvia)

markdutton · Wed Oct 16, 2024 5:29 am

I upgraded my Virtual router today (x86-64) from 7.14.1 to 7.16.1. All field routers are still on 7.14.1

Router came up fine, but no field routers could connect to the Capsman (legacy).

The capsman manager looked normal. It looked to be running. I stopped and restarted capsman. Rebooted router again. It simply would not take any connections.

Log had the following three lines repeating on remote devices.

10:08:16 caps,info CAP selected CAPsMAN DataCentre (::ffff:172.17.1.254:5246)
10:08:36 caps,info CAP connect to DataCentre (::ffff:172.17.1.254:5246) failed: timeout
10:08:36 caps,info CAP failed to join DataCentre (::ffff:172.17.1.254:5246)

I only had a 5 minute window so I had to downgrade the router. I went back to 7.15.3 and CAPs all came up immediately.

Is there any reason this should happen? I have very short outage Windows, so I need to have some sort of plan before I take it up again.

There is nothing in the log indicating an issue. It was like CAPSMAN was just not listening.

starlingus · Wed Oct 16, 2024 10:53 am

We had MT officials here claiming several time it's to much effort for them to improve on this. So that's how it is and it will hardly change.

I believe that, like any other company, Mikrotik is doing everything possible to achieve success. They are also targeting large enterprises with core network solutions, which makes reliable software essential. The introduction of the long-term release branch was a significant step forward, offering thoroughly tested and versions, verified by early-adopter customers of stable versions. As I mentioned before, I understand why v7 isn’t ready for the long-term release just yet. However, this doesn’t mean that efforts to cater to the user groups I previously described should be set aside. Using an odd/even versioning system, for example, could be one of many solutions to this challenge.

By the way, I really appreciate what Mikrotik is doing with WinBox. The development of v4, shaped by immediate user feedback through the forum, truly feels like it’s being built with the user in mind.

infabo · Wed Oct 16, 2024 11:23 am

Of course you should no longer buy devices that have only 16MB, it will be a dead-end.

As long as there is no smaller variant of the CAP AX, people will continue to buy the CAP AC. The fact that smaller designs are possible is demonstrated by competitors. You can install AX hardware comparable to the CAP AX in a 160mm diameter. This is proven by Unifi and TP-Link. TP-Link also offers wall-mounted access points, such as the EAP615-Wall, with dimensions that are even smaller than that (143 × 86 × 20 mm). No one likes to install a bulky device on their ceiling (228 x 48 mm). If at least the coverage were good - but according to many reports on the forum, that's also an issue.

pe1chl · Wed Oct 16, 2024 11:38 am

We had MT officials here claiming several time it's to much effort for them to improve on this. So that's how it is and it will hardly change.

I believe that, like any other company, Mikrotik is doing everything possible to achieve success. They are also targeting large enterprises with core network solutions, which makes reliable software essential. The introduction of the long-term release branch was a significant step forward, offering thoroughly tested and versions, verified by early-adopter customers of stable versions.

But that is not what the long-term version was! It would have been nice if that is what it was.
But it has regularly happened that on one day, version x.xx was marked "stable" and x.xx-1.4 was released in the "long-term" channel, where previously only x.xx-1.3 existed, including some backports of fixes in x.xx which had not been tested except in x.xxrc releases.
I would agree that a "stable" version which has been tested a lot should be marked long-term.
That at least allows us to "downgrade" to that version after an upgrade to a new "stable" turns out problematic.

Note that from version 7.17 it will no longer be possible to downgrade using the current methods, like uploading a previous version or switching to a partition that holds a previous version, unless you have physical access to the device at least once. You will only be allowed to downgrade to a version existing in a channel, and there is no lower version than "stable" for v7.
I have suggested that at least, when they do not want to maintain a long-term version, the channel would be renamed to previous-stable and used for that, but my input is not appreciated.

pe1chl · Wed Oct 16, 2024 11:41 am

Of course you should no longer buy devices that have only 16MB, it will be a dead-end.
As long as there is no smaller variant of the CAP AX, people will continue to buy the CAP AC. The fact that smaller designs are possible is demonstrated by competitors. You can install AX hardware comparable to the CAP AX in a 160mm diameter.

The performance of the antennas, especially the MIMO features, depends on the physical size of the device.
Yes you can make smaller devices that contain the electronics, but the antenna performance will necessarily be worse.
(assuming that the larger physical size is effectively used and is not just "a design" of a plastic case)

infabo · Wed Oct 16, 2024 11:56 am

look yourself: viewtopic.php?t=209550#p1087603

pe1chl · Wed Oct 16, 2024 12:08 pm

It seems the physical dimensions of the unit are quite well-used.
If the performance of the antennas is optimal for this size is less easy to derive from photos.
But I can assure you that when you have an antenna that performs optimally at some physical size, shrinking it will ALWAYS make it worse.

infabo · Wed Oct 16, 2024 12:43 pm

Even wifi7 ceiling APs from other vendors are smaller in diameter than CAP AX. When other vendors smallest devices are 220mm - how huge will Mikrotik go? 300mm CAP BE?

4L3xN3t · Wed Oct 16, 2024 12:48 pm

Not to be rude...but I think you all have gone a bit too much off topic.
Version numbering standards has nothing to do with this specific release and wifi performances depend on hardware, as you said too.
Sorry, but I think that when someone receives a notification for a new post in a topic about a new sw release should be able to come and see if the sw could cause an issue that could affect his devices.
It's ok one or two offroad posts but there are 17 of them now

infabo · Wed Oct 16, 2024 1:22 pm

The capsman manager looked normal. It looked to be running. I stopped and restarted capsman. Rebooted router again. It simply would not take any connections.

Log had the following three lines repeating on remote devices.

10:08:16 caps,info CAP selected CAPsMAN DataCentre (::ffff:172.17.1.254:5246)
10:08:36 caps,info CAP connect to DataCentre (::ffff:172.17.1.254:5246) failed: timeout
10:08:36 caps,info CAP failed to join DataCentre (::ffff:172.17.1.254:5246)

Why do you use ipv4 mapped ipv6 address? Is this also the case on 7.15.3 after downgrade? How can you restart capsman service independently from router?

nmt1900 · Wed Oct 16, 2024 2:07 pm

Capsman log is always showing IPv4 addresses of remote CAP's like that - even when IPv4 addresses are used everywhere and IPv6 is disabled. It has been like this since before v7

infabo · Wed Oct 16, 2024 2:33 pm

My logs show MAC addresses.

caps,info connected to foobar@DC:2C:xx:xx:xx:xx

Maybe wireless capsman specific.

nmt1900 · Wed Oct 16, 2024 2:53 pm

That depends if CAP has L2 or L3 connection. It can work both ways and it is visible in capsman Remote CAP section - connection is identified there either by IPv6 address, IPv4 address - or MAC address if it is L2 connection.

infabo · Wed Oct 16, 2024 3:23 pm

Sorry, but I think that when someone receives a notification for a new post in a topic

I do not get any notifications by email even for subscribed topics. Been a long time since I last received such an email. Either broken or disabled (globally?).

4L3xN3t · Wed Oct 16, 2024 3:29 pm

Sorry, but I think that when someone receives a notification for a new post in a topic
I do not get any notifications by email even for subscribed topics. Been a long time since I last received such an email. Either broken or disabled (globally?).

I receive notifications, but AFAIK it's configurable. The point is to keep clean and on purpose the post :)

Wed Oct 16, 2024 3:42 pm

[off topic]

I do not get any notifications by email even for subscribed topics. Been a long time since I last received such an email. Either broken or disabled (globally?).

I receive notifications, but AFAIK it's configurable. The point is to keep clean and on purpose the post

Notifications were a bit on/off the past months.
Apparently it's working again now.

[/off topic]

mustang1986 · Wed Oct 16, 2024 9:48 pm

Not good news....
I use 44 piece cap AC with qcom-ac (more VLAN and ~70 piece wifi client with 802.11r fast BSS transitions ( roaming), routeros 7.15.3 and 7.16rc1-5).
After 7-10 days runs out the cap's memory.
I am on 8+ days and RAM is declining indeed. I have not enabled graphing so I can't proof it. But 2 days ago it was 31 or 32MiB free-memory. Now it is down to 29.3MiB. Let's see where this is going.
Code: Select all
/system/resource/print 
                   uptime: 1w1d9m
                  version: 7.16 (stable)
               build-time: 2024-09-20 13:00:27
         factory-software: 6.44.6
              free-memory: 29.3MiB
             total-memory: 128.0MiB
                      cpu: ARM
                cpu-count: 4
            cpu-frequency: 448MHz
                 cpu-load: 1%
           free-hdd-space: 736.0KiB
          total-hdd-space: 16.0MiB
  write-sect-since-reboot: 1128
         write-sect-total: 32915
        architecture-name: arm
               board-name: cAP ac
                 platform: MikroTik

We wait for 7.17? :D

infabo · Wed Oct 16, 2024 10:26 pm

Unfortunately I upgraded to 7.16.1 the day it was released. So my post was on Oct 04, 2024 10:01 am where it reached 8d uptime. Uptime is now 5d and some hours. Based on that, cap ac uptime must have at least 14 days. It purely depends on the config, and my CAP AC is basically "reset-configuration caps-mode=yes". Assign some hostname (identity) and disables IP services except SSH. Done.

yhfung · Thu Oct 17, 2024 7:13 am

Wireguard and DNS static size over a certain limit causing an overflow problem

The CCR1009-7G-1C-PC (client) and CCR1009-7G-1C-1S+PC (server) installed with v7.16.1, connected to the Internet. Both routers are connected without any problem via wireguard. The server can ping 10.10.100.2 to the client. The client can ping 10.10.100.1 to the server as well.

I installed CN.rsc, LAN.rsc, gfslist.rsc in client from https://github.com/ruijzhan/chnroute.

/tool fetch url=https://raw.githubusercontent.com/ruijzhan/chnroute/master/CN.rsc
/tool fetch url=https://raw.githubusercontent.com/ruijzhan/chnroute/master/LAN.rsc
/tool fetch url=https://raw.githubusercontent.com/ruijzhan/chnroute/master/gfwlist.rsc

import file-name=CN.rsc
import file-name=LAN.rsc
:global dnsserver 10.10.100.1; import file-name=gfwlist.rsc

/ip/dns/set cache-size=20560KiB

The client could ping 10.10.100.1 without any problem. Also no problem were found by both "put [resolve google.com]" and "put [resolve google.com server=10.10.100.1]".

[admin@MikroTik] > put [resolve google.com server=10.10.100.1]
142.250.198.110

[admin@MikroTik] > put [resolve google.com]                   
142.250.198.110

[admin@MikroTik] > ping 10.10.100.1
  SEQ HOST                                     SIZE TTL TIME       STATUS                
    0 10.10.100.1                                56  64 37ms873us 
    1 10.10.100.1                                56  64 38ms309us 
    2 10.10.100.1                                56  64 38ms761us 
    3 10.10.100.1                                56  64 38ms47us  
    sent=4 received=4 packet-loss=0% min-rtt=37ms873us avg-rtt=38ms247us 
   max-rtt=38ms761us

If the client router was rebooted, the three commands failed.

[admin@MikroTik] > put [resolve google.com server=10.10.100.1]                  
failure: dns name exists, but no appropriate record

[admin@MikroTik] > put [resolve google.com]                                     
failure: dns name exists, but no appropriate record

[admin@MikroTik] > ping 10.10.100.1                                             
  SEQ HOST                                     SIZE TTL TIME       STATUS                
    0                                                              126 (No error infor...
    0 10.10.100.3                                84  64 469us      host unreachable      
    1                                                              126 (No error infor...
    1 10.10.100.3                                84  64 461us      host unreachable      
    2                                                              126 (No error infor...
    2 10.10.100.3                                84  64 456us      host unreachable      
    sent=3 received=0 packet-loss=100%

Which command gave problem? Finally I found the command "ping 10.10.100.1". If it succeeded, other two succeeded. So I reduced the file size of gfwlast.rsc, starting from the fourth line ":do { add forward-to=$dnsserver type=FWD address-list=gfw_list regexp=".*000webhost\\.com\$" } on-error={}", the number of lines could be smaller.

:global dnsserver
/ip dns static remove [/ip dns static find forward-to=$dnsserver ]
/ip dns static
:do { add forward-to=$dnsserver type=FWD address-list=gfw_list regexp=".*000webhost\\.com\$" } on-error={}
:do { add forward-to=$dnsserver type=FWD address-list=gfw_list regexp=".*030buy\\.com\$" } on-error={}
:do { add forward-to=$dnsserver type=FWD address-list=gfw_list regexp=".*0rz\\.tw\$" } on-error={}
.
.
.
/ip dns cache flush

If the original gfwlist.rsc were copied to two files, they were namely gfwlist_v.rsc (fail) and gfwlist_j.rsc (pass). They were determined by the binary method. The line number of gfwlist_v.rsc with ":do" statement was 5348 (minus the first three line and the last one, i.e. 5352 - 4 = 5348). Similarly the line number of gfwlist_j.rsc with ":do" statement was 5242.

This method can determine the triggered point when the problem was found.

dhoppe · Thu Oct 17, 2024 10:02 am

I was able to upgrade my MikroTik RB5009 and hAP ax^2 without any problems, but have some issues with my CRS310.

I was able to upgrade / reboot the packages without any issues, but when I did the upgrade of the firmware the switch got stuck during the reboot and the fan was running at 100%. I waited for ~30 minutes and pulled the plug. After powering on the switch it was running the the current version (packages and firmware). But when I do a shutdown the switch will get stuck and the fan is running again at 100%.

Is this a bug? Any idea how I can solve this behaviour?

elbob2002 · Thu Oct 17, 2024 10:30 am

Take a backup and run netinstall

bratislav · Thu Oct 17, 2024 11:30 am

Even wifi7 ceiling APs from other vendors are smaller in diameter than CAP AX. When other vendors smallest devices are 220mm - how huge will Mikrotik go? 300mm CAP BE?

I am sure Mikrotik could easily make something smaller, hAP ax2 is much smaller than cAP ax for example and it has 5 ethernet ports so they could probably just remove 4 of those ports and DC in connector (leaving only PoE), stick it in a cAP case and that's it...
The question is why, it isn't like there is a lack of space on the ceiling and performance would most likely suffer...

ludvik · Thu Oct 17, 2024 12:18 pm

RB1100AHx2

Ether 11 do not show autonegotiation, rate, duplex, advertising and link partner advertising on Interface status page. In SNMP it is wrong too.

4L3xN3t · Thu Oct 17, 2024 12:37 pm

I was able to upgrade my MikroTik RB5009 and hAP ax^2 without any problems, but have some issues with my CRS310.

I was able to upgrade / reboot the packages without any issues, but when I did the upgrade of the firmware the switch got stuck during the reboot and the fan was running at 100%. I waited for ~30 minutes and pulled the plug. After powering on the switch it was running the the current version (packages and firmware). But when I do a shutdown the switch will get stuck and the fan is running again at 100%.

Is this a bug? Any idea how I can solve this behaviour?

have you tried discovering via winbox if it's detectable? maybe you can access it using MAC address instead of IP.
I had similiar issues with a CRS309 (it has no fans, but was stuck on boot and had to use netinstall) and it was because the "detect internet" function turned on

mszru · Thu Oct 17, 2024 4:01 pm

OpenVPN server generates debug messages with "unknown" topics. I'm not sure if this is the 7.16.1 issue, as I haven't run the OpenVPN server on previous versions of RouterOS.

2024-10-17 OpenVPN log.png

erlinden · Thu Oct 17, 2024 4:03 pm

OpenVPN server generates debug messages with "unknown" topics. I'm not sure if this is the 7.16.1 issue, as I haven't run the OpenVPN server on previous versions of RouterOS.

Show us the...config:

/export file=anynameyoulike

Remove serial and any other private info and post ebtween code tags by using the </> button.

Paternot · Thu Oct 17, 2024 5:06 pm

I'm running one RB5009 with 7.16.1
As of now, everything runs fine - IPSec, Wireguard, BGP...
But I just upgraded my desktop mainboard, and I think I found a cosmetic bug. This new board has 2,5Gbps ethernet, so I used it. It works perfectly well (at least looks like it does), but on the "link partner advertising" section I can't see 2,5Gbps. Shouldn't appear there?

I am connected at 2,5Gbps. The router thinks so. The client thinks so. But the status section has this little quirk. Is it normal, with some network chipsets? The mainboard is one Asus TUF gaming B550M-PLUS. The network card is one Realtek RTL8125 (revision 05).

CGGXANNX · Thu Oct 17, 2024 5:17 pm

on the "link partner advertising" section I can't see 2,5Gbps. Shouldn't appear there?.

On my RB5009 ether1 is connected to a 2.5GbE switch and it also doesn't show the 2.5G rate under Link Partner Advertising. But the negotiated rate is still 2.5Gbps and there is no problem reaching that speed in real transfer tests.

ether1.png

mszru · Thu Oct 17, 2024 7:10 pm

OpenVPN server generates debug messages with "unknown" topics. I'm not sure if this is the 7.16.1 issue, as I haven't run the OpenVPN server on previous versions of RouterOS.
Show us the...config:

Nothing special in the config. Here is the ovpn part of it:

/interface/ovpn-server/server/print                                                            
                     enabled: yes
                        port: 1194
                        mode: ip
                    protocol: udp
                     netmask: 24
                 mac-address: AB:CD:EF:70:3F:33
                     max-mtu: 1500
           keepalive-timeout: 60
             default-profile: default-encryption
                 certificate: vpn-server
  require-client-certificate: yes
                 tls-version: any
                        auth: sha1,sha256
                      cipher: aes256-cbc,aes256-gcm
                   reneg-sec: 3600
            redirect-gateway: disabled
                 push-routes: 
             enable-tun-ipv6: no
             tun-server-ipv6: ::

/ppp/secret/print proplist=name,service,profile,local-address,remote-address where service=any 
Columns: NAME, SERVICE, PROFILE, LOCAL-ADDRESS, REMOTE-ADDRESS
 # NAME          SERVICE  PROFILE             LOCAL-ADDRESS  REMOTE-ADDRESS
10 client1       any      default-encryption  10.0.0.1       10.0.0.2

I have exported .ovpn file from WinBox and imported it in the OpenVPN app on the iPhone.

client
dev tun
remote my.domain.name 1194 udp
tun-mtu 1500
tls-client
nobind
user nobody
group nogroup
ping 15
ping-restart 45
persist-tun
persist-key
mute-replay-warnings
verb 3
cipher AES-256-GCM
auth none
pull
auth-user-pass
connect-retry 1
reneg-sec 3600
explicit-exit-notify 1
remote-cert-tls server
<ca>
-----BEGIN CERTIFICATE-----
. . .
-----END CERTIFICATE-----
</ca>
<cert>
-----BEGIN CERTIFICATE-----
. . .
-----END CERTIFICATE-----
</cert>
<key>
-----BEGIN ENCRYPTED PRIVATE KEY-----
. . .
-----END ENCRYPTED PRIVATE KEY-----
</key>

The connection was successfully established from the cellular network.

/ppp/active/print where service=ovpn 
Columns: NAME, SERVICE, CALLER-ID, ADDRESS, UPTIME, ENCODING
# NAME          SERVICE  CALLER-ID       ADDRESS   UPTIME  ENCODING                 
4 client1       ovpn     #.#.140.253     10.0.0.2  46s     AES-256-GCM/[null-digest]

I had a logging rule that suppressed ovpn+info output, so only errors were printed. I've disabled it and got complete log:
Part 1 - Connection establishment.

 17:17:43 ovpn,info connection established from #.#.140.253, port: 27481 to #.#.176.53
 17:17:43 ovpn,info #.#.140.253: using encoding - AES-256-GCM/[null-digest]
 17:17:43 ovpn,info,account client1 logged in, 10.0.0.2 from #.#.140.253
 17:17:43 ovpn,info <ovpn-client1>: connected

Part 2 - iPhone's display went off and after a while the client was disconnected and dynamic ovpn-client interface removed.

 17:19:43 ovpn,info <#.#.140.253>: disconnected <nothing received for a while>
 17:19:43 ovpn,info <ovpn-client1>: terminating... - nothing received for a while
 17:19:44 ovpn,info,account client1 logged out, 121 2800 6608 50 118 from #.#.140.253
 17:19:44 ovpn,info <ovpn-client1>: disconnected

Part 3 - The moment I touched the phone's display, a new connection was established, followed by the error I reported earlier... and disconnection. After a while a connection was re-established.

 17:19:50 ovpn,info connection established from #.#.140.253, port: 27481 to #.#.176.53
 17:19:50 ovpn,debug,error,65472,3760,1520 recvd P_DATA packet, dropping
 17:19:50 ovpn,info <#.#.140.253>: disconnected <bad packet received>
 17:19:53 ovpn,info connection established from #.#.140.253, port: 27229 to #.#.176.53
 17:19:53 ovpn,info #.#.140.253: using encoding - AES-256-GCM/[null-digest]
 17:19:53 ovpn,info,account client1 logged in, 10.0.0.2 from #.#.140.253
 17:19:53 ovpn,info <ovpn-client1>: connected

There are two problems, as far as I can tell:

The "auth" parameter in the ovpn server settings is not properly exported.
"recvd P_DATA packet, dropping" error is printed with 3 "unknown" topics in WinBox, while in Terminal we can see topic numbers: 65472,3760,1520.

mszru · Thu Oct 17, 2024 7:43 pm

There are two problems, as far as I can tell:

The "auth" parameter in the ovpn server settings is not properly exported.

"recvd P_DATA packet, dropping" error is printed with 3 "unknown" topics in WinBox, while in Terminal we can see topic numbers: 65472,3760,1520.

The first one is not a bug... I have already found the answer.

jlgonzalez · Fri Oct 18, 2024 10:29 am

Coming from v7.13.5, I have upgraded my router to this version. I'm having some issues when executing a certain script from a scheduler. An error is printed in the log, but the script is executed as expected. The error I'm talking about is:

script,error executing script from scheduler failed, please check it manually

This error has been reported since 7.15 version. After reading different posts from Reddit and this forum, I've come to the conclusion that the problem comes from having a :return statement in my code. A little snippet:

:if ($VrrpIntPriorityIsModified) do={
	:log info "[vrrp-change-priority]: $LanVrrpLowestPriority -> $LanVrrpHighestPriority.";
        :return "";
 }
 
 # Continue doing other things.

I don't understand why Mikrotik team considered that returning from a function must be treated as an error, since that is something commonly used in all programming languages to stop execution of the program after certain conditions are met. Since Mikrotik scripting does not support an exit function, (and using :error doesn't have sense in this context), I don't know what to do to avoid rewriting my whole set of scripts if I want to remove those annoying messages in logging.

I think this issue should be fixed, as many people have reported this problem. I'm open to talk and debate about this topic to propose a clean solution.

Thanks in advance!

pe1chl · Fri Oct 18, 2024 10:44 am

Display of the routing table (IP->Routes) in Winbox (3.41) is quite unstable.
After route changes have occurred (by BGP), the displayed routes often are just plain wrong.
Nexthop is wrong, Immediate gateway is wrong, AS path is wrong.
When winbox is closed and re-opened, it is correct again for that moment. Just closing the IP->Routes window and re-opening it does not fix it. In fact, opening the IP->Routes window sometimes crashes winbox.

baragoon · Fri Oct 18, 2024 11:11 am

Hi,
Routing filters are partialy broken on 7.16+
Instead of expected pref=201 it set to 1, looks like + or - just ignored.
Works fine on 7.15.3

https://help.mikrotik.com/docs/spaces/R ... nd+Filters* typo (set bgp-local-ref instead of set bgp-local-pref)

Command: set
The command is used to set a new value to writeable properties. Value can be set from other readable properties of matching types. For numeric properties, it is possible to prefix the value with +/- which will increment or decrement the current property value by a given amount. For example, "set bgp-local-ref +1" will increment current LOCAL_PREF by one, or extract value from other readable num property, "set distance +ospf-ext-metric"

set bgp-local-pref 200
if (dst == 206.168.193.0/24) {set bgp-local-pref +1; accept}
if (dst !=0.0.0.0/0 && dst-len in 8-24) {accept}

> routing/route/print detail where dst-address =206.168.193.0/24 
Flags: X - disabled, F - filtered, U - unreachable, A - active; 
c - connect, s - static, r - rip, b - bgp, o - ospf, i - isis, d - dhcp, v - vpn, m - modem, a - ldp-address, l - ldp-m
apping, g - slaac, y - bgp-mpls-vpn; 
H - hw-offloaded; + - ecmp, B - blackhole 
 Ab   afi=ip4 contribution=active dst-address=206.168.193.0/24 routing-table=main gateway=100.66.125.17 
       immediate-gw=100.66.125.17%BGP.Ex-KAN distance=20 scope=40 target-scope=10 belongs-to="bgp-IP-100.66.127.254" 
       bgp.peer-cache-id=*2800003 .as-path="400671" .large-communities=400671:1:100 .local-pref=1 .origin=igp 
       debug.fwp-ptr=0x20443240

pe1chl · Fri Oct 18, 2024 5:40 pm

@baragoon do you also notice during your debugging that the routes displayed are sometimes completely different from actually installed routes?
Or do you only use commandmode and no winbox? (probably it does not happen in commandmode)

maxfava · Fri Oct 18, 2024 7:38 pm

Hi,
after upgrading to 7.16.1 from 7.15.3
I noted that Framed-Route passed from radius is lost after a while.
I have PPPOE server and L2TP server and noted that Framed-Route passed from radius is taken after I kick the ppp active connection, but, after one day some of them dynamic Framed-Route is no longer in the route table even if the ppp connection is active and never disconnected.

I have not debugget yet so I cannot say is related to a specific event, since is too early, the issue happened yesterday for first time and today for second time on 3 differnt accounts.

antiqued4 · Sat Oct 19, 2024 7:15 pm

The issue described in viewtopic.php?t=207022 has become much worse in 7.16
(BGP sessions close when another session closes)
In previous versions it appeared to affect only BGP sessions with the same local IP, now it sometimes affects ALL sessions...
When a peer on L2TP/IPsec disconnects because their public IP has changed and they re-establish the L2TP/IPsec session, I have observed several times that all BGP sessions (15 total) go to Idle state and have to re-connect.
Nobody else noticed that?

Same problem here, CCR2216 when a session goes down they all go down together and go up again, the same thing happens if OSPF goes down, the BGP sessions go down.
Version 7.15.3 I didn't see this problem

nerxu · Sun Oct 20, 2024 12:29 pm

tested on CCR2116
V7.16 and 7.16.1
Winbox v4 and v3

when open
tools-graphing Interface/Resource....

Winbox crash/off
tested MacOS / Windows
--------
V6.49.X - EVERYTHING OK

pe1chl · Sun Oct 20, 2024 12:33 pm

The issue described in viewtopic.php?t=207022 has become much worse in 7.16
(BGP sessions close when another session closes)
In previous versions it appeared to affect only BGP sessions with the same local IP, now it sometimes affects ALL sessions...
When a peer on L2TP/IPsec disconnects because their public IP has changed and they re-establish the L2TP/IPsec session, I have observed several times that all BGP sessions (15 total) go to Idle state and have to re-connect.
Nobody else noticed that?
Same problem here, CCR2216 when a session goes down they all go down together and go up again, the same thing happens if OSPF goes down, the BGP sessions go down.
Version 7.15.3 I didn't see this problem

I could work around it by setting input.affinity=main output.affinity=input on the template (and checking that it indeed was inherited to the connections).
Now it is more like in previous versions: sometimes an unrelated session closes, but no longer all together.
But I have not rebooted after that, it could be that just changing that configuration is what made the difference, and not the exact configuration it was changed to.

baragoon · Mon Oct 21, 2024 9:47 am

@pe1chl tried via ssh and winbox terminal - the same, 7.16+ (CHR) affected

antiqued4 · Mon Oct 21, 2024 5:44 pm

Same problem here, CCR2216 when a session goes down they all go down together and go up again, the same thing happens if OSPF goes down, the BGP sessions go down.
Version 7.15.3 I didn't see this problem
I could work around it by setting input.affinity=main output.affinity=input on the template (and checking that it indeed was inherited to the connections).
Now it is more like in previous versions: sometimes an unrelated session closes, but no longer all together.
But I have not rebooted after that, it could be that just changing that configuration is what made the difference, and not the exact configuration it was changed to.

I talked to Mikrotik support, they told me to try, in the sessions that receive fullrouting and where fullrouting is sent, to use it alone. Other sessions use input.affinity=main output.affinity=input.
I haven't tested it yet, I've disabled all fullrouting and so far without any drops, I've also made some adjustments to OSPF. I will reactivate fullrouting and test if the sessions still crash.

pe1chl · Mon Oct 21, 2024 7:06 pm

My routers do not receive internet routing tables but only a couple of local networks (company networks routed over VPN), but the problem is the same. It for sure is not related to large routing tables.
I have support ticket SUP-159987 open for it since 23/Jul/24 but there is no real progress...

rextended · Tue Oct 22, 2024 12:54 pm

Netwatch bug: On script $ip provide the wrong IP.
For example, if www.example.com is used, instead of 93.184.215.14
is returned 249018461, that is 0E D7 B8 5D = 14.215.184.93, the IP on reversed bytes position...

test code

/tool netwatch
add disabled=no host=www.example.com type=dns \
    up-script=":log info \"\$ip ==>> \$(0.0.0.0 + \$ip)\"" \
    down-script=":log info \"\$ip ==>> \$(0.0.0.0 + \$ip)\""

pe1chl · Tue Oct 22, 2024 5:41 pm

I did not test that yet, but is it really so that in the $ip variable a 32-bit numeric value is returned instead of a dotted quad string?
And you could convert that to a string by adding 0.0.0.0 to it?
I would expect a function like inet_ntoa() to be required for that conversion.
The IP address will normally be in "network byte order" and the CPU architecture can have the reverse byte order, causing such bugs.
(which may not be present on a different architecture)

DjM · Tue Oct 22, 2024 11:09 pm

For the netwatch bug I have already opened SUP-168169.

rextended · Wed Oct 23, 2024 10:16 am

And you could convert that to a string by adding 0.0.0.0 to it?

I always done on that way, ":toip" do not convert numbert to IP...
viewtopic.php?t=177551#p953746

pe1chl · Wed Oct 23, 2024 11:03 am

Well, it is not surprising (to me) that it does not work. Maybe there has been a strange workaround in the network code that made it work before and is now removed?
It is completely normal in Linux that the byte order is "wrong" when you look at the bare 32-bit address that way, at least when you have a little-endian machine like x86/amd64.
(of course one reason to prefer big-endian processors in routers is that this problem does not occur there)

bratislav · Wed Oct 23, 2024 11:13 am

on the "link partner advertising" section I can't see 2,5Gbps. Shouldn't appear there?.
On my RB5009 ether1 is connected to a 2.5GbE switch and it also doesn't show the 2.5G rate under Link Partner Advertising. But the negotiated rate is still 2.5Gbps and there is no problem reaching that speed in real transfer tests.

Would be interesting to see what

interface/ethernet/monitor numbers=0 duration=1

says, maybe WinBox issue...

bratislav · Wed Oct 23, 2024 12:40 pm

There is still a bag in MSTI status data, if we change MSTP bridge priority from 0x7000 to 0x6000 for example:

interface/bridge/print proplist=name,protocol-mode,priority 
Flags: X - disabled, R - running 
 0 R name="bridge" protocol-mode=mstp priority=0x6000

MSTI status data for dynamic instance 0 displays old value:

interface/bridge/msti/print where identifier=0
Flags: D - DYNAMIC
Columns: BRIDGE, IDENTIFIER, PRIORITY, VLAN-MAPPING
#   BRIDGE  IDENTIFIER  PRIORITY  VLAN-MAPPING
0 D bridge           0  0x7000    1-6

Although bridge does advertise correct value as can be seen on peer switches:

interface/bridge/monitor numbers=0
                     ;;; defconf
                  state: enabled
    current-mac-address: 48:A9:8A:7C:63:30
            root-bridge: no
         root-bridge-id: 0x6000...

The only way to change displayed value is to reboot the switch, which is not optimal since it will obviously cause traffic disruption.
This bug exists from ROS 7.14 as far as I can tell.

Milecus · Wed Oct 23, 2024 2:10 pm

@bratislav: interface/ethernet/monitor numbers=0 duration=1
name: ether1
status: link-ok
auto-negotiation: done
rate: 2.5Gbps
full-duplex: yes
tx-flow-control: no
rx-flow-control: no
supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-half,1G-baseT-full,
2.5G-baseT
advertising: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-half,1G-baseT-full,
2.5G-baseT
link-partner-advertising: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-full

The problem is not in WinBox)

szalkerous · Thu Oct 24, 2024 12:03 am

tried all ways imaginable - couldn't get it to work besides downgrading...
revisited the issue - well - _before_ upgrading one has to remove the parameter "vrf-interface" from any static vrf routes that do typically carry routing-table=vrf and vrf-interface=NAME which obviously breaks the routes after v7.16 upgrade.
also dialer-interfaces that enable "add default route" do not add a working default route into its assigned vrf, one has to add a route like:
Code: Select all
add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=PPPoE-VRF@vrfO routing-table=vrfO scope=30 suppress-hw-offload=no target-scope=10
it's a pitty an upgrade breaks the config this way.

I too had my VRF setup break completely moving from 7.15.2 to 7.16.1

Glad these posts were here to give me an idea of where to look.

Sadly not sure where mine is broken since I already had scripts to build the routes for my DHCP "add default route" problems.

I reviewed all the literature at https://help.mikrotik.com/docs/spaces/R ... infirewall and compared it to the information at https://wiki.mikrotik.com/Manual:PCC where I originally based my PCC/VRF setup for the two DHCP ISP WANs from. I don't see where I went wrong in my config.

Looks like this might be the first time I ever downgrade. Very disappointing.

bratislav · Thu Oct 24, 2024 11:17 am

@bratislav: interface/ethernet/monitor numbers=0 duration=1
name: ether1
status: link-ok
auto-negotiation: done
rate: 2.5Gbps
full-duplex: yes
tx-flow-control: no
rx-flow-control: no
supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-half,1G-baseT-full,
2.5G-baseT
advertising: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-half,1G-baseT-full,
2.5G-baseT
link-partner-advertising: 10M-baseT-half,10M-baseT-full,100M-baseT-half,
100M-baseT-full,1G-baseT-full

The problem is not in WinBox)

I checked couple of mines too, and some do not give anything for partner on SFP+ ports, but nevertheless connect at 10Gbps...

                               name: sfp-sfpplus1
                             status: link-ok
                   auto-negotiation: done
                               rate: 10Gbps
                        full-duplex: yes
                    tx-flow-control: no
                    rx-flow-control: no
                          supported: 10M-baseT-half,10M-baseT-full,100M-baseT-half,100M-baseT-full,1G-baseT-half,1G-baseT-full,1G-baseX,2.5G-baseT,2.5G-baseX,
                                     5G-baseT,10G-baseT,10G-baseSR-LR,10G-baseCR
                      sfp-supported: 1G-baseT-full,1G-baseX,2.5G-baseT,2.5G-baseX,5G-baseT,10G-baseCR,25G-baseCR
                        advertising: 1G-baseT-full,1G-baseX,2.5G-baseT,2.5G-baseX,5G-baseT,10G-baseCR
           link-partner-advertising:

sirbryan · Fri Oct 25, 2024 6:54 pm

My routers do not receive internet routing tables but only a couple of local networks (company networks routed over VPN), but the problem is the same. It for sure is not related to large routing tables.

So did you change everything to input=main, output=input?

After upgrading my borders and core (5 2116's in total) to 7.16, I get random problems with BGP sessions disconnecting and reconnecting. Eventually the router no longer prints routes, and I have to reboot. Happened yesterday after shutting down some external BGP sessions. Unfortunately I forgot to get a supout file.

I also had one case where the router showed a route in the routing table as going via one interface, but traffic was continuing to traverse the old interface. I tried shutting down the old interface, but instead of rerouting traffic (as expected), traffic stopped flowing and the router started acting really sluggish.

The majority of my BGP sessions are using "alone" (IPV4/IPV6 full-table peers or internal "most-table" peers, the rest that weren't explicitly assigned were at defaults).

pe1chl · Fri Oct 25, 2024 11:05 pm

Yes there sure are terrible issues with BGP in current releases, but it is difficult to debug them.
This week I had a case where one BGP peer connects, routes are received OK (remote is RouterOS v6 so none of that stupid "routes are sent only after one keepalive timer" rubbish), then after "holdtime" the session disconnects with a "HoldTimer expired" and re-connects, this in an endless loop.
After disabling/re-enabling the BGP connection it works again as normal.
Sent a supout to support, they cannot see anything wrong.

Also, I regularly see things that are obviously wrong in the routing table. Wrong route selected, wrong AS-Path, that kind of thing.
But that seems related to the winbox connection. When I close winbox and re-open it, the routing table looks different than before, and often more correct. This is especially apparent after a change in routes has occurred, e.g. I disabled a BGP connection but routes related to it still are shown active in the routing table.

Then I have problems with the limit on the number of alternative routes stored. But that seems to always have been then case, only it became apparent after topology changes in my network. Sometimes "prefix count" in a BGP session remains at zero even though the prefixes are being received, only because there are other paths where the same prefixes are received. However, those paths can fail and then there is no route available, although it should have been received and stored before.
There appears to be a hard-coded limit on number of alternative routes stored, I would want to increase it.

buset1974 · Mon Oct 28, 2024 11:50 am

Yes there sure are terrible issues with BGP in current releases, but it is difficult to debug them.
This week I had a case where one BGP peer connects, routes are received OK (remote is RouterOS v6 so none of that stupid "routes are sent only after one keepalive timer" rubbish), then after "holdtime" the session disconnects with a "HoldTimer expired" and re-connects, this in an endless loop.
After disabling/re-enabling the BGP connection it works again as normal.
Sent a supout to support, they cannot see anything wrong.

Also, I regularly see things that are obviously wrong in the routing table. Wrong route selected, wrong AS-Path, that kind of thing.
But that seems related to the winbox connection. When I close winbox and re-open it, the routing table looks different than before, and often more correct. This is especially apparent after a change in routes has occurred, e.g. I disabled a BGP connection but routes related to it still are shown active in the routing table.

Then I have problems with the limit on the number of alternative routes stored. But that seems to always have been then case, only it became apparent after topology changes in my network. Sometimes "prefix count" in a BGP session remains at zero even though the prefixes are being received, only because there are other paths where the same prefixes are received. However, those paths can fail and then there is no route available, although it should have been received and stored before.
There appears to be a hard-coded limit on number of alternative routes stored, I would want to increase it.

Yes bgp loaded with large routes can make the whole router get stuck (freeze) , even supout failed to be created, so its hard to explain this situation with MT, happen in 7.
16.1 and ticket has been created and still no answer.

derdeagle · Mon Oct 28, 2024 12:26 pm

After updating to 7.16.1 I am observing a strange behavior regarding the default route coming from a DHCP lease. I at least think it could have something to do with ROS 7.16.1 as there is no indication something else in my environment changed.
I updated on October 22. This first happened on October 25 and again on October 28 - in those two data points it is always three days later. Until now I only tried disabling and enabling the /ip/dhcp-client entry which immediately solved the problem as the default route was in the output of the routing table.

/routing/route/print where dst-address="0.0.0.0/0"
/ip/dhcp-client/disable numbers=0
/ip/dhcp-client/enable numbers=0
/routing/route/print where dst-address="0.0.0.0/0"

I have not yet tried a release or renew of the lease.

When searching the forum I couldn't find any relating post to this. Is it nevertheless something known or that could be investigated?

nmt1900 · Mon Oct 28, 2024 1:15 pm

I had a similar situation here with IPv6. It was first time since update for PPPoE WAN connection to reconnect and after this DHCPv6 client had no prefix. Release/renew commands did not help but disabling and re-enabling DHCPv6 client brought prefix back immediately.

pe1chl · Mon Oct 28, 2024 2:25 pm

Yes bgp loaded with large routes can make the whole router get stuck (freeze) , even supout failed to be created, so its hard to explain this situation with MT, happen in 7.
16.1 and ticket has been created and still no answer.

To be clear: my issues have nothing to do with large route tables. They do occur in a private network with only 22 prefixes and a partial mesh between 8 routers.

szpeter80 · Wed Oct 30, 2024 8:47 pm

I experience on RB5009 (RouterOS 7.16.1) "out of memory condition was detected" errors, then the router reboots. It looks like something slowly leaks memory, about 300Mbyte/24h.
I have a CAP AX (cAPGi-5HaxD2HaxD) with the same software and it does not happen (mem use is about 350M constant).

How can i check the process tree to see what is taking up memory ?

bratislav · Thu Oct 31, 2024 3:05 pm

I'm afraid this is not fixed yet.
cAP AC, reboot after 2d22h due to kernel failure.
PRTG monitoring shows increasing memory usage until reboot happens.

Back to daily scheduled reboot for now 8)

I already have an open ticket, will send autosupout there.
Not good news....
I use 44 piece cap AC with qcom-ac (more VLAN and ~70 piece wifi client with 802.11r fast BSS transitions ( roaming), routeros 7.15.3 and 7.16rc1-5).
After 7-10 days runs out the cap's memory.

I switched all our ac devices (cap ac, hap ac) to a regular wireless some times ago after months of kernel and OOM failures and reboots.
I have tried all recent ROS versions, (up to and including 7.16.1) with one of ours hAP ac2 and wifi-qcom-ac and there is maybe a slight improvement but nothing to be called stable (kernel failure do inevitably occur after couple of days) so I will continue to use old wireless package for production for now...

pe1chl · Thu Oct 31, 2024 3:18 pm

How can i check the process tree to see what is taking up memory ?

Unfortunately there is no user-level tool to show that (like there is for processor usage).
MikroTik claim that they can see it in a supout.rif. Of course you can upload a supout.rif to your account at mikrotik.com to show the content of a supout.rif so you could try that.
Or else you could make a support ticket and include the supout.rif and hope for a quick response.
That would have the advantage that they become aware of the failure scenario and can fix it in the software.

Louis2 · Thu Oct 31, 2024 9:16 pm

Hello,

I own a CRS317 and this release seams to have some CRS317 related fixes. However in the change log also the remark:
CRS317 devices ("/system routerboard upgrade" required)! What is that!?

So can any body explain:
- what is mend by ^routerboard upgrade^ and
- what exact procedure I should follow during the upgrade process not to lose / kill my CRS317

Thanks,

Louis

Thu Oct 31, 2024 10:48 pm

Winbox
System routerboard upgrade.
Cli: same path

Simple as that.

Louis2 · Thu Oct 31, 2024 11:22 pm

Yep I found a description

https://help.mikrotik.com/docs/spaces/R ... stallation

No problem, I never did that, and I was perhaps a bit to afraid, since I one's lost a CRS317 perhaps due to a unsuccessful update

infabo · Wed Nov 06, 2024 10:37 am

What is this kind of bug? "too strong signal, signal strength 159". Should I report this?

2024-11-06_09-38.png

lutze · Wed Nov 06, 2024 3:05 pm

Hello,

I own a CRS317 and this release seams to have some CRS317 related fixes. However in the change log also the remark:
CRS317 devices ("/system routerboard upgrade" required)! What is that!?

So can any body explain:
- what is mend by ^routerboard upgrade^ and
- what exact procedure I should follow during the upgrade process not to lose / kill my CRS317

Thanks,

Louis

Mikrotik devices, just like many others, have actually two pieces of software: firmware (think "BIOS") and the operating system itself. So what this remark says is, that you have to update the "BIOS" too. So in Winbox, go to System -> Routerboard, click the Upgrade button and then reboot. (You can update the firmware only to the one matching the running system; this is a bit annoying, that when you want update both operating system and firmware, you have to reboot twice).

infabo · Wed Nov 06, 2024 6:06 pm

Why do I see positive signal values in registration table for one client only?

2024-11-06_17-06.png

itimo01 · Thu Nov 07, 2024 8:54 am

Why do I see positive signal values in registration table for one client only?

2024-11-06_17-06.png

Damn thats some superb wifi ;)

AdrianR · Thu Nov 07, 2024 9:17 am

Why do I see positive signal values in registration table for one client only?

2024-11-06_17-06.png
Damn thats some superb wifi ;)

hmmm... we probably have here the first clues about subspace communications?. =)
Next stop, Alpha Centauri?

nmt1900 · Thu Nov 07, 2024 11:14 am

Why do I see positive signal values in registration table for one client only?

Maybe the value is truncated and it is -104, which is not very realistic either but still more believable.

infabo · Thu Nov 07, 2024 11:15 am

No idea what it is. Sometimes it shows "-74" or some realistic value. But most of the time it is a positive value. It goes up to 149 I have seen. Today it shows strength in interval of -83...-80. As it is realistic. But I did not reboot capsman nor did I reboot cap and not even the client was rebooted. It just "healed itself" somehow. I mean, it did not heal itself actually. It must be some kind of "edge-case" that is not handled correctly - not appearing today.

nmt1900 · Thu Nov 07, 2024 2:27 pm

Could it be Winbox 4 related issue with displaying these values? Can you check them with Winbox 3 or some other way?

itimo01 · Thu Nov 07, 2024 2:30 pm

Could it be Winbox 4 related issue with displaying these values? Can you check them with Winbox 3 or some other way?

The log entry also states +159

ToTheFull · Thu Nov 07, 2024 2:39 pm

No idea what it is. Sometimes it shows "-74" or some realistic value. But most of the time it is a positive value. It goes up to 149 I have seen. Today it shows strength in interval of -83...-80. As it is realistic. But I did not reboot capsman nor did I reboot cap and not even the client was rebooted. It just "healed itself" somehow. I mean, it did not heal itself actually. It must be some kind of "edge-case" that is not handled correctly - not appearing today.

I had this nonesense a while back. can't recall what version now (7.15/7.16 ish), but it isn't specific to wAP ax I don't thnk!
I don't have it now I am on 7.17beta4

MrRobotdev · Sat Nov 09, 2024 7:25 pm

All my static leases for other Mikrotik devices got messed up after update to 7.16. Switches and APs .Looks like the MAC on the bridges got reset somehow.
Read again the whole patch note and could not find anything about this. Then again, I just woke up so I could be a bit slow.
If you update remotely and you have lots of other MIkrotik devices behind the router, double check after the update that the leases are as you expect.

Same here, Completely a mess after the update, mostly with the new AX devices. The old HAPAC2 have not been affected. By luck? i dont know...

The strange thing here is that despite i tried to allocate a static IP to the AP from the RB4011 router, this was not possible with the first 2-3 trials... anyway now seems ok but you never know after a reboot...

Eventually, it seems that in these 2 HAPAX2 that the problem occurred, i had not set admin MAC address like the other devices. I do not know what does it mean but i set them. Hope that helps me. Until now the static IP survives the reboot.

bratislav · Tue Nov 12, 2024 10:54 am

Eventually, it seems that in these 2 HAPAX2 that the problem occurred, i had not set admin MAC address like the other devices. I do not know what does it mean but i set them. Hope that helps me. Until now the static IP survives the reboot.

You should always set admin MAC address on the bridge (best practice is to use MAC of the first ethernet interface that is part of the bridge), otherwise during the boot process, depending on interface statuses (up, down ...), bridge could get MAC other than the one that is used for DHCP static lease...

mkx · Tue Nov 12, 2024 12:33 pm

... best practice is to use MAC of the first ethernet interface that is part of the bridge ...

While this might be one of best approaches, it's not flawless ... if one removes "first ethernet interface" from bridge and forgets to change bridge MAC address, it's possible that some problems arise (in certain scenarios). IMO it's better to construct LAA MAC (based on MAC of first ethernet interface part of bridge) ... which makes possibility to have two interfaces with same MAC lower.

foraster · Wed Nov 13, 2024 11:46 am

Trying to upgrade a CCR1009 working as a wifiwave2 capsman from ROS 7.12 (has routeros and wifiwave2 packages)

I get an error:

wifi-qccom-ac-7.16.1-tile.npk missing, use ignore-missing or disable package(s)

Will I loose the capsman configuration ?

mkx · Wed Nov 13, 2024 12:03 pm

Will I loose the capsman configuration ?

It is a possibility. As far as I remember wifiwave2 config structure is pretty close (if not the same) as the (new) wifi config structure. So even if you'll have to manually upgrade configuration after you upgrade ROS, it shouldn't be a big problem.

Export (at least) contents of /interface/wifiwave2 to ASCII file (execute /interface/wifiwave2/export file=wave2config show-sensitive) and fetch it off CCR before performing upgrade. After upgrade you'll probably have to copy-paste config, replacing /interface/wifiwave2/xxx with /interface/wifi/xxx ...

You'll probably have to disable and remove the (optional) wifiwave2 package as there is no "successor" for tile architecture. However, the base functionality (which includes CAPsMAN) is included in basic (routeros) package.

infabo · Wed Nov 13, 2024 12:32 pm

Trying to upgrade a CCR1009 working as a wifiwave2 capsman from ROS 7.12 (has routeros and wifiwave2 packages)

I get an error:
wifi-qccom-ac-7.16.1-tile.npk missing, use ignore-missing or disable package(s)
Will I loose the capsman configuration ?

You dont need any wifi packagge on your CCR anymore to have a running >wifi< capsman. All included in the routeros package. You most probably won't lose your config.

nmt1900 · Wed Nov 13, 2024 3:58 pm

Trying to upgrade a CCR1009 working as a wifiwave2 capsman from ROS 7.12 (has routeros and wifiwave2 packages)

I get an error:
wifi-qccom-ac-7.16.1-tile.npk missing, use ignore-missing or disable package(s)
Will I loose the capsman configuration ?

As always

export terse file=<filename> show-sensitive

will be your friend if configuration would be lost. You can always restore relevant parts from there.

foraster · Wed Nov 13, 2024 5:14 pm

Trying to upgrade a CCR1009 working as a wifiwave2 capsman from ROS 7.12 (has routeros and wifiwave2 packages)

I get an error:

Will I loose the capsman configuration ?
As always
Code: Select all
export terse file=<filename> show-sensitive
will be your friend if configuration would be lost. You can always restore relevant parts from there.

Indeed! Will try it during the next maintenance window

foraster · Wed Nov 13, 2024 5:15 pm

Will I loose the capsman configuration ?
It is a possibility. As far as I remember wifiwave2 config structure is pretty close (if not the same) as the (new) wifi config structure. So even if you'll have to manually upgrade configuration after you upgrade ROS, it shouldn't be a big problem.

Export (at least) contents of /interface/wifiwave2 to ASCII file (execute /interface/wifiwave2/export file=wave2config show-sensitive) and fetch it off CCR before performing upgrade. After upgrade you'll probably have to copy-paste config, replacing /interface/wifiwave2/xxx with /interface/wifi/xxx ...

You'll probably have to disable and remove the (optional) wifiwave2 package as there is no "successor" for tile architecture. However, the base functionality (which includes CAPsMAN) is included in basic (routeros) package.

Understood. Thanks!

foraster · Wed Nov 13, 2024 5:16 pm

Trying to upgrade a CCR1009 working as a wifiwave2 capsman from ROS 7.12 (has routeros and wifiwave2 packages)

I get an error:

Will I loose the capsman configuration ?
You dont need any wifi packagge on your CCR anymore to have a running >wifi< capsman. All included in the routeros package. You most probably won't lose your config.

Thanks!

gammy69er · Wed Nov 20, 2024 2:56 am

*EDIT*
I Did some more testing that made the below questions irrelevant.
Apparently SW Vlans have ALWAYS bled through when bridge ports are disabled (ran through a few versions of ROS testing)
RSTP did stop them in the end - I just had to be more patient... just odd that whenever i've done these build before i have generally dropped the bridge ports - and never had a vlan bleed through a disabled bridge port - that I noticed.

Just though i'd leave this in place for transparency - but it appears that my recollecion of how things "worked" was incorrect... and although maybe undesired function - appears to just be the way it is for now.

*ORIGINAL POST*
Unsure if anyone else is having any issues with switch vlanning

General Configs for QCA8337 switches have been roughly switch vlans with bridge ports. Bridge Vlans disable hardware offload - so this has been the best way to use vlans while keeping wire speed

Usually - when I disable a bridge port in this scenarion - all traffice across the switch stopps to this port.

Just building a site with 7.16.1 - and have found that during build - when I completed config of a wireless link - that once it links and loops (as expected), if I disable one of the looped bridge ports - the loop continues on what appears to be the vlans.
Previously, disabling the bridge port would stop All traffic to that port - stopping the loop
Long term - this will not be an issue - as these wireless links will not be on the same switch - but it does appear there is an issues - and it makes it more diffidcult to recover from a loop condition out in the field.
This was done on a RB960-PGS-PB

Also to Note - in the case of the OmnitikAC-PoE - that I was also building before - disabling the WLAN port in the bridge DID have the desired affect... probably due to the fact that the wlan was not in the switch. Also - IIRC - disabling the ethernet bridge ports had the desired effect here too.

has anyone else noted any issues with SW vlans in this F/W... do we know if there has been any chnages to the older chips not mentioned?

Any Info appreciated.

vhaisman · Sat Nov 23, 2024 2:06 am

I have enabled dhcpv6 client via terminal and now every command that I do inside ipv6/dhcp-client hangs or times out. Notice the "#error exporting "/ipv6/dhcp-client" (timeout)" in the middle of the text:

[xxx@MikroTik8] /ipv6> export
# 2024-11-23 00:53:51 by RouterOS 7.16.1
# software id = 3GZX-TBTL
#
# model = CRS112-8P-4S
# serial number = F14F0EFCA324
#error exporting "/ipv6/dhcp-client" (timeout)
/ipv6 nd
set [ find default=yes ] disabled=yes mtu=1492 ra-preference=high retransmit-interval=3s
/ipv6 settings
set accept-redirects=no accept-router-advertisements=yes forward=no max-neighbor-entries=8096
[xxx@MikroTik8] /ipv6>

I don't see the DHCPv6 client record in WinBox but when I try to add one, it complains "dhcp-client on that interface already exists".

I even tried to restart the router multiple times.

How can I fix this? How can I remove the dhcpv6 client that is not being shown anywhere?

BrateloSlava · Sat Nov 23, 2024 10:17 am

There's been some kind of confusing situation. It doesn't make any sense. I have two 4011s with firmware 7.16.1 that were fine a week ago. Today I noticed that DHCPv6 client on both devices stopped working normally. I have not made any changes to the settings. Everything “really” broke on its own.

/ipv6 address
# address pool error: pool not found: ipv6-triolan (4)
add eui-64=yes from-pool=ipv6-triolan interface=bridge1
#error exporting "/ipv6/dhcp-client" (timeout)

Screenshot_1.png

Screenshot_2.png

In the backup from the previous weekend, these lines looked like this:

/ipv6 address add address=::2ec8:1bff:feae:493d eui-64=yes from-pool=ipv6-triolan interface=bridge1
/ipv6 dhcp-client add add-default-route=yes interface=sfp-sfpplus1 pool-name=ipv6-triolan rapid-commit=no request=prefix script=":if (\$\"pd-valid\" = 1) do={\r\
    \n  /ipv6 firewall address-list remove [find list=allowed]\r\
    \n  :delay 1s\r\
    \n  /ipv6 firewall address-list add address=\$\"pd-prefix\" comment=\"!!! Check YOUR pool from ISP\" list=allowed\r\
    \n  /ipv6 firewall address-list add address=fe80::/16 list=allowed\r\
    \n  /ipv6 firewall address-list add address=ff02::/16 comment=multicast list=allowed\r\
    \n  :delay 1s\r\
    \n} \r\
    \n" use-interface-duid=yes use-peer-dns=no

mkx · Sat Nov 23, 2024 2:01 pm

There's been some kind of confusing situation. It doesn't make any sense. I have two 4011s with firmware 7.16.1 that were fine a week ago. Today I noticed that DHCPv6 client on both devices stopped working normally. I have not made any changes to the settings.

Did devices reboot in between by any chance?

BrateloSlava · Sat Nov 23, 2024 5:13 pm

Did devices reboot in between by any chance?

Yes, both devices rebooted during this period.
By the way, noticed a similar problem on another 4011. This is the third one.
I had a similar problem on version 7.16. Completely different location, but also on 4011. The solution was to downgrade ROS to 7.15.3, disable and delete all IPv6 settings, upgrade ROS to 7.16 and configure IPv6 again. I tried the same steps (ROS downgrade, etc.) on these three routers. However, it did not work. I cannot re-configure IPv6.

I would like to point out, that I have only observed a similar problem with 4011. The routers are located in neighboring houses in a cottage village. In the building, where the village security is located - hAP ac3, in the building next to it - CCR1016-12G. And no problems with IPv6.

BrateloSlava · Sat Nov 23, 2024 10:01 pm

And one more question. I have routers, that do not use IPv6 at all. Why do such devices assign an internal IPv6 address to IPIP tunnels? The use of the protocol is disabled, the addresses are invalid.

Screenshot_1.png

prawira · Sun Nov 24, 2024 11:17 am

hello,

firstly, i found that dhcp-snooping feature on crs3xx does not work on this version, i will open separate ticket for this.
secondly, i wonder if dhcp vendor-class can give an option of static-only for the address-pool of choosen value.
third, i have hap-ax3 here. work good for ethernet and wireless as expected with vlan-1; but when i create virtual ssid with other vlan, it won't work. datapath with vlan tagged also does not work.

thank you

P

peri · Wed Nov 27, 2024 7:41 am

While using RouterOS version 7.16.*, I encountered the following issue:
I need to resolve the domain name test.ab.lan locally, while forwarding other *.lan domain names to a third-party DNS server. I implemented the following code:

/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.1.1 name=test.ab.lan type=A
add forward-to=192.168.2.1 regexp="(^|\\.)lan\$" type=FWD

This code worked perfectly in version 7.15.* and earlier. However, in version 7.16.*, an issue arises.
When other machines query test.ab.lan from this device, it seems that the FWD rule is executed first instead of responding with the address 192.168.1.1 as specified.
I find this behavior counterintuitive and unreasonable.

SUP-172504

Wed Nov 27, 2024 10:25 am

What's new in 7.16.2 (2024-Nov-26 14:09):

*) certificate - do not download CRL if there is not enough free RAM;
*) certificate - fixed handling of capsman-cap certificates (introduced in v7.16);
*) dhcpv4-server/relay - added additional error messages for DHCP servers and relays;
*) dns - fixed lookup order for static DNS entries (introduced in v7.16.1);
*) ethernet - improved linking after reboot for hAP ax lite devices ("/system routerboard upgrade" required);
*) gps - changed default GPS antenna setting for LtAP mini with internal LTE/GPS combo antenna;
*) leds - fixed bogus argument for "leds" property (introduced in v7.16);
*) leds - fixed PoE-in LEDs for CRS318-1Fi-15Fr-2S device;
*) modem - KNOT BG77 modem, improved handling of modem unexpected restarts;
*) route - fixed possible issue with inactive routes after reboot (introduced in v7.16);
*) routerboot - improved stability for IPQ8072 and IPQ6010 when flash-boot is used ("/system routerboard upgrade" required);

infabo · Wed Nov 27, 2024 10:42 am

*) dns - fixed lookup order for static DNS entries (introduced in v7.16.1);

I did not know that there is an documented lookup order at all? Any details? Is this the problem reported here: viewtopic.php?p=1111498#p1111473 ?

rextended · Wed Nov 27, 2024 10:46 am

It's time..
7.16.2 -> long-term
7.17 -> stable

infabo · Wed Nov 27, 2024 10:53 am

Won't happen. It would be ridiculous to introduce security measures in stable branch while keeping "loose device-mode" in a long-term branch for another long period of time.

Wed Nov 27, 2024 10:54 am

Personal view:
the fact 7.16.2 is being made, makes me think/hope it might take a bit longer before 7.17 becomes stable since all bug fixes here, are also available in 7.17 chain.
And that might be a good thing so more issues can be ironed out for 7.17.

wispmikrotik · Wed Nov 27, 2024 10:56 am

Hi,

And in v7.16.X there are still problems with ipsec policies, resolved in v7.17rc1.

Surely you are already working on the new v7.18 beta.

Greetings

bommi · Wed Nov 27, 2024 11:11 am

Hi,

I did an upgrade on my capsman (CHR) and caps (cap AX and hAP ax2) from 7.16.1 to 7.16.2.
In the logs of the caps I can see this message:
"no suitable CAPsMAN"

Anybody else seeing the same issue?

Wed Nov 27, 2024 11:29 am

Since this version has some specific changes related to certificates for capsman, it might be needed to clear certificates on capsman controller.

bommi · Wed Nov 27, 2024 11:34 am

I just did another upgrade on the capsman to 7.17rc1 and the caps are only again.
In 7.17rc1 includes the same fix as 7.16.2:
*) certificate - fixed handling of capsman-cap certificates (introduced in v7.16);

Therefore I suspect this to be some other issue.

Guscht · Wed Nov 27, 2024 11:57 am

smol home-net upgraded to v7.16.2, everything works for me:

Zwischenablage_11-27-2024_01.jpg

mirolm · Wed Nov 27, 2024 12:47 pm

I just did another upgrade on the capsman to 7.17rc1 and the caps are only again.
In 7.17rc1 includes the same fix as 7.16.2:
*) certificate - fixed handling of capsman-cap certificates (introduced in v7.16);

Therefore I suspect this to be some other issue.

I had issue with enabling "request-peer-certificate" while upgraging from hexs to the new hex on my network. On enabling this caps refused to connect with ssl errors. The only way to make this work was without enabling this. With 7.16.2 installed i no longer run into this problem.

Guscht · Wed Nov 27, 2024 5:13 pm

It's time..
7.16.2 -> long-term
7.17 -> stable

I think you had to much Pizza and Vino :D
V7 is stil sooooo far away from a "long-term" aka production-ready!!

yuripg1 · Wed Nov 27, 2024 5:30 pm

It's time..
7.16.2 -> long-term
7.17 -> stable
I think you had to much Pizza and Vino :D
V7 is stil sooooo far away from a "long-term" aka production-ready!!

If you think in terms of stability alone, of course. There are a lot of small things to iron out.

But I believe any long term release would start from a similar place as 7.16.2. The main point would be to set a cut-off in terms of features and then work on improving stability.

I don't know about you, but I think a version 7.16.9 would turn out to be awesome 😁

peri · Thu Nov 28, 2024 3:53 am

While using RouterOS version 7.16.*, I encountered the following issue:
I need to resolve the domain name test.ab.lan locally, while forwarding other *.lan domain names to a third-party DNS server. I implemented the following code:
Code: Select all
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.1.1 name=test.ab.lan type=A
add forward-to=192.168.2.1 regexp="(^|\\.)lan\$" type=FWD
This code worked perfectly in version 7.15.* and earlier. However, in version 7.16.*, an issue arises.
When other machines query test.ab.lan from this device, it seems that the FWD rule is executed first instead of responding with the address 192.168.1.1 as specified.
I find this behavior counterintuitive and unreasonable.

SUP-172504

Thank you for the fix. It is now working in version 7.16.2. However, there is still an issue: the response time is timing out. The behavior is as follows:

When testing the local subnet on Windows:

nslookup test.ab.lan 192.168.73.222
Server:  chradmin.lan
Address: 192.168.73.222

Non-authoritative response:
DNS request timed out.
    timeout was 2 seconds.
Name:    test.ab.lan
Address: 192.168.1.1

The first response times out, indicating there is still a problem.

SUP-172504

tpedko · Fri Nov 29, 2024 10:06 am

Mikrotik support pay attention to SUP-172615, BGP+BFD problem on CCR2116-12G-4S+

pe1chl · Fri Nov 29, 2024 11:27 am

What is that BGP+BFD problem? I have some BGP problems but not BFD related...

tpedko · Fri Nov 29, 2024 1:34 pm

What is that BGP+BFD problem? I have some BGP problems but not BFD related...

77 bgp peers, when enabled on all bfd sessions are destroyed. Works stably only on 6 enabled peers.

pe1chl · Fri Nov 29, 2024 3:24 pm

Ok I do not have that many peers on a 7.16 router. but I do see another problem: multiple routes to the same peer over different paths are not stored. do you see that as well? It worked fine before.
E.g. we have peers with a GRE/IPsec, a GRE6/IPsec and a L2TP/IPsec tunnel (over LTE) between them for backup purposes.
When the GRE tunnels fail due to fiber/vdsl failure, there are no fallback routes over the L2TP available until the BGP peer is disconnected/reconnected.

tpedko · Fri Nov 29, 2024 4:58 pm

I noticed this problem

yhfung · Mon Dec 02, 2024 5:13 am

viewtopic.php?p=1103784#p1103784

When I upgraded to v7.16.2 on RB941G-2HnD, I downloaded gfwlist.rsc in mentioned above https link and found it in failed. For example "ping cctv.com".

Please upgrade in the next version firmware.

peri · Mon Dec 02, 2024 7:18 am

While using RouterOS version 7.16.*, I encountered the following issue:
I need to resolve the domain name test.ab.lan locally, while forwarding other *.lan domain names to a third-party DNS server. I implemented the following code:
Code: Select all
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.1.1 name=test.ab.lan type=A
add forward-to=192.168.2.1 regexp="(^|\\.)lan\$" type=FWD
This code worked perfectly in version 7.15.* and earlier. However, in version 7.16.*, an issue arises.
When other machines query test.ab.lan from this device, it seems that the FWD rule is executed first instead of responding with the address 192.168.1.1 as specified.
I find this behavior counterintuitive and unreasonable.

SUP-172504
Thank you for the fix. It is now working in version 7.16.2. However, there is still an issue: the response time is timing out. The behavior is as follows:

When testing the local subnet on Windows:
Code: Select all
nslookup test.ab.lan 192.168.73.222
Server:  chradmin.lan
Address: 192.168.73.222

Non-authoritative response:
DNS request timed out.
    timeout was 2 seconds.
Name:    test.ab.lan
Address: 192.168.1.1
The first response times out, indicating there is still a problem.

SUP-172504

I have found the reason for the timeout on the first response: it was because I didn’t configure a custom AAAA record for the domain name.
Using a command that only queries the A record does not result in a timeout. Alternatively, adding an AAAA record resolves the issue as well.
SUP-172504 is closed。

junbr0 · Mon Dec 02, 2024 11:58 am

7.16.1 dns doh bogged down cpu to 100% when internet inactive. mdns repeater turned on.

onovy · Mon Dec 02, 2024 10:03 pm

IPv6 over 6rd is broken (terribly slow) in 7.16 for me and it looks like I'm not alone: viewtopic.php?p=1112599

pe1chl · Tue Dec 03, 2024 4:47 pm

It seems that in the 7.16 (and 7.16.x) versions BGP has become unreliable. Today we had some internet outages and the backup route config I have used for a long time starting with v6 simply no longer works. I have noticed it before and tried different things but never got it working reliably anymore.
A very apparent bug is that after BGP connection establishment, the list of routes is not sent immediately but only after one "Hold Time" interval (default one minute). That was acknowledged by MikroTik and would be fixed in an upcoming release, but it hasn't been fixed even in 7.17rc2 let alone in 7.16.2...
Another problem is that BGP often does not store multiple routes to the same peer in the route table, when multiple direct paths exist to the same peer.
In my config I have a GRE/IPsec, a GRE6/IPsec tunnel (via DSL/Fiber) and an L2TP/IPsec (via 4G) to each peer, and have configured the latter to use a lower local pref, but in the sessions list the number of prefixes via the L2TP/IPsec is often zero, and when the others fail there are no routes.
I have also tried to use "prepend" to insert an extra hop but the problem remains the same (the L2TP/IPsec routes are only to be used when the others have failed).

Do others experience the same problems? It worked reliably before, but it seems the fixes in BGP for this release have introduced these problems, and there are no BGP fixes in 7.17rc so we basically are in a dead end. Several times people at remote locations had to resort to powercycling the router, which was never necessary before.

kirderf · Tue Dec 03, 2024 9:18 pm

I upgraded a CCR2004-16G-2S+ a couple of days ago and almost all my scripts stopped working, they gave the error "invalid internal item number". I tried breaking down the failing lines of code to see where the issue was, but I was in no way able to get it to work. Downgraded to 7.15.3 and everything is working again.

Cha0s · Tue Dec 03, 2024 10:00 pm

It seems that in the 7.16 (and 7.16.x) versions BGP has become unreliable. Today we had some internet outages and the backup route config I have used for a long time starting with v6 simply no longer works. I have noticed it before and tried different things but never got it working reliably anymore.
A very apparent bug is that after BGP connection establishment, the list of routes is not sent immediately but only after one "Hold Time" interval (default one minute). That was acknowledged by MikroTik and would be fixed in an upcoming release, but it hasn't been fixed even in 7.17rc2 let alone in 7.16.2...
Another problem is that BGP often does not store multiple routes to the same peer in the route table, when multiple direct paths exist to the same peer.
In my config I have a GRE/IPsec, a GRE6/IPsec tunnel (via DSL/Fiber) and an L2TP/IPsec (via 4G) to each peer, and have configured the latter to use a lower local pref, but in the sessions list the number of prefixes via the L2TP/IPsec is often zero, and when the others fail there are no routes.
I have also tried to use "prepend" to insert an extra hop but the problem remains the same (the L2TP/IPsec routes are only to be used when the others have failed).

Do others experience the same problems? It worked reliably before, but it seems the fixes in BGP for this release have introduced these problems, and there are no BGP fixes in 7.17rc so we basically are in a dead end. Several times people at remote locations had to resort to powercycling the router, which was never necessary before.

I too have issues after 7.16 with BGP/routing.
Not the ones you describe, but since v7.16 there were multiple instances on multiple routers that the routing process would take up 100% CPU (single core) and the logs would get repeated errors about SNMP timeouts. Only reboot would resolve that.

Then on v7.17 there have been multiple times that all peers get disconnected at the same time, like BGP is completely crashing, and then re-establish the connections. At least that recovers on its own and no reboot is needed.
And there's also a problem with garbage routes staying into the routing table after a peer gets disconnected and reconnected multiple times in a short period of time.
e.g.: I except 300 routes from a single peer, but it counts double that. It's not a cosmetic issue on BGP sessions table, but also in routing table there would be multiple inactive identical routes from the same peer.
Disabling/Enabling the peer does not clear up the garbage routes. With a reboot it clears up. But when BGP resets (crashes?) itself as mentioned above, then all erroneous routes disappear on their own without a reboot.

I haven't able to find any pattern to any of these problems. They may occur after 1 day of uptime, or many days of uptime, and on routers that I don't even login almost never (ie no user actions are being performed when the issues occur).

pe1chl · Tue Dec 03, 2024 10:23 pm

I have seen those issues in 7.16 as well: a single disconnected peer disconnects multiple or all peers at the same time, and routes via disconnected peers still appearing in the table.
The first I was able to improve a bit by forcing all BGP handling into a single process (input.affinity=main output.affinity=input in the bgp template), the second indeed seems to happen sometimes but is often not reproducible.
It is not the (other) issue that in winbox when the winbox session is closed and re-opened the routing table looks different.
E.g. entries with "unknown" or "*12345678" in the "immediate gateway" field are OK again after a close/open. That one seems to be only cosmetic.
But the extra or the missing routes are not, the destinations really become unreachable due to that.

It is unfortunate that it is no longer working correctly, but it is even worse that it appears nothing is being done about it (no bgp changes in 7.17)...

vahid67 · Tue Dec 03, 2024 11:48 pm

Hello,
I'm trying to import openvpn client profile as the document suggests for cipher AES-128-GSM, the Auth parameter must be null but when I set that to null in the .ovpn config file it can't be imported and it returns "unsupported auth 'NULL'"

My device is hAP lite.

MTNick · Wed Dec 04, 2024 12:46 am

Greetings Mikrotik team.

ROS 7.16.2

Is there a reason using standard DNS, the in use DNS server always ends up being Cloudflare? No matter which order I put the servers in, the DNS in use always ends up being Cloudflare within a few minutes of arranging them. Or by running the dns leak test. Why is Cloudflare preferred over any other DNS server? Is it response time, quickest vs slowest?

Example that can be arranged in any order but the Mikrotik will always end up using 1.1.1.2 or 1.0.0.2.
76.76.2.2
94.140.14.14
1.1.1.2
76.76.10.2
94.140.15.15
1.0.0.2

dioeyandika · Wed Dec 04, 2024 1:31 am

RB 750Gr3
Trying experimenting with BGP have 3 connection, one connection have full route of internet (about 900k prefix) filtered google prefix and some my ISP peer, second connection is for BGP blackhole (spamhouse,cisco talos etc), third connection is BGP for geo (used filter to accept prefix in my country) before ROS V17.16.2 (i forget which one) the BGP is rock solid for days, now it really unstable the connection stuck 0 prefix i need to disable and enable few times, i though my connection is bad but event i use static public ip (L2TP client in RB 750gr3 in remote location this have static public ip) still really unstable, scratching this problem over a week i remember i add blackhole RFC6890 ip list then i delete that, it stay for a day now, and for the option (input.affinity=main output.affinity=main) i only receive prefix not advertised it, it just for experiment, my thought is for my problem maybe in my case "In v7 it is not possible to turn off synchronization with IGP routes (the network will be advertised only if the corresponding IGP route is present in the routing table)." i know maybe my config is wrong but that just guessing the problem

pe1chl · Wed Dec 04, 2024 10:52 am

Is there a reason using standard DNS, the in use DNS server always ends up being Cloudflare? No matter which order I put the servers in, the DNS in use always ends up being Cloudflare within a few minutes of arranging them. Or by running the dns leak test. Why is Cloudflare preferred over any other DNS server? Is it response time, quickest vs slowest?

There are two algorithms commonly in use for DNS server selection. I do not know which one MikroTik uses, I have never closely monitored that.
But they are:
1. try all servers in turn and send the majority of the requests to the server(s) that respond the quickest (that is what bind9 does)
2. send all requests to a single server from the list until it fails to respond in time, then move on to the next server in the list

MTNick · Wed Dec 04, 2024 1:40 pm

There are two algorithms commonly in use for DNS server selection. I do not know which one MikroTik uses, I have never closely monitored that.
But they are:
1. try all servers in turn and send the majority of the requests to the server(s) that respond the quickest (that is what bind9 does)
2. send all requests to a single server from the list until it fails to respond in time, then move on to the next server in the list

Thank you for the response pe1chi. Cloudflare is the quickest to respond by far. So I'd assume it's # 1. It would be nice if it was in the order listed.

76.76.2.2 - 70ms
94.140.14.14 - 80ms
1.1.1.2 - 18ms

pe1chl · Wed Dec 04, 2024 1:47 pm

Well, one can always wish for more advanced options.
What I would like is to have groups of servers, and then at first the servers from the first group are used in some optimized way like I described, and only when all servers from the first group are failing to respond, servers from the second group are used in a similar way.
That way I can specify the resolvers offered by the ISP in the first group, and put some fallback servers from Cloudflare or Google etc in the second group as a fallback.
Also, I would like to optionally assign source addresses and/or route marks to each of the servers, to be used when sending queries to those.
(that is important when more than one ISP is connected in parallel)

But I guess there are more important things to be fixed in the DNS resolver...

MTNick · Wed Dec 04, 2024 2:11 pm

Agreed. Not a big deal. Just wondered how the DNS worked in the Mikrotiks.

What you described, would following the dns list by order be somewhat similar? It would be a step in that direction in my opinion

McGremlin · Wed Dec 04, 2024 2:52 pm

Hi!
For testing purposes I have upgraded firmware from 7.12.1 to 7.16.2 on 3 Mikrotiks (RB951G-2HnD (mispbe), RBwAPG-5HacT2HnD (mipsbe) and RBwAPG-5HacD2HnD (arm)) working as access points with simple config, for few hours they are working fine:
- MAC Winbox access (or no access depending on interface or vlan),
- Bridge VLAN Filtering,
- RADIUS WiFi authentication,
- WiFi with WPA2 PSK security profile,
- script which checks failure login attempts every XX seconds (via scheduller) and sends e-mail notification if there were any.
I didn't need to change anything in the config after upgrades, everything just run smoothly.
I really hope it will look the same for my gateways :D

pe1chl · Wed Dec 04, 2024 5:21 pm

What you described, would following the dns list by order be somewhat similar? It would be a step in that direction in my opinion

Somewhat similar, but not the same.
Usually you get several alternative DNS resolver addresses from your ISP, e.g. in my case 4 (2 IPv4, 2 IPv6), and you would want to load balance across them and not send all traffic to the first until it fails.
Then when all 4 fail, you would want to select servers from the next group, in this case public servers.

gianry · Sat Dec 07, 2024 7:29 pm

I don't see the Flow Control fix as requested since two years ...

infabo · Sat Dec 07, 2024 11:05 pm

First time ever my cap ac (running wifi-qcom-ac) locked up yesterday (wifi stopped broadcasting, device not reachable). Needed to power cycle. Log only said "rebooted probably of power loss". What changed in 7.16.2?

webtelza · Mon Dec 09, 2024 7:45 am

On my CCR2216 which runs L3HW offloading, BGP, OSPF I have a problem with kernel failure and router then reboots. Routers connected to the 2216 have no network access until I disable L3HW offloading on the 2216. It seems that L3HW Offloading on 7.16.2 causes some crash with OSPF. When L3HW offloading is switched off, all routers immediately have internet access.

I also saw this when I upgraded from 7.16.1 to 7.16.2, there was no connection on the other routers until L3HW offloading was disabled. After a few days of stability on the 2216 I enabled L3HW offloading which worked fine for 3 days until this morning when there was kernel failures

jayooo · Tue Dec 10, 2024 6:04 am

Here's a bad bug:

If you turn on safe mode, make a change to the routing table, and then roll-back ... the routing table changes are still there. Safe mode does not undo routing table changes!

Tue Dec 10, 2024 8:48 am

Something odd going on with partitioning ...

[xyz@RB5009] > part
[xyz@RB5009] /partitions>
activate copy-to find repartition restore-config-from set
comment edit print reset save-config-to
[xyz@RB5009] /partitions> print
Flags: A - ACTIVE; R - RUNNING
Columns: NAME, FALLBACK-TO, VERSION, SIZE
# NAME FALLBACK-TO VERSION SIZE
0 AR part0 next RouterOS v7.16.2 2024-11-26 12:09:40 512MiB
1 part1 next RouterOS v7.16.2 2024-11-26 12:09:40 512MiB
[xyz@RB5009] /partitions> save-config-to part1
failure: target partition does not have RouterOS --> ???
[xyz@RB5009] /partitions> copy-to part1
status: copying (143%)
-- [Q quit|D dump|C-z pause]

After some minutes: status: ERROR: could not copy file: File exists

kcarhc · Tue Dec 10, 2024 9:54 am

While using RouterOS 7.16.2, I discovered that DNS Static rules do not seem to execute in the order they were added. This prevents some expected DNS resolution behaviors from being achieved. Specifically, here is my configuration example:

/ip dns static
add address-list=DNS_APP-COPILOT \
    forward-to=dns.cloudflare name=edge.microsoft.com type=FWD
# Additional static DNS entries can be added here
add name=edge.microsoft.com type=NXDOMAIN

Logically, the FWD rule should take precedence and forward edge.microsoft.com to Cloudflare’s DNS servers. However, in reality, when a request for edge.microsoft.com is made to RouterOS, it returns an NXDOMAIN response, indicating that the domain does not exist. This means that the subsequent NXDOMAIN rule is being triggered before the FWD rule.

Comparison with Other DNS Software:

In other mainstream DNS software, such as dnsmasq and Unbound, DNS rules are matched and executed sequentially from top to bottom. Below are configuration examples for each software based on the above scenario:

dnsmasq
dnsmasq matches DNS rules in the order they are added, with earlier rules taking precedence.

In dnsmasq, the server directive is used to forward edge.microsoft.com to Cloudflare’s DNS servers and is placed before the address directive. This way, when a DNS query for edge.microsoft.com is received, dnsmasq first applies the forwarding rule. If the forwarding fails, it then applies the NXDOMAIN rule.

Unbound
Unbound matches rules in the order they appear in the configuration file, ensuring that specific rules take precedence over general ones.

In Unbound, the local-data directive is used to redirect edge.microsoft.com to Cloudflare’s IP address and is placed before the local-zone directive. This ensures that when a DNS query for edge.microsoft.com is received, Unbound first applies the forwarding rule. If the forwarding fails, it then applies the refusal (NXDOMAIN) rule.

Why Rule Order Matters:

The order of rules determines the priority and execution sequence. If rules are not executed in order, the following issues may arise:

Security Issues: Incorrect DNS resolution can lead to accessing incorrect or malicious websites. For example, phishing sites may exploit erroneous DNS rules to trick users into entering sensitive information.

Inability to Access Intended Websites: Proper forwarding rules are overridden by subsequent NXDOMAIN rules, preventing users from accessing required services or websites.

Difficulty in Troubleshooting: A disordered rule sequence complicates the process of diagnosing and fixing DNS issues, increasing maintenance difficulty.

Precedence Rules Fail: On configuration servers, subsequent conditions may override earlier settings, causing the initial rules to become ineffective. This practice is highly discouraged on enterprise-grade routers, as these devices typically execute rules in the order they are added to ensure each rule’s priority and intended effect.
Recommendations and Requests:

Confirm if It's a Bug: If this behavior is unintended, please verify whether it’s a bug in RouterOS 7.16.2 and prioritize a fix.
Restore Sequential Rule Execution: If the change in rule execution order was intentional, consider reverting to the previous behavior where rules are executed in the order they were added to maintain the expected prioritization.

Update on Changelog Fix: The update log for version 7.16.2 includes the following entry:
scss

*) dns - fixed lookup order for static DNS entries (introduced in v7.16.1);

Therefore, I suspect that the lookup order issue has not been completely resolved.

The execution order of rules is crucial for the correctness of DNS configurations. The issue in RouterOS 7.16.2, where DNS Static rules do not execute sequentially, may lead to serious DNS resolution errors, affecting the normal operation of network services. I hope the official team takes this issue seriously and resolves it promptly.

rextended · Tue Dec 10, 2024 3:13 pm

It looks like it was written by ChatGPT rather than a person, and it probably is.

pe1chl · Tue Dec 10, 2024 4:32 pm

Why???

infabo · Tue Dec 10, 2024 5:06 pm

It may be assisted by ChatGPT or just good old copy paste from google results.

yhfung · Wed Dec 11, 2024 5:50 pm

In v7.16.2 firmware，there were some problems with WiFi in hAP ac3. WiFi issue were resolved by using v7.16.1 firmware.

The v7.16.2 firmware was reloaded again and restored previous old settings, WiFis were alright.

sirbryan · Wed Dec 11, 2024 7:36 pm

I have seen those issues in 7.16 as well: a single disconnected peer disconnects multiple or all peers at the same time, and routes via disconnected peers still appearing in the table.
The first I was able to improve a bit by forcing all BGP handling into a single process (input.affinity=main output.affinity=input in the bgp template), the second indeed seems to happen sometimes but is often not reproducible.

The second that you mention describes best what I'm seeing on one of my busiest border routers.

Have you tried backing that/those router(s) to 7.14 to see if the problem goes away?

pe1chl · Wed Dec 11, 2024 8:09 pm

No, I don't have a test lab to do that.

oeyre · Wed Dec 11, 2024 11:35 pm

Everybody has a testing environment. Some people are lucky enough enough to have a totally separate environment to run production in.

pe1chl · Thu Dec 12, 2024 2:13 pm

Yes, but not me.
Testing this requires a network of like 6 routers with tunnels between them and BGP running on it.
So several locations with different IP addresses are required, 2 addresses at each location.
We have that in production but we do not have a separate test environment that replicates it.
(sure I have a test router and a CHR but that is not enough to debug this problem)

kcarhc · Thu Dec 12, 2024 2:47 pm

It looks like it was written by ChatGPT rather than a person, and it probably is.

The text written by a human, translated into English using GPT.

sirbryan · Fri Dec 13, 2024 6:24 am

No, I don't have a test lab to do that.

I do. if you're interested in cloning a sanitized version of your configs, I have four RB5009's and two CCR2004's racked up, with two more 5009's on the shelf.

Or we could spin up a bunch of CHR VM's, use GNS3, etc.

I backed my 2116 off to 7.15.3 early this morning to see if it exhibits the same "stuck routes" behavior as 7.16 has.

Fif · Fri Dec 13, 2024 7:18 am

There still are issues with the cert dates being reported incorrectly.

/certificate/print detail proplist=issuer,country,organization,common-name,serial-number,fingerprint,akid,skid,invalid-before,invalid-after  where common-name=R10  
Flags: K - private-key; L - crl; C - smart-card-key; A - authority; 
I - issued, R - revoked; E - expired; T - trusted 
 1  L    T issuer=C=US,O=Internet Security Research Group,CN=ISRG Root X1 
           country="US" organization="Let's Encrypt" common-name="R10" 
           serial-number="4ba85293f79a2fa273064ba8048d75d0" 
           fingerprint="9d7c3f1aa6ad2b2ec0d5cf1e246f8d9ae6cbc9fd0755ad37bb974b1
            f2fb603f3" 
           akid=79b459e67bb6e5e40173800888c81a58f6e99b6e 
           skid=bbbcc347a5e4bca9c6c3a4720c108da235e1c8e8 
           invalid-before=2160-04-18 23:28:16 
           invalid-after=2163-04-18 23:28:15

Note the wrong invalid-before and invalid-after dates in 2060 and 2063!
Same cert seen from WebFig shows the correct dates (see attached screenshot).

This seems to be a general issue, all my certs show wrong dates, not just the Let's Crypt R10 chain cert.

This issue has been acknowledged by Mikrotik support and should be fixed in the future.
Since it's mostly cosmetic, I do not expect a quick resolution.
Ticket is SUP-172469.

Sidewindr · Fri Dec 13, 2024 8:10 am

I have an issue with v7.16 and v7.16.2 where the hotspot no longer pops up the login window. It works initially then after a period of time it stops working. Reboot resolves the issue for a while and then it comes back. Issue possibly on tile architecture only, not experienced it yet on other architectures.

Another issue on arm was ETH1-ETH4 stopped working, showed as stopped in the RouterOS interface list but had link lights onthe ports.. A reboot resolved the issue.

pe1chl · Fri Dec 13, 2024 2:48 pm

No, I don't have a test lab to do that.
I do. if you're interested in cloning a sanitized version of your configs, I have four RB5009's and two CCR2004's racked up, with two more 5009's on the shelf.

The important part of the config is like this:
1 CCR2004, 3 RB5009, 3 RB951G.
The latter are leafnodes connecting to the CCR using L2TP/IPsec.
The 3 RB5009 have GRE/IPsec to the CCR, 2 of them also have GRE6/IPsec and all have an L2TP/IPsec.
There are also some GRE6/IPsec tunnels between the RB5009.
Over all of these tunnels a eBGP connection (different private AS on each router) is configured and routes to the local networks are exchanged.
A total of 22 prefixes.
On the RB5009 the L2TP/IPsec is a backup over LTE and it is configured in routing filters to have a lower local-pref.

Bugs we see are:
- when a L2TP/IPsec disconnects due to the LTE interface getting a new IP (happens every 8h or 24h here depending on provider) the BGP connections to other peers (unaffected by the IP change) get closed and have to re-connect
- the routes with a lower local-pref randomly are not put in the routing table at all (received messages in the BGP peer indicate the routes are received but prefixes remains 0 and the routes are not in the table in inactive state, so when another tunnel fails they are not available as backup)
- routes are only received after one hold-time after establishing the BGP connection (default 60 seconds)
- another issue is that the number of alternate routes even with the same local-pref seems to be hardcoded, it can be the cause of the above problem.

It all worked fine in v6 (same network config, older routers were CCR1009 and RB750Gr3+RB951G) and also in earlier v7 releases, I think at least up to 7.12 but maybe 7.15 too.
The problem is visible when looking in the routing tables, but it rears its ugly head only when there is an internet interruption somewhere, and the network breaks down although valid tunnels are still present, only there are no routes.
At such times the users tend to powercycle the equipment and that fixes things, but it should not be necessary.
The RB915G only come into play because their BGP connection gets reset when another (RB5009) has their link reset, but the routing tables there are OK because they are leafnodes with only one uplink.