I’ll describe my situation. When the main provider goes down, I have a USB LTE modem for backup. After some time, I noticed that the router began to slow down alot and the CPU loading was to high, almost always 100%. I dumped the config and saw a HUGE bunch of entries like: /disk set slot1 slot=slot1 type=hardware to… set slot1785 slot=slot1785 type=hardware. After I reset the router and re-uploaded the config, previously deleting these lines, device work fine again. Some time passed and this problem returned again. Could you explain me please how can I clear the config without resetting the router and how can protect from this issue in the future?
I asap greatly need a help from developers or community! If anyone knows how to fix this problem, I would be extremely grateful.
That is (was) a known issue with CHR in VMware (or similar): http://forum.mikrotik.com/t/routeros-7-9-2-x86-disk-issue/167122/1
Post some details on the hardware you are using, it is possible that for some reasons the USB LTE modem is (also) detected as a disk, at each reboot and/or every time the LTE is disabled or enabled, 1785 seems to me a very high number both for reboot and for failover switches, anyway, so maybe the adding of the disk slot happens on other events too.
Or maybe you have some kind of periodically running script that creates the problem?
Setting through a scheduler a script that removes that entries once a day/week/month should be relatively easy, but it would be better - if possible - to find the root cause for that behaviour.
Yep, I noticed that this happens when the modem is disconnected ftom the usb slot or if possible someone touches it. Device - hap ac lite. Yes, a script that erase all slots entries at once and which runs once per day via the cron would probably be at least some kind of temporary solution, but I’m waiting for one of the Mikrotik admins on this forum to tell me how to disable loging of this sh!t. Do you know command that clear all slot resords on the disk?
There is my config:
/interface bridge
add name=bridge1
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX \
disabled=no distance=indoors frequency=auto installation=indoor mode=\
ap-bridge ssid=XXX wireless-protocol=802.11
set [ find default-name=wlan2 ] band=5ghz-a/n/ac channel-width=\
20/40/80mhz-XXXX distance=indoors frequency=auto installation=indoor \
mode=ap-bridge ssid=XXX wireless-protocol=802.11
/disk
set slot1 slot=slot1 type=hardware
...
...
...
set slot1785 slot=slot1785 type=hardware
/interface list
add name=WAN
add name=LAN
/interface lte apn
set [ find default=yes ] add-default-route=no ip-type=ipv4 use-network-apn=no \
use-peer-dns=no
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip hotspot profile
set [ find default=yes ] html-directory=hotspot
/ip pool
add name=dhcp ranges=10.10.10.10-10.10.10.254
/ip dhcp-server
add address-pool=dhcp interface=bridge1 lease-time=10m name=bridge1
/snmp community
set [ find default=yes ] addresses=1.2.3.4/32
/system logging action
set 3 bsd-syslog=yes remote=1.2.3.4 syslog-facility=local0
/user group
add name=prometheus policy="read,api,!local,!telnet,!ssh,!ftp,!reboot,!wr\
ite,!policy,!test,!winbox,!password,!web,!sniff,!sensitive,!romon,!rest-api"
/interface bridge port
add bridge=bridge1 interface=ether2
add bridge=bridge1 interface=ether3 internal-path-cost=10 path-cost=10
add bridge=bridge1 interface=ether4 internal-path-cost=10 path-cost=10
add bridge=bridge1 interface=ether5 internal-path-cost=10 path-cost=10
add bridge=bridge1 ingress-filtering=no interface=wlan1 internal-path-cost=10 \
path-cost=10
add bridge=bridge1 ingress-filtering=no interface=wlan2 internal-path-cost=10 \
path-cost=10
/ip neighbor discovery-settings
set discover-interface-list=LAN
/ip settings
set max-neighbor-entries=8192
/ipv6 settings
set disable-ipv6=yes forward=no max-neighbor-entries=8192
/interface detect-internet
set detect-interface-list=all
/interface list member
add interface=bridge1 list=LAN
add interface=ether1 list=WAN
add interface=lte1 list=WAN
/interface ovpn-server server
set auth=sha1,md5
/ip address
add address=10.10.10.1/24 interface=bridge1 network=10.10.10.0
add address=1.2.3.1/24 interface=ether1 network=1.2.3.0
/ip dhcp-client
add add-default-route=no interface=ether1
/ip dhcp-server lease
add address=10.10.10.2 client-id=1:d7:b3:c2:61:ee:5e mac-address=\
D1:BB:C1:6C:EE:E5 server=bridge1
/ip dhcp-server network
add address=10.10.10.0/24 gateway=10.10.10.1 netmask=24
/ip dns
set allow-remote-requests=yes servers=8.8.8.8,8.8.4.4,1.1.1.1 \
verify-doh-cert=yes
/ip firewall nat
add action=masquerade chain=srcnat out-interface-list=WAN
/ip route
add check-gateway=ping disabled=no distance=2 dst-address=0.0.0.0/0 gateway=\
9.9.9.9 pref-src="" routing-table=main scope=10 suppress-hw-offload=no \
target-scope=12
add disabled=no distance=2 dst-address=9.9.9.9/32 gateway=1.2.3.4 \
pref-src="" routing-table=main scope=10 suppress-hw-offload=no \
target-scope=11
/ip service
set telnet disabled=yes
set ftp disabled=yes
set www disabled=yes
set api-ssl disabled=yes
/ip ssh
set always-allow-password-login=yes forwarding-enabled=remote strong-crypto=\
yes
/routing bfd configuration
add disabled=no interfaces=all min-rx=200ms min-tx=200ms multiplier=5
/snmp
set enabled=yes trap-version=3
/system logging
set 0 action=remote prefix=:Info
set 1 action=remote prefix=:Error
set 2 action=remote prefix=:Warning
set 3 action=remote prefix=:Critical
add action=remote prefix=:Firewall topics=firewall
add action=remote prefix=:Account topics=account
add action=remote prefix=:Caps topics=caps
add action=remote prefix=:Wireles topics=wireless
Yep, still 1785 is simply too high, over a year period it means 5 disconnections per day, every day, over a three month period it means 20 disconnections per day, etc.
Even if your cabled ISP service sucks, it is improbable that the reason is there.
As well I don’t think that anyone is “touching” the USB modem or the ac lite that often.
Could it be a mechanical/electrical issue with the contacts of the USB connector?
In any case there is AFAIK no reason why the counter should increase when the failover happens, whilst it is possible that it happens when/if you reboot the router or insert/remove the stick, a defective contact on the USB may lead to the same as inserting/removing the stick.
Can you try (besides cleaning the contacts) with a (short) USB extension cord (so that the USB port has not to bear the weight of the USB modem)?
It could well be a cold solder on the USB port (either on the LTE router or on the Mikrotik side) but dirty/loose contacts are more common.
Yes, I already did it, installing a short USB extension cable but the USB. I also thought about this (the modem is very light weight tho), but the issue is different - why does this even happen?? As far as we can see in the topic, according to one of the links you posted, the guy wrote about it one year ago! Thats the problem has existed since version 7.9.2 but no one still fixed this yet.
Why it happens (if it happens) at reboot is definitely a bug, while if it happens when inserting/removing - while still a bug of some kind - it is more understandable, the OS has a disk detected and you suddenly remove it, and when you reinsert it a new device is detected, I would say that it can happen on many operating systems with “hot plug” devices.
The referenced thread was about a (virtual) CD/DVD or memory reader in the virtual machine, even if the behaviour is the same the cause may be different.
I have seen some USB (3G) modems that behave (at least under WIndows) as “dual” devices, i.e. both as modem and as storage (the storage part is intended to hold the software and/or drivers) so that they are seen also as disk, and I suspect your device is similar.
I cannot see anything in your configuration that could be connected to this behaviour
Which exact make/model is the USB LTE thingy? (maybe there is a documented way or a firmware update for it that removes the disk-like device)
In any case the number of connections/disconnections seem to me not normal, i.e. even if the Ros has its faults, these “flapping” shouldn’t happen, that is what makes me think of a possible connector/contacts physical issue.
What has to be understood/tested is whether the device is actually “assigned” (or whatever) to the last slot or not.
I.e. the first time this happens you have:
set slot1 slot=slot1 type=hardware
and the second time you have:
set slot1 slot=slot2 type=hardware
The question is, does the modem work if you remove the first entry or if you remove the second (or both as this setting does not affect the modem)?
As I can see, it happens whenever either the device is unplugged from the slot or when the router loses power, even just rebooting the router properly (I’ve just tested this by myself)! This is definitely buggy behavior. I’m not sure but maybe the developers needed this to test some functionality and they forgot after to turn it off in the final firmware, since few people complained about it. It’s just my thoughts maybe I’m wrong.
Which exact make/model is the USB LTE thingy? (maybe there is a documented way or a firmware update for it that removes the disk-like device)
ZTE MF79U
In any case the number of connections/disconnections seem to me not normal, i.e. even if the Ros has its faults, these “flapping” shouldn’t happen, that is what makes me think of a possible connector/contacts physical issue.
I totally agree
What has to be understood/tested is whether the device is actually “assigned” (or whatever) to the last slot or not.
I.e. the first time this happens you have:
set slot1 slot=slot1 type=hardware
and the second time you have:
set slot1 slot=slot2 type=hardware
The question is, does the modem work if you remove the first entry or if you remove the second (or both as this setting does not affect the modem)?
I’d like to do this, but actually idk what command can be used to delete one, several or all records at once.
Command /disk remove slot1 works fine, but now need to somehow force to remove all slots via the one command. I’ve sent a bug report and still waiting any help from the Mikrotik team.
Yea, last night I added [find] and it worked. But even if pull out the modem from the router and apply remove command, then after reboot the Slot1 still remain in the config. idk wtf ¯_(ツ)_/¯
Well, I’ll be use this temporary crutch until the developers fix it in future firmware..
Good.
If the good guys @ Mikrotik support give you a better idea/workaround or promise a fix in a next Ros release, please post it here.
I failed to find any reference to the ZTE MF79U being supported on Mikrotik devices, so it could well be a (minor) incompatibility of some kind, but since it seems like a common enough device, you are probably not the one and only one affected by the problem.
The reason is (confirmed by MikroTik support) the modem is detected also as a disk. If modem is re-plugged or router reboots, new slot entries are getting added. I have seen a maximum of ~1000 slots created.
Presumably because of this I got boot loop recently while trying to upgrade the router and had to perform netinstall.
My router is powered by mini UPS, so reboots because of main power loss are not possible. And before upgrading the device I unplug the modem.
I have opened an incident several months ago. There is no fix yet.
Yep, but there must be something else triggering the addition of a new slot, besides rebooting and unplugging and re-plugging the USB device.
In your case, reboots amount to 0 (zero) and you would need to unplug and re-plug the device 3 times a day, each day, for one year to reach 1000+ if this latter is the only cause.