Community discussions

 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Very high sector writes

Thu Jul 04, 2019 2:48 pm

Hi All: I'm running RouterOS 6.43.16 (long-term) on RB751g-2HnD and I'm seeing more than 7000 sector writes per day, I've opened webfig and checked the System->Resources menu and I can see how it writes 32 sectors every 6 minutes, approximately. Not only that, it almost reached 400000 sector writes after only 30 days last month, which I think it might be excessive, as nothing is supposed to be writing to disk at all. Just in case, I reset the router to default configuration and disabled DHCP writes to disk, there are no graphs running and logging is to memory only, but anyway the 32 writes every 6 minutes continue to happen, with over 7000 write after just 24 hours. This is my configuration in case anyone sees what could be causing this, unless it might be a bug.
/interface bridge
add admin-mac=D4:CA:6D:51:FB:51 auto-mac=no comment=defconf name=bridge
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX distance=indoors frequency=auto ht-supported-mcs=\
    mcs-0,mcs-1,mcs-2,mcs-3,mcs-4,mcs-5,mcs-6,mcs-7,mcs-8,mcs-9,mcs-10,mcs-11,mcs-12,mcs-13,mcs-14,mcs-15 mode=ap-bridge ssid=MikroTik-51FB55 wireless-protocol=802.11
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=192.168.8.21-192.168.8.100
/ip dhcp-server
add address-pool=dhcp disabled=no interface=bridge lease-time=1d name=defconf
/tool user-manager customer
set admin access=own-routers,own-users,own-profiles,own-limits,config-payment-gw
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=wlan1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/ip address
add address=192.168.8.1/24 comment=defconf interface=ether2 network=192.168.8.0
/ip cloud
set update-time=no
/ip dhcp-client
add comment=defconf dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server config
set store-leases-disk=never
/ip dhcp-server lease
add address=192.168.8.4 client-id=1:40:8d:5c:96:c1:24 mac-address=40:8D:5C:96:C1:24 server=defconf
/ip dhcp-server network
add address=192.168.8.0/24 comment=defconf gateway=192.168.8.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.8.1 name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/system clock
set time-zone-name=America/Argentina/Salta
/system lcd
set contrast=0 enabled=no port=parallel type=24x4
/system lcd page
set time disabled=yes display-time=5s
set resources disabled=yes display-time=5s
set uptime disabled=yes display-time=5s
set packets disabled=yes display-time=5s
set bits disabled=yes display-time=5s
set version disabled=yes display-time=5s
set identity disabled=yes display-time=5s
set bridge disabled=yes display-time=5s
set wlan1 disabled=yes display-time=5s
set ether1 disabled=yes display-time=5s
set ether2 disabled=yes display-time=5s
set ether3 disabled=yes display-time=5s
set ether4 disabled=yes display-time=5s
set ether5 disabled=yes display-time=5s
/system logging
add topics=debug
/system package update
set channel=long-term
/system watchdog
set watchdog-timer=no
/tool graphing
set store-every=24hours
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool user-manager database
set db-path=user-manager
Last edited by arielgrin on Thu Jul 04, 2019 9:23 pm, edited 1 time in total.
 
mkx
Forum Guru
Forum Guru
Posts: 2918
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector write

Thu Jul 04, 2019 2:54 pm

My guess: user manager ... try to attach an USB flash disk and configure user manager to store its database to USB stick.
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector write

Thu Jul 04, 2019 9:21 pm

Thanks @mkx for your reply.

I have disabled user-manager package, as I don't need it. I rebooted the router and will check again after a couple of hours to see if the writes have stopped.

Do you happen to see anything else that could be a cause for writes? User manager has been always enabled, but never used, and this write frequency just increased in the last couple of months, it was not like this before.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Thu Jul 04, 2019 10:41 pm

Indeed it seems it is not user-manager, or at least not only that. The router has been running for less than an hour and there are already 650 writes logged.
 
Pea
Member Candidate
Member Candidate
Posts: 191
Joined: Fri Jul 17, 2015 11:07 pm
Location: Czech

Re: Very high sector writes

Thu Jul 04, 2019 10:42 pm

Many DHCP leases? Try:
/ip dhcp-server config set store-leases-disk=never
Edit: I see you have this already, so it must be something else...
Last edited by Pea on Thu Jul 04, 2019 10:45 pm, edited 1 time in total.
 
mkx
Forum Guru
Forum Guru
Posts: 2918
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Thu Jul 04, 2019 10:45 pm

I wonder if this setting

/ip dhcp-server config
set store-leases-disk=never

really disables writing to flash. Try to set it to something like 6h and see if it makes any difference.
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 1:13 am

I will try changing the setting. I thought that store-leases-disk=never was the way to go to avoid leases being written to disk, but I will try other setting. In any case, there are only 5 leases and the lease time is 1 day, so the refresh frequency is really low, once every 12 hours for just 5 leases. The router has been running for 4 hours and it already has more than 2000 sector writes.
 
mkx
Forum Guru
Forum Guru
Posts: 2918
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Fri Jul 05, 2019 9:16 am

I thought that store-leases-disk=never was the way to go to avoid leases being written to disk

It should ... but there's always room for some bugs :wink:
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 1:20 pm

Well I changed the setting to store the leases every 24 hours, but the writes continue to raise. The router has been running for 12 hours and there are more than 4000 writes already. I don't know what else to check or do, in less than 2 months there have been more than 1 million writes. I think that is definitely excessive.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 10:09 pm

Router has been running for 36 hours and there are more than 20000 sector writes. I don't know what else to do. Should I contact support? I don't have a support contract or anything like that.
 
sindy
Forum Guru
Forum Guru
Posts: 3809
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Fri Jul 05, 2019 10:28 pm

As you have already tried a reset to default configuration, I would try to export (not backup) the configuration into a file, download the file to PC, netinstall the router, and check again with default configuration. If the ghost writes stop appearing, I'd re-import the configuration, otherwise I'd use some other RouterOS release (e.g. 6.43.15, as 6.43.16 only fixes some 60 GHz related issue).

support@mikrotik.com does accept information about well-documented real bugs even without a support contract, but they cannot deal with "how do I change the default IP address" class of questions, that's why they suggest this forum as the primary support channel and refer to the support from your reseller.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
2frogs
Long time Member
Long time Member
Posts: 540
Joined: Fri Dec 03, 2010 1:38 am

Re: Very high sector writes

Sat Jul 06, 2019 3:25 am

/system logging
add topics=debug
Have tried disabling this?
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Sat Jul 06, 2019 6:52 pm

All versions of RouterOS above 6.34.6 in the old line of devices from MikroTik cause an increased amount of write to flash memory.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Sun Jul 07, 2019 4:25 pm

That setting is no longer active, the router has been reset to factory defaults, but the writes keep on raising no matter what.
 
sid5632
Member
Member
Posts: 351
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Sun Jul 07, 2019 9:10 pm

Netinstall it then. I expect it's been compromised.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Mon Jul 08, 2019 3:37 pm

Will try netinstall and report. Just curious, compromised how? Do you mean hacked? Or something else?
 
sid5632
Member
Member
Posts: 351
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Mon Jul 08, 2019 7:08 pm

Yes, hacked.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Wed Jul 10, 2019 10:49 pm

I don't see how could it get hacked. There are no open ports nor forwarded ports on the wan interface. None of the admin tools, like webfig or ssh are open to the exterior. The only one using the router is me.
 
sid5632
Member
Member
Posts: 351
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Wed Jul 10, 2019 11:06 pm

It was only a guess based on almost zero information.
You will only find out if it's cured by Netinstalling it.
 
2frogs
Long time Member
Long time Member
Posts: 540
Joined: Fri Dec 03, 2010 1:38 am

Re: Very high sector writes

Wed Jul 10, 2019 11:18 pm

Most likely a partially failed update or some corruption in OS.
 
sindy
Forum Guru
Forum Guru
Posts: 3809
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Wed Jul 10, 2019 11:31 pm

There are no open ports nor forwarded ports on the wan interface. None of the admin tools, like webfig or ssh are open to the exterior. The only one using the router is me.
The times when this used to be enough to feel reasonably safe are unfortunately gone. I don't say it is the case here, but cross-platform malware does exist, which squats on a PC in your LAN to which it gets via connections permitted by the firewall of the router (such as browsing on infected web sites), and from there it starts malicious activity towards other machines on the LAN and also towards the router, as even security-aware users usually only do what you've described above, i.e. prevent attacks from WAN side but keep all managament access open for any source in LAN.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Thu Jul 11, 2019 3:54 pm

Netinstall will not help you.
All versions above 6.34.6 have nand refresh.

What's new in 6.35 (2016-Apr-14 12:55):
*) nand - implemented once a week nand refresh to improve stored data integrity (will increase sector writes);
 
sindy
Forum Guru
Forum Guru
Posts: 3809
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Thu Jul 11, 2019 4:36 pm

*) nand - implemented once a week nand refresh to improve stored data integrity (will increase sector writes);
20000 sector writes in 36 hours give something like 93000 sector writes per week. What's the NAND Flash size in your device?
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Thu Jul 11, 2019 6:33 pm

I have netinstalled 6.45.1 and the issue persists. But the writes are not done once a week, as in the "refresh" update, they are done every couple of minutes. After 1 hour there is already more than 300 writes. So the netinstall procedure hasn't fixed it. I noticed that even though leases are set to never write to disk, any time a lease is granted, renewed or removed, write count raises by 16. No matter what interval I use, be it never, 1 hour, 5 hours, 1 day, etc, any lease change causes sector writes immediately.
I have already contacted support, they are investigating, so I will keep this thread updated with any info that might help.
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Fri Jul 12, 2019 10:54 pm

What's the NAND Flash size in your device?
128.0 MiB, RB2011UAS-2HnD

My device is old :-)
You do not have the required permissions to view the files attached to this post.

Who is online

Users browsing this forum: No registered users and 76 guests