Community discussions

MikroTik App
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Very high sector writes

Thu Jul 04, 2019 2:48 pm

Hi All: I'm running RouterOS 6.43.16 (long-term) on RB751g-2HnD and I'm seeing more than 7000 sector writes per day, I've opened webfig and checked the System->Resources menu and I can see how it writes 32 sectors every 6 minutes, approximately. Not only that, it almost reached 400000 sector writes after only 30 days last month, which I think it might be excessive, as nothing is supposed to be writing to disk at all. Just in case, I reset the router to default configuration and disabled DHCP writes to disk, there are no graphs running and logging is to memory only, but anyway the 32 writes every 6 minutes continue to happen, with over 7000 write after just 24 hours. This is my configuration in case anyone sees what could be causing this, unless it might be a bug.
/interface bridge
add admin-mac=D4:CA:6D:51:FB:51 auto-mac=no comment=defconf name=bridge
/interface wireless
set [ find default-name=wlan1 ] band=2ghz-b/g/n channel-width=20/40mhz-XX distance=indoors frequency=auto ht-supported-mcs=\
    mcs-0,mcs-1,mcs-2,mcs-3,mcs-4,mcs-5,mcs-6,mcs-7,mcs-8,mcs-9,mcs-10,mcs-11,mcs-12,mcs-13,mcs-14,mcs-15 mode=ap-bridge ssid=MikroTik-51FB55 wireless-protocol=802.11
/interface list
add comment=defconf name=WAN
add comment=defconf name=LAN
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/ip pool
add name=dhcp ranges=192.168.8.21-192.168.8.100
/ip dhcp-server
add address-pool=dhcp disabled=no interface=bridge lease-time=1d name=defconf
/tool user-manager customer
set admin access=own-routers,own-users,own-profiles,own-limits,config-payment-gw
/interface bridge port
add bridge=bridge comment=defconf interface=ether2
add bridge=bridge comment=defconf interface=ether3
add bridge=bridge comment=defconf interface=ether4
add bridge=bridge comment=defconf interface=ether5
add bridge=bridge comment=defconf interface=wlan1
/ip neighbor discovery-settings
set discover-interface-list=LAN
/interface list member
add comment=defconf interface=bridge list=LAN
add comment=defconf interface=ether1 list=WAN
/ip address
add address=192.168.8.1/24 comment=defconf interface=ether2 network=192.168.8.0
/ip cloud
set update-time=no
/ip dhcp-client
add comment=defconf dhcp-options=hostname,clientid disabled=no interface=ether1
/ip dhcp-server config
set store-leases-disk=never
/ip dhcp-server lease
add address=192.168.8.4 client-id=1:40:8d:5c:96:c1:24 mac-address=40:8D:5C:96:C1:24 server=defconf
/ip dhcp-server network
add address=192.168.8.0/24 comment=defconf gateway=192.168.8.1 netmask=24
/ip dns
set allow-remote-requests=yes
/ip dns static
add address=192.168.8.1 name=router.lan
/ip firewall filter
add action=accept chain=input comment="defconf: accept established,related,untracked" connection-state=established,related,untracked
add action=drop chain=input comment="defconf: drop invalid" connection-state=invalid
add action=accept chain=input comment="defconf: accept ICMP" protocol=icmp
add action=drop chain=input comment="defconf: drop all not coming from LAN" in-interface-list=!LAN
add action=accept chain=forward comment="defconf: accept in ipsec policy" ipsec-policy=in,ipsec
add action=accept chain=forward comment="defconf: accept out ipsec policy" ipsec-policy=out,ipsec
add action=fasttrack-connection chain=forward comment="defconf: fasttrack" connection-state=established,related
add action=accept chain=forward comment="defconf: accept established,related, untracked" connection-state=established,related,untracked
add action=drop chain=forward comment="defconf: drop invalid" connection-state=invalid
add action=drop chain=forward comment="defconf:  drop all from WAN not DSTNATed" connection-nat-state=!dstnat connection-state=new in-interface-list=WAN
/ip firewall nat
add action=masquerade chain=srcnat comment="defconf: masquerade" ipsec-policy=out,none out-interface-list=WAN
/system clock
set time-zone-name=America/Argentina/Salta
/system lcd
set contrast=0 enabled=no port=parallel type=24x4
/system lcd page
set time disabled=yes display-time=5s
set resources disabled=yes display-time=5s
set uptime disabled=yes display-time=5s
set packets disabled=yes display-time=5s
set bits disabled=yes display-time=5s
set version disabled=yes display-time=5s
set identity disabled=yes display-time=5s
set bridge disabled=yes display-time=5s
set wlan1 disabled=yes display-time=5s
set ether1 disabled=yes display-time=5s
set ether2 disabled=yes display-time=5s
set ether3 disabled=yes display-time=5s
set ether4 disabled=yes display-time=5s
set ether5 disabled=yes display-time=5s
/system logging
add topics=debug
/system package update
set channel=long-term
/system watchdog
set watchdog-timer=no
/tool graphing
set store-every=24hours
/tool mac-server
set allowed-interface-list=LAN
/tool mac-server mac-winbox
set allowed-interface-list=LAN
/tool user-manager database
set db-path=user-manager
Last edited by arielgrin on Thu Jul 04, 2019 9:23 pm, edited 1 time in total.
 
mkx
Forum Guru
Forum Guru
Posts: 5815
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector write

Thu Jul 04, 2019 2:54 pm

My guess: user manager ... try to attach an USB flash disk and configure user manager to store its database to USB stick.
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector write

Thu Jul 04, 2019 9:21 pm

Thanks @mkx for your reply.

I have disabled user-manager package, as I don't need it. I rebooted the router and will check again after a couple of hours to see if the writes have stopped.

Do you happen to see anything else that could be a cause for writes? User manager has been always enabled, but never used, and this write frequency just increased in the last couple of months, it was not like this before.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Thu Jul 04, 2019 10:41 pm

Indeed it seems it is not user-manager, or at least not only that. The router has been running for less than an hour and there are already 650 writes logged.
 
Pea
Member Candidate
Member Candidate
Posts: 228
Joined: Fri Jul 17, 2015 11:07 pm
Location: Czech

Re: Very high sector writes

Thu Jul 04, 2019 10:42 pm

Many DHCP leases? Try:
/ip dhcp-server config set store-leases-disk=never
Edit: I see you have this already, so it must be something else...
Last edited by Pea on Thu Jul 04, 2019 10:45 pm, edited 1 time in total.
 
mkx
Forum Guru
Forum Guru
Posts: 5815
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Thu Jul 04, 2019 10:45 pm

I wonder if this setting

/ip dhcp-server config
set store-leases-disk=never

really disables writing to flash. Try to set it to something like 6h and see if it makes any difference.
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 1:13 am

I will try changing the setting. I thought that store-leases-disk=never was the way to go to avoid leases being written to disk, but I will try other setting. In any case, there are only 5 leases and the lease time is 1 day, so the refresh frequency is really low, once every 12 hours for just 5 leases. The router has been running for 4 hours and it already has more than 2000 sector writes.
 
mkx
Forum Guru
Forum Guru
Posts: 5815
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Fri Jul 05, 2019 9:16 am

I thought that store-leases-disk=never was the way to go to avoid leases being written to disk

It should ... but there's always room for some bugs :wink:
BR,
Metod
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 1:20 pm

Well I changed the setting to store the leases every 24 hours, but the writes continue to raise. The router has been running for 12 hours and there are more than 4000 writes already. I don't know what else to check or do, in less than 2 months there have been more than 1 million writes. I think that is definitely excessive.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Fri Jul 05, 2019 10:09 pm

Router has been running for 36 hours and there are more than 20000 sector writes. I don't know what else to do. Should I contact support? I don't have a support contract or anything like that.
 
sindy
Forum Guru
Forum Guru
Posts: 7178
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Fri Jul 05, 2019 10:28 pm

As you have already tried a reset to default configuration, I would try to export (not backup) the configuration into a file, download the file to PC, netinstall the router, and check again with default configuration. If the ghost writes stop appearing, I'd re-import the configuration, otherwise I'd use some other RouterOS release (e.g. 6.43.15, as 6.43.16 only fixes some 60 GHz related issue).

support@mikrotik.com does accept information about well-documented real bugs even without a support contract, but they cannot deal with "how do I change the default IP address" class of questions, that's why they suggest this forum as the primary support channel and refer to the support from your reseller.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
2frogs
Long time Member
Long time Member
Posts: 647
Joined: Fri Dec 03, 2010 1:38 am

Re: Very high sector writes

Sat Jul 06, 2019 3:25 am

/system logging
add topics=debug
Have tried disabling this?
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Sat Jul 06, 2019 6:52 pm

All versions of RouterOS above 6.34.6 in the old line of devices from MikroTik cause an increased amount of write to flash memory.
RB450Gx4 / RB2011UiAS-2HnD
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Sun Jul 07, 2019 4:25 pm

That setting is no longer active, the router has been reset to factory defaults, but the writes keep on raising no matter what.
 
sid5632
Member
Member
Posts: 444
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Sun Jul 07, 2019 9:10 pm

Netinstall it then. I expect it's been compromised.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Mon Jul 08, 2019 3:37 pm

Will try netinstall and report. Just curious, compromised how? Do you mean hacked? Or something else?
 
sid5632
Member
Member
Posts: 444
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Mon Jul 08, 2019 7:08 pm

Yes, hacked.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Wed Jul 10, 2019 10:49 pm

I don't see how could it get hacked. There are no open ports nor forwarded ports on the wan interface. None of the admin tools, like webfig or ssh are open to the exterior. The only one using the router is me.
 
sid5632
Member
Member
Posts: 444
Joined: Fri Feb 17, 2017 6:05 pm

Re: Very high sector writes

Wed Jul 10, 2019 11:06 pm

It was only a guess based on almost zero information.
You will only find out if it's cured by Netinstalling it.
 
2frogs
Long time Member
Long time Member
Posts: 647
Joined: Fri Dec 03, 2010 1:38 am

Re: Very high sector writes

Wed Jul 10, 2019 11:18 pm

Most likely a partially failed update or some corruption in OS.
 
sindy
Forum Guru
Forum Guru
Posts: 7178
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Wed Jul 10, 2019 11:31 pm

There are no open ports nor forwarded ports on the wan interface. None of the admin tools, like webfig or ssh are open to the exterior. The only one using the router is me.
The times when this used to be enough to feel reasonably safe are unfortunately gone. I don't say it is the case here, but cross-platform malware does exist, which squats on a PC in your LAN to which it gets via connections permitted by the firewall of the router (such as browsing on infected web sites), and from there it starts malicious activity towards other machines on the LAN and also towards the router, as even security-aware users usually only do what you've described above, i.e. prevent attacks from WAN side but keep all managament access open for any source in LAN.
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Thu Jul 11, 2019 3:54 pm

Netinstall will not help you.
All versions above 6.34.6 have nand refresh.

What's new in 6.35 (2016-Apr-14 12:55):
*) nand - implemented once a week nand refresh to improve stored data integrity (will increase sector writes);
RB450Gx4 / RB2011UiAS-2HnD
 
sindy
Forum Guru
Forum Guru
Posts: 7178
Joined: Mon Dec 04, 2017 9:19 pm

Re: Very high sector writes

Thu Jul 11, 2019 4:36 pm

*) nand - implemented once a week nand refresh to improve stored data integrity (will increase sector writes);
20000 sector writes in 36 hours give something like 93000 sector writes per week. What's the NAND Flash size in your device?
Instead of writing novels, post /export hide-sensitive. Use find&replace in your favourite text editor to systematically replace all occurrences of each public IP address potentially identifying you by a distinctive pattern such as my.public.ip.1.
 
arielgrin
just joined
Topic Author
Posts: 12
Joined: Fri Jun 07, 2013 3:40 pm

Re: Very high sector writes

Thu Jul 11, 2019 6:33 pm

I have netinstalled 6.45.1 and the issue persists. But the writes are not done once a week, as in the "refresh" update, they are done every couple of minutes. After 1 hour there is already more than 300 writes. So the netinstall procedure hasn't fixed it. I noticed that even though leases are set to never write to disk, any time a lease is granted, renewed or removed, write count raises by 16. No matter what interval I use, be it never, 1 hour, 5 hours, 1 day, etc, any lease change causes sector writes immediately.
I have already contacted support, they are investigating, so I will keep this thread updated with any info that might help.
 
mn62
just joined
Posts: 3
Joined: Tue Feb 26, 2019 1:07 pm

Re: Very high sector writes

Fri Jul 12, 2019 10:54 pm

What's the NAND Flash size in your device?
128.0 MiB, RB2011UAS-2HnD

My device is old :-)
You do not have the required permissions to view the files attached to this post.
RB450Gx4 / RB2011UiAS-2HnD
 
User avatar
Anastasia
newbie
Posts: 45
Joined: Wed Oct 28, 2015 7:12 pm

Re: Very high sector writes

Thu Jan 16, 2020 3:37 pm

Hi All: I'm running RouterOS 6.43.16 (long-term) on RB751g-2HnD and I'm seeing more than 7000 sector writes per day
did you solve the problem? What have you done for this?
 
shiyiqiang08
Frequent Visitor
Frequent Visitor
Posts: 61
Joined: Wed Dec 05, 2018 7:35 am

Re: Very high sector writes

Sat Jan 18, 2020 8:11 am

do you enable the vpn service?
or you use vpn?
i find vpn will cause sector writting....
 
DiamondBA
just joined
Posts: 4
Joined: Sat Feb 09, 2019 11:36 pm

Re: Very high sector writes

Fri Apr 30, 2021 9:31 am

I have similar behaviour with my CAP ac since last couple releases (6.48.x). Used only as ap station with 3-5 clients, no DHCP, no graphing, no logging to disk.

It looks like SNTP client is causing writes to flash - at the moment installed 6.48.2, SNTP client on - 6-8K wrtes per 24h, SNTP client off - 1-1.5K writes per 24h.
I have 3 CAP ac (different configurations), but the ones with h/w revision v2 are not making high sector writes with SNTP on/off, only older model without revision has high sector writes.
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 3212
Joined: Tue Feb 25, 2014 12:49 pm
Location: Capalbio, Tuscany, Italy

Re: Very high sector writes

Fri Apr 30, 2021 10:38 am

From 4 July 2019 to now no one has guessed...

For now only this comes to my mind:

1) Last logged in on users are stored on NAND, cannot be disabled,
Just opening winbox or terminal
For example, just this are 2 writes

2) RouterBOARD do not have battery for RTC :((((
SNTP and NTP store time on NAND, it's happen often.
It's for set time at bootup as closest right time.
it cannot be disabled, also if SNTP and NTP is disabled

3) On DEFAULT, critical logs, like failed login access, are stored on NAND till 100 events, the newer clear the older, etc.
These logs are stored on NAND and displayed on terminal on open and removed just after displayed on terminal
for disble the DEFAULT:
/system logging action set echo remember=no

4) Terminal history:
Everytime you write something on terminal, that go to "history" of commands.
Is persistent and keeped on NAND after reboots
it cannot be disabled
hint: for clear history on terminal
/console clear-history

5) DHCP Leases are stored on default every 5 minutes on "disk"
For change the update interval:
/ip dhcp-server config set interim-update=00:05:00
For disable saving
/ip dhcp-server config set store-leases-disk=never

Sector write history count?
Yes... it's writed somewhere... ;))

EDIT: added point 5)
Last edited by rextended on Fri May 07, 2021 6:54 pm, edited 6 times in total.
I'm Italian, not English. Sorry for my imperfect grammar.
 
KayBur
just joined
Posts: 12
Joined: Thu Apr 29, 2021 3:33 pm
Location: Springfield

Re: Very high sector writes

Fri Apr 30, 2021 4:12 pm

I have similar behaviour with my CAP ac since last couple releases (6.48.x). Used only as ap station with 3-5 clients, no DHCP, no graphing, no logging to disk.

It looks like SNTP client is causing writes to flash - at the moment installed 6.48.2, SNTP client on - 6-8K wrtes per 24h, SNTP client off - 1-1.5K writes per 24h.
I have 3 CAP ac (different configurations), but the ones with h/w revision v2 are not making high sector writes with SNTP on/off, only older model without revision has high sector writes.
This might be a silly idea, but what if there simply isn't enough power to commit that much data? Or just an update bug.
 
DiamondBA
just joined
Posts: 4
Joined: Sat Feb 09, 2019 11:36 pm

Re: Very high sector writes

Tue May 04, 2021 2:21 pm


2) RouterBOARD do not have battery for RTC :((((
NTP store time on NAND, it's happen often.
It's for set time at bootup as closest right time.
it cannot be disabled, also if NTP is disabled
I can definitely confirm that my high sector writes occur when SNTP client enabled. With SNTP client disabled I have 600-800 writes per 24h now and nothing else is chagned in configuration to get writes dropped from 8K to 800 per day.

Whether time adjustments are causing other modules to create more NAND writes (kind of chain reaction) or SNTP client itself stores a lot of info on disk - that is question to MikroTik.
 
mkx
Forum Guru
Forum Guru
Posts: 5815
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Tue May 04, 2021 2:49 pm

If this indeed has anything to do with SNTP client, then it's NTP client from stand-alone ntp package guilty as well.
BR,
Metod
 
DiamondBA
just joined
Posts: 4
Joined: Sat Feb 09, 2019 11:36 pm

Re: Very high sector writes

Thu May 06, 2021 1:58 pm

If this indeed has anything to do with SNTP client, then it's NTP client from stand-alone ntp package guilty as well.
If I'm not mistaken, separate NTP package should be installed additionally (I did that recently to get NTP server running on my hex S), SNTP client is part of default packaging.
 
mkx
Forum Guru
Forum Guru
Posts: 5815
Joined: Thu Mar 03, 2016 10:23 pm

Re: Very high sector writes

Fri May 07, 2021 12:14 am

What I'm saying is that I also see enormous number of sector writes, but my devices all have the separate ntp package installed. AFAIK separate ntp package provides different ntp client than system package. So if it's ntp client that causes high sector writes, it's ntp client from separate package doing it as well.
BR,
Metod
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 3212
Joined: Tue Feb 25, 2014 12:49 pm
Location: Capalbio, Tuscany, Italy

Re: Very high sector writes

Fri May 07, 2021 6:54 pm

I'm Italian, not English. Sorry for my imperfect grammar.

Who is online

Users browsing this forum: akakua, Google [Bot], Guscht, Majestic-12 [Bot], marko1982m, mkx, musialny, thekev, unam83, Xgraver and 210 guests