Page 1 of 1

Low disk space while disk is almost free

Posted: Tue May 22, 2018 11:23 pm
by upower3
I got two CCR-1009, both have flash disk partitioned into two parts (with different ROS versions, "just in case"). Now the're on 6.42.1 both.

Since they have 128 Mb of flash disk built-in, and that disk deviced into 2 partitions, the "current" partition is 64 Mb of size. The partition in fact has only few text files on it (config backups), roughtly of 500 Kb in total (as far as I can count it). But when I try to do /system package update install, it says there is no room to store the file downloaded, that I only have 10 Mb free while the file is 15 Mb in size.

So the question is, is this normal? The same is on the second router, so looks like the situation is steady.

Another question is how can I upgrade ROS anyway? The routers are on remote location so no netinstall possible. I would say I need to do some 'checkdisk' on the disk but no way to do that on ROS, right?

I also don't understand the overall space consumption on ROS: looks like installed ROS files are uncompressed and hidden, so in 64 Mb disk there are some like 50 Mb of uncompressed "system" files and no room to store compressed .npk file while downloading it?

Re: Low disk space while disk is almost free

Posted: Wed May 23, 2018 12:12 am
by pe1chl
"Normal" for a CCR-1009 is a disk space usage of about 40MB.
So when you partition your flash in 2 partitions there should be about 24MB free.
This is enough to do an upgrade.
I have heard before that in some conditions there could be files on the filesystem that are unaccounted for and the usual "solution" being recommended is to netinstall the device.
I agree that this is not optimal when the router is remote.
It would be better when there was some way to fix this kind of problem without netinstall, e.g. some (small) package that cleans up the filesystem.

In your case you could also consider to re-partition the router back to 1 partition so you can upgrade and then change it to 2 partitions again.
With some luck it could free the space...

Re: Low disk space while disk is almost free

Posted: Wed May 23, 2018 12:17 am
by upower3
Yep, I was wrong, the total disk is 64 Mb. But mine (half) is still has only 10 Mb free. Why?

Re: Low disk space while disk is almost free

Posted: Wed May 23, 2018 11:12 pm
by upower3
In your case you could also consider to re-partition the router back to 1 partition so you can upgrade and then change it to 2 partitions again.
With some luck it could free the space...
If I just repartition it to 1 partition router asks me to reboot and I suspect I'll loose all the disk and need to use netinstall which is hard to do. Can you please advice what happens when I repartition the disk back from 2 to one partition, will the router be able to boot anyway?

Re: Low disk space while disk is almost free

Posted: Thu May 24, 2018 1:16 am
by pe1chl
Actually I never tried that... I know that when there is 1 partition and you change it to 2, the result is that there are 2 partitions after reboot and the 1st still has the same RouterOS+config as the original disk had. The 2nd is empty and you can then copy the 1st to the 2nd.
I would think when going from 2 to 1, the 2nd is lost and the entire disk has the same content as the 1st had.
However, I am not sure and you should not risk that when there is no way to recover it.

Re: Low disk space while disk is almost free

Posted: Thu May 24, 2018 7:16 pm
by Sob
Try asking MikroTik support, I don't think anyone else can help you if you want to avoid netinstall. They should be able to access disk if they want. And they might be interested to know what happened.

I had the same problem with CHR. It may be related, unfortunately almost for sure not useful for you, but anyway...

I have testing CHR with 64MB disk, which is kept on RouterOS current branch. Upgrade to 6.42.1 was successful, but next upgrade to 6.42.2 complained about low disk space and there really was under 10MB free. I took the virtual disk, connected it to Linux VM and found out that aside from ~10MB Dude database, most disk space was taken by two files (~20MB each), /var/pdb/routeros-x86/image and /bootimage. Assuming that one of them is there probably by mistake and not needed, I deleted the latter, RouterOS survived and happily upgraded to next version.

Re: Low disk space while disk is almost free

Posted: Sat May 26, 2018 10:10 pm
by upower3
Did the repartition from 2 to only 1 partition and it worked, I got whole free space on one partiton, so I was able to do the ROS upgrade from 6.42.1 to 6.42.3.

But even after upgrade I found there are 44.7 Mb out of 128 Mib used, so looks like there is still some hidden data on the disk, abd I have no way to inspect it.

The ROS filesystem is some kind of cryptic and not easy to check from user's point.

Re: Low disk space while disk is almost free

Posted: Sun May 27, 2018 12:47 pm
by pe1chl
This is normal disk space usage on a CCR. I usually see around 40 MB used. Apparently the upgrade has solved your problem.
Now you can partition back to 2 partitions and have 20MB free which is enough to do an upgrade.

Re: Low disk space while disk is almost free

Posted: Sun May 27, 2018 12:50 pm
by upower3
This is normal disk space usage on a CCR. I usually see around 40 MB used. Apparently the upgrade has solved your problem.
Thank you for this confirmation. There is no official page on that so I was unsure.
I think I will do the repartiton but then again, may I ever need more disk space in the future for problem like this?

Re: Low disk space while disk is almost free

Posted: Sun May 27, 2018 12:58 pm
by pe1chl
I manage 2 CCR1009 routers and both are partitioned in 2 (I copy the current part before upgrading for easy rollback)
and I have never seen the problem you had. It probably was caused by an interrupted upgrade or some other one-time
problem. When you want to be really sure there is nu luring problem it would be best to netinstall it the next time you
visit the site for some other reason. Of course first make a backup and restore that, or when you don't use certificates,
public keys and other things that are not present in an export it would be even safer to /export the config to file,
download the file to your laptop, then netinstall the router without any config and then connect the router via
mac-telnet/mac-winbox and import that exported config into the empty router config (first check that it is empty by
doing /export to the screen and remove whatever is there).
Advantage of /export is that you can see exactly what is in there and be certain that you are not again loading something
that is not good.

Re: Low disk space while disk is almost free

Posted: Mon May 28, 2018 9:33 pm
by pe1chl
I updated one of my CCR1009 and I run into the same problem: before update on 6.42.1 there was 24.5 MB free, after update to 6.42.3 there is only 11.5 MB free.
So another update not possible.
I will mention it in the current version topic.

Re: Low disk space while disk is almost free

Posted: Tue May 29, 2018 12:16 am
by upower3
I bit buggy update script I suppose. I did that mentioned on that version topic and noone seems to notice.

Hope they fix it in 6.42.3+.

Re: Low disk space while disk is almost free

Posted: Wed Jun 27, 2018 12:39 am
by bda
Hi,

I have all this problems on more than 15 CCR, and more than 50 litle boxes, like hEX. Even RB850Gx2 - affected...
Support telling me to netinstall, but I can not do it across our country!!! Weird!

Update: 6.42.1 to 6.42.4 - and we have less than 4.9 mbyte on some 16bm platforms, and 10-12 mbytes on 64mb partitions... HELP!

Re: Low disk space while disk is almost free

Posted: Wed Jun 27, 2018 12:49 am
by upower3
HELP!
Do the repartition (if you use it) to one partition only(this helped me). If this is not your case, bad luck then, netinstall I suppose.

Re: Low disk space while disk is almost free

Posted: Wed Jun 27, 2018 4:13 am
by Sob
I'm sure that by now they know exactly what caused it and how it could be fixed. So they really should release some tiny fixer package for affected versions. Because asking possibly many thousands of people, who didn't do anything wrong and only trusted their supposedly stable branch, to netinstall routers, if they want to upgrade ever again, is not reasonable at all.

Re: Low disk space while disk is almost free

Posted: Wed Jun 27, 2018 7:14 am
by bda
HELP!
Do the repartition (if you use it) to one partition only(this helped me). If this is not your case, bad luck then, netinstall I suppose.
Repartition does not help... Netinstall - great, but I have many device far from me, as I said - across our country...

Re: Low disk space while disk is almost free

Posted: Wed Jun 27, 2018 10:54 am
by pe1chl
Yes it is a terrible problem - but MikroTik seem to ignore it and think that netinstall is a good solution.
I have one CCR in a datacenter that is difficult to access (have to rely on other people) and one at work which should not be worked on during office hours.
Normally no problem to update them from home, but for netinstall a lot of arrangements have to be made :( :(
For such "trivial" problems (likely just forgotten to delete some large file) there should be a fix package that can be installed in the small remaining diskspace
to make space for the next normal update.

Re: Low disk space while disk is almost free

Posted: Thu Jul 12, 2018 10:03 am
by jcmlanzas
Fortunately i could upgrade by copying back the image from the backup image i had to the current active one.
Once reloaded. Free space came back to "normal" (around 40Mb) and i could make it.
The way mikrotik is managing the space and filesystems on devices i quite disastrous.
I installed a 2Gb SD on my CCR1009 and i couldn't make it functional yet :S

Re: Low disk space while disk is almost free

Posted: Thu Jul 12, 2018 11:08 am
by pe1chl
There is now a fix package that works, I used it to reclaim the space on two CCR1009s.
See the topics about 6.42.5 and 6.43RC

Re: Low disk space while disk is almost free

Posted: Thu Jul 12, 2018 11:32 am
by upower3
See the topics about 6.42.5 and 6.43RC
I wasn't able to find these, it this a package that included into new version (btw, this is dangerous "current", not more stable "bugfix", isn't it?), which is not possible to install if the space it used already by ROS tmp files, or some small package that can tidy up everything around beforw new ROS be installed?

Re: Low disk space while disk is almost free

Posted: Thu Jul 12, 2018 5:12 pm
by mkx
See the topics about 6.42.5 and 6.43RC
I wasn't able to find these, it this a package that included into new version (btw, this is dangerous "current", not more stable "bugfix", isn't it?), which is not possible to install if the space it used already by ROS tmp files, or some small package that can tidy up everything around beforw new ROS be installed?
Among other places it was in Anouncements - thread about 6.42.5. Near the end of first page.
It's the later (small package to reclaim lost space).

Re: Low disk space while disk is almost free

Posted: Mon May 27, 2019 8:21 pm
by neoprogger
Searched for so long - THANK YOU for linking this, I was about to netinstall my coreswitch just for this.
You saved me some work - cheers!

Re: Low disk space while disk is almost free

Posted: Tue May 28, 2019 1:26 am
by upower3
You can imagine how "happy" I was when first hit the disk space problem mysqlf (two brand new CCRs at the same time!) and the fix was good in time for me. Sad to umderstand how many people were urged to find their steps to fix it before the package was published.

Re: Low disk space while disk is almost free

Posted: Tue May 28, 2019 2:34 am
by pe1chl
Well, I used the first version of the repair package and it destroyed the partition it was run on.
Thanks to the fallback to second partition on boot failure it came back online. Pfffew.
Fortunately the newer fixed package was released soon and I could repair it.