Hello.
There are situations when you store some data in RAM disk (for faster access or to avoid flash wear) and this data should be saved to permanent storage when the router is rebooted. Currently, you can only copy the files manually or create a "SafeReboot" script, which will do the job and you need to run it each time before rebooting the router. But you may just forget to do it and it won't work when you update RouterOS using 'Download&Install' for example.
To solve this problem, a system should execute a script each time it's going to reboot or shutdown. So, it will be possible to run the code to save all the data. Important note is that this script should also have a "feedback" from the code to check if it was executed successfully or not. In case there is an error, reboot/shutdown should be aborted. It could be implemented by checking the value of some variable that will be set by the user in the code. For example, let's call it $ExecResult. If it's true, reboot will continue, if false, it will be aborted. If there is some error during execution of the code itself, it should also be aborted. That's how I see it.
Please consider implementing this.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
Still with this approach you will have issue when ROS is not soft rebooted/shutdown due to power loss for example.
If you have data which needs to be available on boot in any reboot/shutdown case then it must be on persistent storage, use external or network drive if you are concerned about internal flash wear-out. Network drive can be faster than slow USB. If you have many read operations, for even faster read you can still use combination with tmpfs and persistent storage where on file create/update/delete on persistent storage, such file then can be copied to or deleted from tmpfs; and read files always from tmpfs at same relative path. Also some startup script can be created to just copy all from persistent storage to tmpfs. In such case you will have persistence and fast read IO.
IMO better to implement ROS feature for persistent global variables (eg. some new ROS section in /system/script as key-value storage persistant in ROS config) with option to have secret flag on them (if is set item can be exported only with show-sensitve and shown with masked value on UI) instead need for writing them into files or into some ROS config item like Layer7 or PPP secret.
If you have data which needs to be available on boot in any reboot/shutdown case then it must be on persistent storage, use external or network drive if you are concerned about internal flash wear-out. Network drive can be faster than slow USB. If you have many read operations, for even faster read you can still use combination with tmpfs and persistent storage where on file create/update/delete on persistent storage, such file then can be copied to or deleted from tmpfs; and read files always from tmpfs at same relative path. Also some startup script can be created to just copy all from persistent storage to tmpfs. In such case you will have persistence and fast read IO.
IMO better to implement ROS feature for persistent global variables (eg. some new ROS section in /system/script as key-value storage persistant in ROS config) with option to have secret flag on them (if is set item can be exported only with show-sensitve and shown with masked value on UI) instead need for writing them into files or into some ROS config item like Layer7 or PPP secret.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
Having UPS to avoid power losses is a user's responsibility, it's obvious.Still with this approach you will have issue when ROS is not soft rebooted/shutdown due to power loss for example.
If you have data which needs to be available on boot then it must be on persistent storage, use external or network drive if you are concerned about internal flash wear-out. Network drive can be faster than slow USB. If you have many read operations, for even faster read you can still use combination with tmpfs and persistent storage where on file create/update/delete on persistent storage, such file then can be copied to or deleted from tmpfs; and read files always from tmpfs at same relative path. Also some startup script can be created to just copy all from persistent storage to tmpfs. In such case you will have persistence and fast read IO.
IMO better to implement ROS feature for persistent global variables (eg. some new ROS section in /system/script as key-value storage persistant in ROS config) with option to have secret flag on them (if is set item can be exported only with show-sensitve and shown with masked value on UI) instead need for writing them into files or into some ROS config item like Layer7 or PPP secret.
Yes, the case is using a combination of tmpfs/persistent storage. It's possible to transfer the data from storage to RAM on startup, and I'm already using a script to do this, but there is no any mechanism to do vice versa in case of system reboot/shutdown. You need to do it manually in current implementation.
Re: Feature request: Run script before system reboot
Currently to avoid any manual job you can when creating/modifying/deleting file on persistent storage, copy it to or delete from tmpfs from same script, it is redundant operation but it will work, it will have more write operations on persistent storage FS than only doing it only on shutdown, but is more safe. Startup script will do the sync on next boot.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
Manual job in my case is copying from tmpfs to persistent storage before reboot, nothing else. I'm not doing anything on storage itself, only copying from storage to RAM on startup (by script, automatically) and vice versa on reboot/shutdown (manually). And currently this job cannot be avoided in any way rather than adding a feature for running a script before reboot/shutdown. Making backups each 10 minutes is not a solution either, because in such case there is no any sense in using tmpfs.
Re: Feature request: Run script before system reboot
Why is such issue to write files on persistent storage? How many writes per sec you have for such files? For eg. I have USB thumb drive for container sotrage, Pi-hole in container is doing log writes, adlist db refresh every day, etc. It's running for years w/o issues, when it fails I will buy another drive and restore files from backup...
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
In my case it's about 20-30 GB/day. But with all respect, the topic was created to solve a specific data management problem when using RAM for storing the data, it's not about choosing storage type.
There are different reasons when you can't or shouldn't use HDDs, it's not only high writing rates and faster access speeds. There are even situations when you simply don't have a HDD and this cannot be solved as simple as "go to the shop and buy it"... Moreover, USB HDD connection is not very reliable by itself, it may suddenly fail because of cable issues like bad contact/oxidation/dirt etc., resulting in data corruption. In such and other cases, internal RAM disk is a solution.
There are different reasons when you can't or shouldn't use HDDs, it's not only high writing rates and faster access speeds. There are even situations when you simply don't have a HDD and this cannot be solved as simple as "go to the shop and buy it"... Moreover, USB HDD connection is not very reliable by itself, it may suddenly fail because of cable issues like bad contact/oxidation/dirt etc., resulting in data corruption. In such and other cases, internal RAM disk is a solution.
Re: Feature request: Run script before system reboot
I'm aware of your request and somehow sceptic that it will be implemented, that's why I'm tring to figure out reasouns why you cannot use persistant storage and find workarounds.
20-30 GB/day is not much for external SSD/NVMe drive, for USB thumb drive can be. Yes, connection can be issue if this is outdoor device, if not I will not worry, other option can be network drive and at the end everything can fail at certain point, even UPS if you relay on it and expecting that storage will be at always at sync because of it.
20-30 GB/day is not much for external SSD/NVMe drive, for USB thumb drive can be. Yes, connection can be issue if this is outdoor device, if not I will not worry, other option can be network drive and at the end everything can fail at certain point, even UPS if you relay on it and expecting that storage will be at always at sync because of it.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
Well, it's not only for solving my own problems, it may solve similar problems for others. BTW, I've seen similar requests somewhere on the web already.
I won't agree with your statement about SSDs, 30GB/day it very much for them, though it depends on brand and type. Regarding USB connections, it's not something I worry about, it's a problem I faced with many times in indoor environment. The fact that everything can fail doesn't mean that you should ignore potential failing points, you should reduce them as much as possible. Personally, I was already losing the data for stupid reasons like failed HDD or poorly connected cable. It was very painful, and I don't want to repeat these mistakes. But it's another topic.
I won't agree with your statement about SSDs, 30GB/day it very much for them, though it depends on brand and type. Regarding USB connections, it's not something I worry about, it's a problem I faced with many times in indoor environment. The fact that everything can fail doesn't mean that you should ignore potential failing points, you should reduce them as much as possible. Personally, I was already losing the data for stupid reasons like failed HDD or poorly connected cable. It was very painful, and I don't want to repeat these mistakes. But it's another topic.
Re: Feature request: Run script before system reboot
Some reading about SSD's lifespan: https://www.ionos.com/digitalguide/serv ... life-span/. Regarding unreliable connections, that could happen if you often connect/disconnect drive from device, but I guess that should not be the case for non portable device like router, unless it is some Road warrior or LTE router which is used for travel, but I doubt such routers will need do such amount of file writes with rsc script.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
From my own experience, it's opposite. Most failures were with stationary cables that are never touched. I could even explain it, but we are already far from the topic.Regarding unreliable connections, that could happen if you often connect/disconnect drive from device
Re: Feature request: Run script before system reboot
That's strange, unless your devices are placed in area where earthquakes are common, affected with other ambient vibrations or in rooms with high moisture, but then I will be generally worried about devices health, not just connections.
-
-
teslasystems
Frequent Visitor
- Posts: 77
- Joined:
Re: Feature request: Run script before system reboot
That's strange, unless your devices are placed in area where earthquakes are common, affected with other ambient vibrations or in rooms with high moisture, but then I will be generally worried about devices health, not just connections.
No, usual office building. There is nothing strange actually. In such connections dust and oxidation are gradually growing. Finally, this leads to bad contact and failures. But when you connect/disconnect regularly, these factors are mostly eliminated by friction.
We need to stop offtopic.
Re: Feature request: Run script before system reboot
Dusty place it is, there a ways to handle that also 
Ok, let's be on topic... <nothing from me to add more on that>
Ok, let's be on topic... <nothing from me to add more on that>