Community discussions

MikroTik App
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Backup Script cannot upload files

Sat Dec 21, 2019 5:38 pm

Hi there to all, I would like you to ask for your help, because I am facing a strange problem.

We were using to have DUDE SERVER on a Virtual Machine Running on VMWARE.
Sometimes the dude db get corrupted due to the snapshot operation while VEEAM makes the backup of that VM, so we decided to move DUDE SERVER on a RB1100AH Dude Edition.

We moved the DB without any problem, set DUDE SERVER on disk1 because internal memory has no space and DUDE started working without problems.

On the old CHR machine we had a script running all days that take backups of rsc and dudedb and move it to an ftp server.
The original script does not work anymore because it was pointing to files in "/" folder, so I modified the script to point to the rigth folder "/disk1/".

I was able to make it write backups on the rigth destination, but when the script reach line 45 and should upload to the ftp it goes on error
It seems it cannot find the files in the disk1 folder.

I am going to attach the original and the modified scirpt.

Would you be nice to help me finding where I am doing wrong?

Tnx to all
You do not have the required permissions to view the files attached to this post.
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Sat Dec 21, 2019 10:32 pm

Any Idea????
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Sun Dec 22, 2019 12:33 pm

OK OK 309 views, and nobody has an answer...

I think I am gonna move back to the chr our dude...
I cannot leave it without a backup ...

I have been working on it all day on yesterday, but it's really embarassing I can't find what's wrong with it...

Does anybody know some debugging tool for MIKROTIK Scripting language?
I use noepad++ to highligth the code, but there's anything to debug what you are doing, so it's pretty impossible...
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Backup Script cannot upload files

Sun Dec 22, 2019 7:20 pm

I check your script's in later time but about debug of scripts they are:
* run multiple times with changes
* enter in script code more IF, do {...} on-error={...}, put "I reach reach this line in script", log warning "This chapter script work.." means all this are "CheckPoint's" :)
* enter script into terminal and check error's.

I think I use those and not remember other in this fast reply.
BR Marcin
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Mon Dec 30, 2019 11:56 am

Hi @SiB did you succeed in checking my script?

I did not find a solution
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Backup Script cannot upload files

Thu Jan 02, 2020 1:33 am

First, You are a autor of the script? If YES then you just use few put/log $variable to describe a problem. Forus is not offitial support channel and any person who help here do it in his free time/resource etc and we have Christmast time :).

In my scenario I not use TheDude at MikroTik and must do checking on new device.
hAP ac2 with pendrive give be a disk1 type=disk, in file print I see path syntax "disk1/directory/file1.txt" .

I use https://text-compare.com/ to compare both scripts.
Your main/core part is this:
:foreach backupFile in=[/file find] do={
  :set backupFileName ($PathPrefix.[/file get ($PathPrefix.$backupFile) name])
  :if ([:typeof [:find $backupFileName $sfname]] != "nil") do={
    /tool fetch address=$FTPServer port=$FTPPort src-path=($backupFileName) user=$FTPUser mode=ftp password=$FTPPass dst-path="BackupMikrotik/TheDudeServer/$backupFileName" upload=yes
  }
}
TIPs:
* My first tip is to every time when you use fetch then use it in separate { } block. For example then you run fetch after fetch after fetch (3x fetch) you notice that can brake transfers, when you use separate block of code then command will be works in step by step instruction and the stdout not breake next command.
* Good way is use do {tool fetch ....} on-error={log error "My error entry for $backupFile <- backupFile variable ."}
* This foreach loop and have a put/log instructions to give you information what a current file are transfer, error's can be better grab that way.
* Tool/Fetch cannot copy a directory type. Remote Path must exist before copy files. Good way is display local directory by:
/file print where name~"disk1/dude" type=directory
disk1/dude
disk1/dude/files
disk1/dude/files/default
disk1/dude/files/default/images
disk1/dude/files/default/mibs
and compare with remote ftp if they exist.

This give us info that your "$PathPrefix.$backupFile" are double the disk1 like "disk1/disk1/dir/file.ext" and is not proper way.
I thinks this place is core problem of not executive script.
And not putting a additional debuging line in script.

Check your loop with my debug's.
{
:local backupFileName ""
:local PathPrefix "disk1/"
:foreach backupFile in=[/file find] do={
  put "SIB | backupFile: $backupFile"
  put "CheckPoint1"
  :set backupFileName ($PathPrefix.[/file get ($PathPrefix.$backupFile) name])
  put "CheckPoint2"
  do {:put "SIB | backupFileName: $backupFileName"} on-error={put "SIB | backupFileName: ERROR"}
  do {:put "SIB | cmd1:"; :put [/file get ($PathPrefix.$backupFile) name]} on-error={put "SIB | cmd1: ERROR"}
  
  #:if ([:typeof [:find $backupFileName $sfname]] != "nil") do={
    #/tool fetch address=$FTPServer port=$FTPPort src-path=($backupFileName) user=$FTPUser mode=ftp password=$FTPPass dst-path="BackupMikrotik/TheDudeServer/$backupFileName" upload=yes
  #}
}
}
Add/Remove the # at lines.
Copy/Paste with some changes to terminal with clear screen via CTRL+L and Shift+Insert
give you answers where you have errors in script.

Next problem is with your $fname and $sfname and this again have own disk1 suffix again and remote location and local not have got disk1/disk1/disk1/dude/... structure.

Example who copy "disk1/dude/" to remote FTP "BackupMikrotik/TheDudeServer/disk1/dude/..."
{
local backupFileName ""
foreach backupFile in=[/file find where name~"disk1/dude/" type!="directory" size>0 ] do={
  #put "File ID:  $backupFile"
  set $backupFileName [/file get $backupFile name]
  put "*** $backupFileName ***"
  do {/tool fetch address=$FTPServer mode=ftp port=21 upload=yes user=$FTPUser password=$FTPPass src-path=($backupFileName) dst-path="BackupMikrotik/TheDudeServer/$backupFileName"} on-error={put "FTP ERROR with $backupFileName"}
  }
}
I hope with all this information you can re-write your script.
BR Marcin
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Thu Jan 02, 2020 1:55 pm

Thank you SiB for your HELP.
I will give a try to all your tips.

I am sure that the problem is the filename, but did not know how to debug it.

I am goingto try whit thos put commands to monitor the name during the process.

Unfortunately I am not the authour of the script, I found it on this forum many months ago.
I am using this script to backup everything in our infrastructure.
The only one that I cannot find a Solution is this RB.

Thank you for your patience and for your TIME!!!
I know well that people in the forum answer in thei free time and this is not a MT Offical support page.!
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Backup Script cannot upload files

Thu Jan 02, 2020 2:12 pm

You use UM, TheDude and I not understand what all files copy this script.
If your provide more details like /file print of your mtk then it's easier me to work at case.
Currently I can give you answer as above.

If this is your main backup script then good way is start deasseble it to small parts, with comments, put $variableX, run from terminal like system script run name to see all stdout from put command, and be sure this must be done later or not because a ROS is change his CLI every update you do and some day some command reach other parameters.

BR Marcin
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Thu Jan 02, 2020 3:45 pm

The script run a backup of rsc, bin and dudedb.
Then takes all the files with name starting "Backup-" and upload them to ftp server

at the end, it deletes all files with prefix "Backup-"

this works great, and We could make a backup repository for all our devices running this script daily or weekly.
Many times it saved our lives...

Here the problem is the internal drive that changes something in terms of Absolute Path... So I had to modify it in order to write backup files (especially dudedb) on disk1.
The scripts write in the default memory, which is the internal 128M drive, but dudedb now is 400M so it runs, the file is written without any errors filling the internal memory, and then deleted after the FTP Upload.

I realized it because, when I was checking if the script ran, I noticed that the dudedb file was only 95M insted of 400.

So I decided to modify this script.
 
User avatar
dpal80
newbie
Topic Author
Posts: 33
Joined: Tue Apr 30, 2013 5:38 pm
Location: Italy
Contact:

Re: Backup Script cannot upload files

Fri Jan 03, 2020 5:32 pm

Hey There.... I Have Finally come to an END!!!

Thanks to SiB suggestions I put some log around the script and discovered that when the ftp SERVER was uploading the filename was disk1/something.backup for both src and dst file.
So it was unable to upload the file because in the dst folder there was no disk1 folder.

I just added a folder named disk1 in the destination and the script run flawessly without any problem.

It's running since 3 days....

Now I would like to enhance this script to introduce some checks and email alerts.

For example, I would like to monitor free space on DISK and send an email when the disk is 80% used.
Then I would like to handle some error catching events during file export so that if something goes wrong the RB sends back an email to the admin.

If someone has some starting ideas on how to do this, they are welcome!!


Tnx to all!
 
User avatar
SiB
Forum Guru
Forum Guru
Posts: 1888
Joined: Sun Jan 06, 2013 11:19 pm
Location: Poland

Re: Backup Script cannot upload files

Fri Jan 03, 2020 9:12 pm

Hey There.... I Have Finally come to an END!!!
Thanks to SiB suggestions I put some log around the script and discovered that when the ftp SERVER was uploading the filename was disk1/something.backup for both src and dst file.
So it was unable to upload the file because in the dst folder there was no disk1 folder.
Share a corrected version.
.
Now I would like to enhance this script to introduce some checks and email alerts.
For example, I would like to monitor free space on DISK and send an email when the disk is 80% used.
Then I would like to handle some error catching events during file export so that if something goes wrong the RB sends back an email to the admin.
Those are differ stuff. Please first search this forum, Scripting section about them.
Create a new post and link here.

Who is online

Users browsing this forum: JDF and 20 guests