Community discussions

MikroTik App
 
albertozumiani
just joined
Topic Author
Posts: 2
Joined: Tue Nov 20, 2018 6:29 pm

File transfer via SFTP fails

Tue Oct 12, 2021 6:59 pm

Dear all,
I have developed a script to take file from SD card installed on a "hex" to a remote SFTP server.
This is the script
:local url "sftp://sftp.XXXXXXXXXXX.it"
:local port 22
:local username "test-XXXXXX"
:local password "XXXXXXXXXX"

:local localpath "disk1"
:local remotepath "_temp"

:local logprefix "*** SFTP UPLOAD:"

:log info "$logprefix START"
:foreach file in=[/file find where name~"$localpath/"] do={
  :set $filepath [/file get $file name]
  :set $filename [:pick $filepath ([:find $filepath "/"] + 1) [:len $filepath]]
  :log info "$logprefix Found file: $filename"
  :log info "$logprefix Uploading file to: $url/$remotepath/$filename"
  :do {
    /tool fetch upload=yes url="$url/$remotepath/$filename" port=$port user="$username" password="$password" src-path="$filepath" duration=30s keep-result=yes ascii=no
  } on-error={:log error "Error uploading file"}
  :delay delay-time=1s
  :log info "$logprefix Delete file: $filepath"
  /file remove $filepath
  :delay delay-time=1s
}
:log info "$logprefix END"

It almost works but there are some troubles with SSH implementation on ROS, I think.
In particular, with ssh log enabled, I see often following log and transfer fails
Oct/12/2021 17:39:37 ssh,debug transport state: 0 --> 1
Oct/12/2021 17:39:37 ssh,debug transport state: 1 --> 2
Oct/12/2021 17:39:37 ssh,debug client version: SSH-2.0-CrushFTPSSHD
Oct/12/2021 17:39:37 ssh,debug transport state: 2 --> 3
Oct/12/2021 17:39:37 ssh,debug host key algo: ssh-rsa,rsa-sha2-512,ssh-dss,rsa-sha2-256
Oct/12/2021 17:39:37 ssh,debug kex algo: curve25519-sha256,rsa2048-sha256,curve25519-sha256@libssh.org,rsa1024-sha1,diffie-hellman-group18-sha512,diffie-hellman-group17-sha512,diffie-hellman-group16-sha512,diffie-hellman-group15-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp521,ecdh-sha2-nistp384,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,ecdh-sha2-nistp256,ext-info-s
Oct/12/2021 17:39:37 ssh,debug enc algo CS: aes256-ctr,aes128-ctr,aes256-gcm@openssh.com,aes192-ctr,3des-ctr,3des-cbc,blowfish-cbc,arcfour,arcfour128,arcfour256,aes128-gcm@openssh.com
Oct/12/2021 17:39:37 ssh,debug mac algo CS: hmac-md5,hmac-sha2-512-etm@openssh.com,hmac-sha2-512-96,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha1-etm@openssh.com,hmac-sha1,hmac-sha1-96,hmac-md5-etm@openssh.com,hmac-md5-96,hmac-ripemd160,hmac-ripemd160-etm@openssh.com
Oct/12/2021 17:39:37 ssh,debug comp algo CS: none,zlib
Oct/12/2021 17:39:37 ssh,debug packet follows: 0
Oct/12/2021 17:39:37 ssh,debug agreed on: diffie-hellman-group-exchange-sha256 ssh-rsa aes128-ctr aes128-ctr hmac-sha2-256 hmac-sha2-256 none none
Oct/12/2021 17:39:38 ssh,debug pki algorithm: ssh-rsa
Oct/12/2021 17:39:38 ssh,debug transport state: 3 --> 4
Oct/12/2021 17:39:38 ssh,debug requesting auth methods
Oct/12/2021 17:39:38 ssh,debug banner: SFTP Server Ready!
Oct/12/2021 17:39:38 ssh,debug
Oct/12/2021 17:39:38 ssh,debug auth methods: password,publickey,keyboard-interactive
Oct/12/2021 17:39:38 ssh,debug doing password authorization
Oct/12/2021 17:39:38 ssh,debug authorized
Oct/12/2021 17:39:38 ssh,debug transport state: 4 --> 5
Oct/12/2021 17:39:38 ssh,debug channel# 0 open requested
Oct/12/2021 17:39:38 ssh,debug channel #0 opened 0 w:0 p:84d0
Oct/12/2021 17:39:38 ssh,debug channel #0 peer window exceeded
Oct/12/2021 17:39:38 ssh,debug channel #0 peer window consumed: 9 left: fffffff7
Oct/12/2021 17:39:38 ssh,debug channel #0 peer window consumed: 2b left: ffffffcc
Oct/12/2021 17:39:38 ssh,debug channel #0 peer window 0x7fffcc
Oct/12/2021 17:39:38 ssh,debug peer disconnected : 2 Data length of 9 bytes exceeded available window space of 0 bytes.
Oct/12/2021 17:39:38 ssh,debug transport state: 5 --> 0
Oct/12/2021 17:39:38 ssh,debug closing connection: XXX.XXX.XXX.XXX:22 (7)


When file transfer succeeded I have following log
Oct/12/2021 17:39:45 ssh,debug transport state: 0 --> 1
Oct/12/2021 17:39:45 ssh,debug transport state: 1 --> 2
Oct/12/2021 17:39:45 ssh,debug client version: SSH-2.0-CrushFTPSSHD
Oct/12/2021 17:39:45 ssh,debug transport state: 2 --> 3
Oct/12/2021 17:39:45 ssh,debug host key algo: ssh-rsa,rsa-sha2-512,ssh-dss,rsa-sha2-256
Oct/12/2021 17:39:45 ssh,debug kex algo: curve25519-sha256,rsa2048-sha256,curve25519-sha256@libssh.org,rsa1024-sha1,diffie-hellman-group18-sha512,diffie-hellman-group17-sha512,diffie-hellman-group16-sha512,diffie-hellman-group15-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp521,ecdh-sha2-nistp384,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1,ecdh-sha2-nistp256,ext-info-s
Oct/12/2021 17:39:45 ssh,debug enc algo CS: aes256-ctr,aes128-ctr,aes256-gcm@openssh.com,aes192-ctr,3des-ctr,3des-cbc,blowfish-cbc,arcfour,arcfour128,arcfour256,aes128-gcm@openssh.com
Oct/12/2021 17:39:45 ssh,debug mac algo CS: hmac-md5,hmac-sha2-512-etm@openssh.com,hmac-sha2-512-96,hmac-sha2-512,hmac-sha2-256-etm@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha1-etm@openssh.com,hmac-sha1,hmac-sha1-96,hmac-md5-etm@openssh.com,hmac-md5-96,hmac-ripemd160,hmac-ripemd160-etm@openssh.com
Oct/12/2021 17:39:45 ssh,debug comp algo CS: none,zlib
Oct/12/2021 17:39:45 ssh,debug packet follows: 0
Oct/12/2021 17:39:45 ssh,debug agreed on: diffie-hellman-group-exchange-sha256 ssh-rsa aes128-ctr aes128-ctr hmac-sha2-256 hmac-sha2-256 none none
Oct/12/2021 17:39:46 ssh,debug pki algorithm: ssh-rsa
Oct/12/2021 17:39:46 ssh,debug transport state: 3 --> 4
Oct/12/2021 17:39:46 ssh,debug requesting auth methods
Oct/12/2021 17:39:46 ssh,debug banner: SFTP Server Ready!
Oct/12/2021 17:39:46 ssh,debug
Oct/12/2021 17:39:46 ssh,debug auth methods: password,publickey,keyboard-interactive
Oct/12/2021 17:39:46 ssh,debug doing password authorization
Oct/12/2021 17:39:46 ssh,debug authorized
Oct/12/2021 17:39:46 ssh,debug transport state: 4 --> 5
Oct/12/2021 17:39:46 ssh,debug channel# 0 open requested
Oct/12/2021 17:39:46 ssh,debug channel #0 opened 0 w:0 p:84d0
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window exceeded
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window consumed: 9 left: fffffff7
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window consumed: 2b left: ffffffcc
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window 0x7fffcc
Oct/12/2021 17:39:46 ssh,debug channel #0 local window consumed: 6e left: 27ff92
Oct/12/2021 17:39:46 ssh,debug channel #0 check window 0x27ff92 of 280000
Oct/12/2021 17:39:46 ssh,debug sftp server version: 3
Oct/12/2021 17:39:46 ssh,debug channel #0 local window consumed: 29 left: 27ff69
Oct/12/2021 17:39:46 ssh,debug channel #0 check window 0x27ff69 of 280000
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window consumed: 8e9 left: 7ff6e3
Oct/12/2021 17:39:46 ssh,debug channel #0 peer window consumed: 29 left: 7ff6ba
Oct/12/2021 17:39:46 ssh,debug channel #0 local window consumed: 35 left: 27ff34
Oct/12/2021 17:39:46 ssh,debug channel #0 check window 0x27ff34 of 280000
Oct/12/2021 17:39:46 ssh,debug channel #0 local window consumed: 2c left: 27ff08
Oct/12/2021 17:39:46 ssh,debug channel #0 check window 0x27ff08 of 280000
Oct/12/2021 17:39:46 ssh,debug transport state: 5 --> 0
Oct/12/2021 17:39:46 ssh,debug closing connection: XXX.XXX.XXX.XXX:22 (7)
I do not know how to solve tnat because on ROS I have no control over "window space".
Any suggestion?

Best regards
 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Fri Feb 25, 2022 3:08 pm

We also see this Problem with Error:
peer disconnected : 2 Data length of 9 bytes exceeded available window space of 0 bytes.

Sometimes its working but most times not. Also we see that its better working when you have Debug Logging on! Can someone from MT Team help us here?
 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Tue Mar 08, 2022 9:31 am

No one?
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: File transfer via SFTP fails

Tue Mar 08, 2022 1:46 pm

What are you trying to send?
Its easy to setup gmail sending of files.
 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Wed Mar 09, 2022 9:33 am

I just wanna Upload Backup Files. .backup und .rsc, nothing special...
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: File transfer via SFTP fails

Wed Mar 09, 2022 1:01 pm

See this script made by me and rexteded.
viewtopic.php?p=915725#p915725

Do send both binary and readable backup to email
 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Mon Mar 14, 2022 9:35 am

Thx, but with this i sent the Backups with Mail. I want to Upload them with SFTP to my SFTP Server not with Mail :(
 
holvoetn
Forum Guru
Forum Guru
Posts: 5325
Joined: Tue Apr 13, 2021 2:14 am
Location: Belgium

Re: File transfer via SFTP fails

Mon Mar 14, 2022 10:30 am

Have you already made a report to support@mikrotik.com ?
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: File transfer via SFTP fails

Mon Mar 14, 2022 11:22 am

 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Mon Mar 14, 2022 5:32 pm

Thats not my Problem. Its generally working BUT in most times its just create 0 KB Files and end with "peer disconnected : 2 Data length of 9 bytes exceeded available window space of 0 bytes."
 
Jaggl
just joined
Posts: 24
Joined: Mon Aug 27, 2012 3:00 pm

Re: File transfer via SFTP fails

Mon Mar 14, 2022 5:38 pm

Have you already made a report to support@mikrotik.com ?
No, but i just did it :=)
 
texmeshtexas
Member Candidate
Member Candidate
Posts: 151
Joined: Sat Oct 11, 2008 11:17 pm

Re: File transfer via SFTP fails

Thu May 05, 2022 4:34 am

This is the script I use to do this very thing.
:log info "Automated Backup Started"
:delay 2s
:log info "Creating Backup Files. This may take several minutes."
:local sysname [/system identity get name]
:local textfilename
:local backupfilename
:local time [/system clock get time];
:local date [/system clock get date];
:local newdate "";
:local FTPservername "backup.xxxxxx.com";
:local FTPusername "xxxxxxx";
:local FTPpassword "xxxxxx";
:local i;

:for i from=0 to=([:len $date]-1) do={ :local tmp [:pick $date $i];
:if ($tmp !="/") do={ :set newdate "$newdate$tmp" }
:if ($tmp ="/") do={}
}
:if ([:find $sysname " "] !=0) do={
:local name $sysname;
:local newname "";
:for i from=0 to=([:len $name]-1) do={ :local tmp [:pick $name $i];
:if ($tmp !=" ") do={ :set newname "$newname$tmp" }
:if ($tmp =" ") do={ :set newname "$newname_" }
}
:set sysname $newname;
}
:set textfilename ($"newdate" . "-" . $"sysname" . ".rsc")
:set backupfilename ($"newdate" . "-" . $"sysname" . ".backup")
:execute [/export file=$"textfilename"]
:execute [/system backup save  name=$"backupfilename"]
:delay 5s
:log info "Sending files to sFTP server"
/tool fetch url="sftp://$FTPservername/routerback/$textfilename" src-path="$textfilename" user=$FTPusername password=$FTPpassword upload=yes ascii=yes keep-result=yes mode=ftp
/tool fetch url="sftp://$FTPservername/routerback/$backupfilename" src-path="$backupfilename" user=$FTPusername password=$FTPpassword upload=yes ascii=yes keep-result=yes mode=ftp
:log info "Sending Completed"
:log info "Removing Backup files from router file list"
/file remove ($textfilename)
/file remove ($backupfilename)
:log info "Automated Backup Completed"
 
User avatar
Jotne
Forum Guru
Forum Guru
Posts: 3279
Joined: Sat Dec 24, 2016 11:17 am
Location: Magrathean

Re: File transfer via SFTP fails

Thu May 05, 2022 8:05 am

Cleaned up the script some.
Removed ; from half of the script. Not needed at end of line. Only when multiple commands are on same line.
Open up the do= commands to better see the loops.
Added tabs to better see the loops.
:log info "Automated Backup Started"
:delay 2s
:log info "Creating Backup Files. This may take several minutes."
:local sysname [/system identity get name]
:local textfilename
:local backupfilename
:local time [/system clock get time]
:local date [/system clock get date]
:local newdate ""
:local FTPservername "backup.xxxxxx.com"
:local FTPusername "xxxxxxx"
:local FTPpassword "xxxxxx"
:local i

:for i from=0 to=([:len $date]-1) do={
	:local tmp [:pick $date $i]
	:if ($tmp !="/") do={
		:set newdate "$newdate$tmp"
	}
	:if ($tmp ="/") do={}
}
:if ([:find $sysname " "] !=0) do={
	:local name $sysname
	:local newname ""
	:for i from=0 to=([:len $name]-1) do={
		:local tmp [:pick $name $i]
		:if ($tmp !=" ") do={
			:set newname "$newname$tmp"
		}
		:if ($tmp =" ") do={
			:set newname "$newname_"
		}
	}
	:set sysname $newname
}
:set textfilename ($"newdate" . "-" . $"sysname" . ".rsc")
:set backupfilename ($"newdate" . "-" . $"sysname" . ".backup")
:execute [/export file=$"textfilename"]
:execute [/system backup save  name=$"backupfilename"]
:delay 5s
:log info "Sending files to sFTP server"
/tool fetch url="sftp://$FTPservername/routerback/$textfilename" src-path="$textfilename" user=$FTPusername password=$FTPpassword upload=yes ascii=yes keep-result=yes mode=ftp
/tool fetch url="sftp://$FTPservername/routerback/$backupfilename" src-path="$backupfilename" user=$FTPusername password=$FTPpassword upload=yes ascii=yes keep-result=yes mode=ftp
:log info "Sending Completed"
:log info "Removing Backup files from router file list"
/file remove ($textfilename)
/file remove ($backupfilename)
:log info "Automated Backup Completed"

Who is online

Users browsing this forum: aoravent, totpbi, UkRainUa and 19 guests