Community discussions

MikroTik App
 
User avatar
martini
Member Candidate
Member Candidate
Topic Author
Posts: 296
Joined: Tue Dec 21, 2004 12:13 am

Cant read file large then 4085 bytes

Wed Apr 28, 2010 11:28 am

Try to use this script
:global content [/file get [/file find name=test.txt] contents] ;
:global contentLen [ :len $content ] ;

:global lineEnd 0;
:global line "";
:global lastEnd 0;


:do {
       :set lineEnd [:find $content "\r\n" $lastEnd ] ;
       :set line [:pick $content $lastEnd $lineEnd] ;
       :set lastEnd ( $lineEnd + 2 ) ;

       :local tmpArray [:toarray $line] ;
	:if ( [:pick $tmpArray 0] != "" ) do={
	:put $tmpArray;
         /ppp secret add name=[:pick $tmpArray 0] password=[:pick $tmpArray 1] \
             local-address=[:pick $tmpArray 2] remote-address=[:pick $tmpArray 3] \
             profile=[:pick $tmpArray 4] service=[:pick $tmpArray 5];
}
} while ($lineEnd < $contentLen)
But if file large then 4085 bytes nothing happens
Can somebody help with this ??
 
User avatar
mrz
MikroTik Support
MikroTik Support
Posts: 7042
Joined: Wed Feb 07, 2007 12:45 pm
Location: Latvia
Contact:

Re: Cant read file large then 4085 bytes

Wed Apr 28, 2010 11:38 am

This is current limitation, and there is no workaround
Next time use search:
http://forum.mikrotik.com/viewtopic.php ... it#p202842
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: Cant read file large then 4085 bytes

Wed Apr 28, 2010 11:45 am

If the file contents can not be read (due to file size limitation), an empty string is returned.

You could try adding this if file contents cannot be retrieved:
:global content [/file get [/file find name=test.txt] contents] ;
:global contentLen [ :len $content ] ;

+ :if ([:len $contentLen] = 0) do={
+    :error ("Could not retrieve test.txt contents")
+ }


:global lineEnd 0;
:global line "";
:global lastEnd 0;


:do {
       :set lineEnd [:find $content "\r\n" $lastEnd ] ;
       :set line [:pick $content $lastEnd $lineEnd] ;
       :set lastEnd ( $lineEnd + 2 ) ;

       :local tmpArray [:toarray $line] ;
   :if ( [:pick $tmpArray 0] != "" ) do={
   :put $tmpArray;
         /ppp secret add name=[:pick $tmpArray 0] password=[:pick $tmpArray 1] \
             local-address=[:pick $tmpArray 2] remote-address=[:pick $tmpArray 3] \
             profile=[:pick $tmpArray 4] service=[:pick $tmpArray 5];
}
} while ($lineEnd < $contentLen)
 
User avatar
dynek
Member Candidate
Member Candidate
Posts: 222
Joined: Tue Jan 21, 2014 10:03 pm

Re: Cant read file large then 4085 bytes

Fri Apr 15, 2016 3:11 pm

6 years later, 4kb is still the max size of files that can be read from a Mikrotik's script?
This sounds like April's Fool and makes me go back to what computers used to be 30 years ago.
 
jarda
Forum Guru
Forum Guru
Posts: 7756
Joined: Mon Oct 22, 2012 4:46 pm

Fri Apr 15, 2016 4:19 pm

Even 30 years ago there was not such low limit as I remember.
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: Cant read file large then 4085 bytes

Sat Apr 16, 2016 6:47 am

I believe it may have something to do with the buffering mechanism being used to fetch / stream data. At one point, I thought this limitation applied to comments as well, but I think that is no longer the case.

Regardless, it would be nice to at least see some streaming API, even if we an only buffer 4kb chunks, we could eventually read / write the entire contents of a file.
 
midenok
newbie
Posts: 39
Joined: Fri Dec 27, 2013 5:34 pm

Re: Cant read file large then 4085 bytes

Mon Apr 25, 2016 7:40 pm

Hey, this means I can't parse files??? No other ways to read by chunks? I want to parse http://txt.proxyspy.net/proxy.txt
 
dssmiktik
Forum Veteran
Forum Veteran
Posts: 732
Joined: Fri Aug 17, 2007 8:42 am

Re: Cant read file large then 4085 bytes

Tue Apr 26, 2016 5:54 am

Hey, this means I can't parse files??? No other ways to read by chunks? I want to parse http://txt.proxyspy.net/proxy.txt
Unfortunately not if the contents is > 4kb.
You could setup an external service to transform the data into a RouterOS string via local variable. This would allow you to consume the entire file, provided enough memory. If the file already exists on the router, you could use fetch to ship the file to the external service, and retrieve streaming data that way. This is not ideal, but well,it works.
 
midenok
newbie
Posts: 39
Joined: Fri Dec 27, 2013 5:34 pm

Re: Cant read file large then 4085 bytes

Tue Apr 26, 2016 8:37 am

Hey, this means I can't parse files??? No other ways to read by chunks? I want to parse http://txt.proxyspy.net/proxy.txt
Unfortunately not if the contents is > 4kb.
You could setup an external service to transform the data into a RouterOS string via local variable. This would allow you to consume the entire file, provided enough memory. If the file already exists on the router, you could use fetch to ship the file to the external service, and retrieve streaming data that way. This is not ideal, but well,it works.
Nah, much simpler is to run perl under metarouter. But I'm amazed how primitive and lame routerscript is. It literally can't do something a little bit more complex than 2+2. And if it by some star confluence can, then it will be big pain and shaman dances...
 
User avatar
macsrwe
Forum Guru
Forum Guru
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Re: Cant read file large then 4085 bytes

Fri May 13, 2016 10:10 am

:-x !!!

I just wasted five hours tonight writing a script to be periodically run on every one of our routers (including CPEs) to update common configuration scripts if newer versions of those scripts were available on a server.

Because FTP fetch doesn't preserve the date-modified of the source file, it had to depend on being able to determine if the contents of file A were or were not equal to the contents of file B. I tested this out before spending time writing the script, but the files I tried it out on were smaller files and so they worked correctly. I finished the entire script before I discovered that ROS always told me that the files were equal whether or not it was true. That's five hours wasted that I could have used to make progress on something else. And I'm stuck with the conclusion that there is absolutely no other way to do what it is I needed to do.
 
paramex
just joined
Posts: 4
Joined: Tue Nov 14, 2017 9:36 pm

Re: Cant read file large then 4085 bytes

Mon Nov 09, 2020 11:50 pm

put in folder /hotspot read by link
expmple :
your dns : 192.168.100.1 or langit.net
file : contoh.txt

type :http://langit.net/contoh.txt
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1070
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: Cant read file large then 4085 bytes

Tue Nov 10, 2020 5:24 pm

And I'm stuck with the conclusion that there is absolutely no other way to do what it is I needed to do.
Sure there is. 😜

The only limitation is for file size. This does not apply for script size (/system script) for example. So fetch your script to a variable, then store it as script (/system script add name=configuration-update source=$FetchedScript). Next run you compare the newly fetched script with the stored script.

I use something similar with my scripts collected, see my signature for a link. I can use script larger than 4k just fine.

(In fact I've though about a script to transfer and manage configuration updates... Did not touch it yet.)
 
msatter
Forum Guru
Forum Guru
Posts: 2897
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: Cant read file large then 4085 bytes

Tue Nov 10, 2020 7:19 pm

A file up to 64KB can be read in one go to an array.

.RSC files can be much larger and I hsve not found yes a limitation other than the memory of the router itself.

I am still searching for a way to include code as insert and not in script or global.
 
User avatar
macsrwe
Forum Guru
Forum Guru
Posts: 1007
Joined: Mon Apr 02, 2007 5:43 am
Location: Arizona, USA
Contact:

Re: Cant read file large then 4085 bytes

Tue Nov 10, 2020 10:32 pm

Next run you compare the newly fetched script with the stored script.
Here is where you just kicked the can down the road. The running script can't compare the two script files for same or different because of the variable size limitation -- precisely my original complaint.

My original comment was years old. I evaded the problem by just downloading the "new" script every night without bothering to care whether or not it had changed.
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1070
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: Cant read file large then 4085 bytes

Wed Nov 11, 2020 10:38 am

Here is where you just kicked the can down the road. The running script can't compare the two script files for same or different because of the variable size limitation -- precisely my original complaint.
Please read my post again. The limitation is not the variable size - it is just the reading from file. You can use more than 4k if you do not read from file.
My original comment was years old. I evaded the problem by just downloading the "new" script every night without bothering to care whether or not it had changed.
Missed that... And wondering if variable size changed since then.
What I described works since any time in 2018 at least.
 
User avatar
eworm
Forum Guru
Forum Guru
Posts: 1070
Joined: Wed Oct 22, 2014 9:23 am
Location: Oberhausen, Germany
Contact:

Re: Cant read file large then 4085 bytes

Wed Nov 11, 2020 10:54 am

Ah, you are right, it did not work back in 2016.
What I described requires fetching to variable, that was introduced with RouterOS 6.43 in late 2018.
 
PackElend
Member Candidate
Member Candidate
Posts: 269
Joined: Tue Sep 29, 2020 6:05 pm

Re: Cant read file large then 4085 bytes

Sat Apr 23, 2022 5:26 pm

And I'm stuck with the conclusion that there is absolutely no other way to do what it is I needed to do.
So fetch your script to a variable, then store it as script
by fetch you mean
/tool fetch
as
:global temp [/file get "01_scripts/testbig.rsc" contents]
fails as well?
 
PackElend
Member Candidate
Member Candidate
Posts: 269
Joined: Tue Sep 29, 2020 6:05 pm

Re: Cant read file large then 4085 bytes

Tue May 03, 2022 11:28 pm


So fetch your script to a variable, then store it as script
by fetch you mean
/tool fetch
as
:global temp [/file get "01_scripts/testbig.rsc" contents]
fails as well?
got it, for local (LAN only) transfer and deployment:
:global FetchedScript [/tool fetch url="ftp://10.10.9.1/01a_scripts_MGMT/test.rsc" user=ftp mode=ftp password=1234 port=21 host="" as-value  output=user ];
/system script add name=test-fetch-to-script source=$FetchedScript

Who is online

Users browsing this forum: stanisljevic and 23 guests