Script has that permission. It looks like it is related to compression. If we instruct the web server to “no-gzip” the download works.
Ill open a ticket when its reproducible.
Edit:
The issue occurs when the http response headers + payload (after being uncompressed) exceeds 65535 bytes.
But what is really odd is that once in awhile it goes through. SUP-137742
After upgrade on hap ac2 I have zero space left and I can’t even make a backup. How I can save some space? In package list there is a routeros and wireless packages,
Unfortunately it is not only useless (for those that have no intention to use CapsMAN) but it also causes problems…
When you have configuration in winbox to customize windows, e.g. column layout, it is not handled correctly.
Probably the old and new wifi config windows use the same name or ID or whatever how that is handled in winbox.
hAP ac2 is notorious for lack of space. the best is to netinstall it and restore the backup you made BEFORE the upgrade…
You made a backup, right? It says in the first message above:
Before an upgrade:
Remember to make backup/export files before an upgrade and save them on another storage device;
And also consider the fact that wifi-qcom-ac uses around 200kB more flash space than wireless package. So if flash is running tight (and will be, believe me), then the solution might be to stick to wireless.
If one is using ARM ac device as router only (i.e. wireless is not required), then the new arrangement allows to fully uninstall wireless drivers and thus free a few MB of flash (wifi menu is still there as it contains necessary stuff to run CAPsMAN on same device … but that doesn’t require running hardware drivers).
When updating the firmware on a CCR2004-16G-2S+ the “upgrade firmware” string is different than the version shown under “current firmware” once applied. It looks like there is an extra carriage return in the string - so there always seems to be an update available (even if it has already been applied).
Yeah, but as I am locked to the old wireless drivers for now (until the VLAN issues in the new drivers are solved) and I have no intention to use CAPsMAN on that device, I still would prefer to be able to remove the new WiFi package… and keep routeros and wireless on that space-cramped device.
Probably a good thing. It prevents those running auto-upgrade scripts from upgrading in the first couple of hours after release.
The correct version will probably be returned a little later, when there has been no mayhem in this topic.
Some additional information relevant to this issue. It actually has nothing to do with upgrading. When I first experienced the same issue I misidentified it as related to licensing. Here is what I now know:
Only CHR is affected
It occurs when the CHR is rebooted after some extended period of time (weeks maybe?)
#2 is why it is mistaken as an upgrade problem, because you generally aren’t going to reboot your router until such time as you need to perform maintenance.
Like BWC, I was unable to obtain debugging info – especially since Google Compute Engine (GCE) (I suspect the same for AWS) does not output serial console activity during instance disk booting. I probably would have to try on an in-house host to be able to see what’s actually happening when it fails. However, GCE (and I believe also AWS) runs on KVM and we use Hyper-V internally; therefore, it’s debatable whether I could reproduce the issue in-house. It I get time in the new year (or I’m really bored over the holidays, I might setup an instance to test it out; although results could take weeks or months to achieve. Someone who runs Proxmox in-house would be a better candidate to attempt to reproduce the issue.
In the meantime, I reverted to 6.49.x, because I got tired of having to rebuild the router and it turned out that the automated snapshots also ended up with whatever the corruption is and couldn’t be booted.
Hope this info help set MikroTik/others on the right direction to identifying the root cause of this issue.
The same script was working fine under 7.12.1 and earlier versions
following is script
{
/ip firewall address-list
:local update do={
:put "Starting import of address-list: $listname"
:if ($nolog = null) do={:log warning "Starting import of address-list: $listname"}
:local maxretry 3
:local retrywaitingtime 120s
:local retryflag true
:for retry from=1 to=$maxretry step=1 do={
:if (retryflag) do={ :set $retryflag false; :set $sounter 0
:if (retry > 1) do={
:put "Source file changed. Retrying after a $retrywaitingtime wait..."
:if ($nolog = null) do={:log warning "Source file changed. Retring after a $retrywaitingtime wait..."}
:delay $retrywaitingtime }
:local filesize ([/tool fetch url=$url keep-result=no as-value]->"total")
:local start 0
:local maxsize 64000; # reqeusted chunk size
:local end ($maxsize - 1); # because start is zero the maxsize has to be reduced by one
:local partnumber ($filesize / ($maxsize / 1024)); # how many chunk are maxsize
:local remainder ($filesize % ($maxsize / 1024)); # the last partly chunk
:if ($remainder > 0) do={ :set $partnumber ($partnumber + 1) }; # total number of chunks
:if ($heirule != null) do={:put "Using as extra filtering: $heirule"} else={:set $heirule "."}
# remove the current list completely if "erase" is not present (default setting)
:if ($noerase = null) do={
:if ($timeout = null) do={:set $timeout 00:00:00; :do {:foreach i in=[/ip firewall address-list find list=$listname] do={/ip firewall address-list set list=("backup".$listname) $i }} on-error={} } else={
:do {:foreach i in=[/ip firewall address-list find list=$listname dynamic] do={/ip firewall address-list set list=("backup".$listname) $i }} on-error={} };
:put ("Conditional deleting all".$dynamic." entries in address-list: $listname")
:if ($nolog = null) do={:log warning ("Conditional deleting all".$dynamic." entries in address-list: $listname")}
} else={:put "Entries not conditional deleted in address-list: $listname"}; # ENDIF ERASE
:for x from=1 to=$partnumber step=1 do={
# get filesize to be compared to the orignal one and if changed then retry
:local comparesize ([/tool fetch url=$url keep-result=no as-value]->"total")
#:set $comparesize 5
# fetching the chunks from the webserver when the size of the source file has not changed
# empty array when the source file changed. No processing is done till the next complete retry
:if ($comparesize = $filesize) do={:set $data ([:tool fetch url=$url http-header-field="Range: bytes=$start-$end" output=user as-value]->"data")} else={:set $data [:toarray ""]; :set $retryflag true}
#:if ($ownposix = null) do={
# determining the used delimiter in the list if not provided in the config
# this only run once and so the impact on the import time is low
:local ipv4Posix "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}"
:local ipv4rangePosix "^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}/[0-9]{1,2}"
:local domainPosix "^.+\\.[a-z.]{2,7}"
:local sdata $data;
:while ([:len $sdata]!=0 && $delimiter = null) do={ # The check on length of $sdata is for if no delimiter is found.
:local sline [:pick $sdata 0 [:find $sdata "\n"]]; :local slen [:len $sline];
# set posix depending of type of data used in the list
:if ($sline ~ $ipv4Posix) do={:set $posix $ipv4Posix; :set $iden "List identified as a IPv4 list"}
:if ($sline ~ $ipv4rangePosix) do={:set $posix $ipv4rangePosix; :set $iden "List identified as a IPv4 with ranges list"}
:if ($sline ~ $domainPosix) do={:set $posix $domainPosix; :set $iden "List identified as a domain list"}
:if ($sline ~ $posix) do={:put $iden}
:if ($sline ~ $posix) do={ # only explore the line if there is match at the start of the line.
:do {:if ([:pick $sline 0 ($slen-$send)] ~ ($posix."\$") || $send > $slen) do={
:set $delimiter [:pick $sline ($slen-$send) ($slen-($send-1))]; :set $result true} else={:set $send ($send+1)}
} while (!$result);
}; #IF posix
:set $sdata [:pick $sdata ([:find $sdata "\n"]+1) [:len $sdata]];
:if ([:len $delimiter] = 0) do={ :set $delimiter "\n" }; # when empty use NewLine 20220529
:if ($delimiter != null) do={:local sdata [:toarray ""]} ; #Clear array sdata and it is not needed anymore and triggering so the While to end
}; #WHILE END $sdata
:local sdata [:toarray ""]
#} else={:put "User defind Posix: $ownposix"; :set $posix $ownposix } ; # ENDIF ownposix = null
:if ($delimiter = "\n") do={ :set $delimiterShow "New Line" } else={ :set $delimterShow $delimiter }; #20220529
:if ($posix = null && $delimiter != null) do={:set $posix "."; :put "Using config-line defined delimiter: \"$delimiterShow\""}; # delimter provided by config line
:if (!retryflag) do={:put "Reading Part: $x $start - $end"}
:if ($timeout = null) do={:local timeout 00:00:00}; # if no timeout is defined make it a static entry.
# Only remove the first line only if you are not at the start of list
:if ($start > 0) do={:set $data [:pick $data ([:find $data "\n"]+1) [:len $data]]}
:while ([:len $data]!=0) do={
:local line [:pick $data 0 [:find $data "\n"]]; # create only once and checked twice as local variable
:if ( $line ~ $posix && $line~heirule) do={
:do {add list=$listname address=[:pick $data 0 [:find $data $delimiter]] comment=$comment timeout=$timeout; :set $counter ($counter + 1)} on-error={}; # on error avoids any panics
}; # if IP address && extra filter if present
:set $data [:pick $data ([:find $data "\n"]+1) [:len $data]]; # removes the just added IP from the data array
# Cut of the end of the chunks by removing the last lines...very dirty but it works
:if ([:len $data] < 256) do={:set $data [:toarray ""]}
}; # while
:set $start (($start-512) + $maxsize); # shifts the subquential start back by 512
:set $end (($end-512) + $maxsize); # shift the subquential ends back by 512 to keep the
}; # if retryflag
}; #do for x
}; # for retry
:if ($counter < 1) do={:set $resultline "Import was NOT successfull! Check if the list $listname is still being maintained."} else={:set $resultline "Completed reading $counter items into address-list $listname." }
:put $resultline
:if ($nolog = null) do={:log warning $resultline }
:if ($counter > 0) do={:do {/ip firewall address-list remove [find where list=("backup".$listname)]} on-error={} } else={
:do {:foreach i in=[/ip firewall address-list find list=("backup".$listname)] do={/ip firewall address-list set list=$listname $i }} on-error={}
:put "Restoring backup list: $listname"
:if ($nolog = null) do={:log warning "Restoring backup list: $listname"}
}; # if counter restore on failure and remove on success
}; # do
$update url=https://view.sentinel.turris.cz/greylist-data/greylist-latest.csv listname=turris delimiter=, timeout=1d
}