Blacklist Import - File Size

RouterOS General
1

I’m trying to update my blacklist scripts to import the Firehol lists, levels 1, 2 & 3 separately.
I am finding that the resulting address lists in RouterOS are a long way short of having imported the complete list. (11952 entries where there should be approx 40k)
I believe I read somewhere else a file size limitation? Edit: Yes, this is definitely the issue.
If that’s the case, does anyone know of a sensible way around this?
I’m not that experienced with scripting but I am using a Windows server and Powershell to download and save the files to a local web server, as I block all unnecessary traffic on the RouterOS output chain…if I can do anything at that point? I’m aware that several individuals are hosting scripts on their own servers, but I’d rather not automate the use of an external script for obvious reasons.
Any recommendations for creating my own?

Where to get list of malicious hosts (sim to spamhaus dshield cymru torlist) and what can replace malwaredomainlist.com?
2

Powershell

Remove-Item "C:\inetpub\wwwroot\blacklists\firehol_L1.txt"
Remove-Item "C:\inetpub\wwwroot\blacklists\firehol_L2.txt"
Remove-Item "C:\inetpub\wwwroot\blacklists\firehol_L3.txt"
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
$WebClient = New-Object System.Net.WebClient
$WebClient.DownloadFile("https://iplists.firehol.org/files/firehol_level1.netset","C:\inetpub\wwwroot\blacklists\firehol_L1.txt")
$WebClient.DownloadFile("https://iplists.firehol.org/files/firehol_level2.netset","C:\inetpub\wwwroot\blacklists\firehol_L2.txt")
$WebClient.DownloadFile("https://iplists.firehol.org/files/firehol_level3.netset","C:\inetpub\wwwroot\blacklists\firehol_L3.txt")
Remove-Item "C:\inetpub\wwwroot\blacklists\firehol1.rsc"
Remove-Item "C:\inetpub\wwwroot\blacklists\firehol2.rsc"
Remove-Item "C:\inetpub\wwwroot\blacklists\firehol3.rsc"
$script= "C:\inetpub\wwwroot\blacklists\firehol1.rsc"
"/ip firewall address-list remove [/ip firewall address-list find list=firehol_L1]" | Add-Content $script
$blacklist= Get-Content "C:\inetpub\wwwroot\blacklists\firehol_L1.txt" | Where-Object {$_ -notmatch "^#|0.0.0.0|10.0.0.0|192.168.0.0|224.0.0.0"}
ForEach ($address in $blacklist){
"/ip firewall address-list add addres=$address list=firehol_L1 timeout=2d" | Add-Content $script
}
$script= "C:\inetpub\wwwroot\blacklists\firehol2.rsc"
"ip firewall address-list remove [/ip firewall address-list find list=firehol_L2]" | Add-Content $script
$blacklist= Get-Content "C:\inetpub\wwwroot\blacklists\firehol_L2.txt" | Where-Object {$_ -notmatch "^#|0.0.0.0|10.0.0.0|192.168.0.0|224.0.0.0"}
ForEach ($address in $blacklist){
"/ip firewall address-list add addres=$address list=firehol_L2 timeout=2d" | Add-Content $script
}
$script= "C:\inetpub\wwwroot\blacklists\firehol3.rsc"
"ip firewall address-list remove [/ip firewall address-list find list=firehol_L3]" | Add-Content $script
$blacklist= Get-Content "C:\inetpub\wwwroot\blacklists\firehol_L3.txt" | Where-Object {$_ -notmatch "^#|0.0.0.0|10.0.0.0|192.168.0.0|224.0.0.0"}
ForEach ($address in $blacklist){
"/ip firewall address-list add addres=$address list=firehol_L3 timeout=2d" | Add-Content $script
}

RouterOS

/tool fetch address=server host=server mode=https src-path=/blacklists/firehol1.rsc keep-result=yes ;
import firehol1.rsc ;
 /tool fetch address=server host=server mode=https src-path=/blacklists/firehol2.rsc keep-result=yes ;
import firehol2.rsc ;
 /tool fetch address=server host=server mode=https src-path=/blacklists/firehol3.rsc keep-result=yes ;
import firehol3.rsc ;

Note: If using IIS, you’ll need to add a MIME File Type for .rsc as text/plain.