Community discussions

MikroTik App
 
LoveLife
just joined
Topic Author
Posts: 4
Joined: Sat Jul 29, 2017 4:32 pm

HELP with error in script to import the IPv4 full bogons list from www.team-cymru.org

Wed Oct 25, 2017 10:47 am

Hello,

I am a total novice at scripting and simply duplicated someone elses script (https://wiki.polaire.nl/doku.php?id=rou ... gon_update) but with a small change.

The change I've made is, to instead fetch the complete IP4 bogons list from http://www.team-cymru.org/Services/Bogo ... s-ipv4.txt on a nightly schedule which Team Cymru update every 4hrs or so.

The issue I am having is when I run the modified script and a duplicate IP range is detected in the txt file (which there must be I guess), the script fails and displays "failure: already have such entry" in the Terminal Window.

Can anyone suggest a work around to modify the following script so I can avoid this error and allow the import to continue on?

Ideally if a duplicate entry is found in the txt file and it tries to add it to the "bogons" address list it simply ignores adding it to the address list and moves on to the next line thus continue to import the rest of the bogons in the txt file.

NOTE: The first line in that txt file which is being fetched contains "# last updated 1508907008 (Wed Oct 25 04:50:08 2017 GMT)", would this cause an issue? If so could the first line be ignored?
## Builds an address list with bogons based on:
## http://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt

:log info "Fetching bogon list." 
/tool fetch url="http://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt" mode=http

:log info "Removing all bogons." 
/ip firewall address-list remove [/ip firewall address-list find list=bogons]

:global content [/file get [/file find name=fullbogons-ipv4.txt] contents] ;
:global contentLen [:len $content];

:global lineEnd 0;
:global line "";
:global lastEnd 0;

:log info "Adding bogons from file." 
:do {
       :set lineEnd [:find $content "\n" $lastEnd ] ;
       :set line [:pick $content $lastEnd $lineEnd] ;
       :set lastEnd ( $lineEnd + 1 ) ;

       :if ( $line != "" ) do={
         /ip firewall address-list add list=bogons address=$line;
       }
} while ($lineEnd < $contentLen)
I'm using this address list along with other firewall entries to try and avoid being subject to DARKNET troubles such as this https://www.theregister.co.uk/2017/10/2 ... ansomware/. It may help a little, then again it may not, but it's also nice to stop this type of traffic going out or in to the local networks.
 
texmeshtexas
Frequent Visitor
Frequent Visitor
Posts: 51
Joined: Sat Oct 11, 2008 11:17 pm

Re: HELP with error in script to import the IPv4 full bogons list from www.team-cymru.org

Wed Apr 21, 2021 11:51 pm

This does not work because reading the file to the variable is too large. Would work for much smaller file but the bogon list is now over 1300 entries and about 21KB in size.
 
msatter
Forum Guru
Forum Guru
Posts: 2093
Joined: Tue Feb 18, 2014 12:56 am
Location: Netherlands / Nīderlande

Re: HELP with error in script to import the IPv4 full bogons list from www.team-cymru.org

Thu Apr 22, 2021 1:25 am

This does not work because reading the file to the variable is too large. Would work for much smaller file but the bogon list is now over 1300 entries and about 21KB in size.
Less than 64KB file: viewtopic.php?f=9&t=152632&p=758435
Loving my freedom and so, no Twitter, no Facebook/Instagram/WhatsApp, no Apple and no Google/Alphabet, no Amazon/Cloudfront/AWS.

Running:
RouterOS 6.49Beta / Winbox 3.27 64bits
 
User avatar
rextended
Forum Guru
Forum Guru
Posts: 3212
Joined: Tue Feb 25, 2014 12:49 pm
Location: Capalbio, Tuscany, Italy

Re: HELP with error in script to import the IPv4 full bogons list from www.team-cymru.org

Thu Apr 22, 2021 3:02 am

Reordered and corrected some aspects...

:log info "Remove old bogon list"
/ip firewall address-list remove [find where list="bogons"]

:log info "Fetching bogon list"
:global content ([/tool fetch url="https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt" mode=https output=user as-value]->"data")

:global contentLen [:len $content]
:global lineEnd -1
:global line ""
:global lastEnd -1

:log info "Adding bogons from memory"
:do { :set lineEnd [:find $content "\n" $lastEnd]
      :set line [:pick $content $lastEnd $lineEnd]
      :set lastEnd ($lineEnd + 1)

      :if ([:typeof [:toip $line]] = "ip") do={

          /ip firewall address-list add list="bogons" timeout=1w address=$line

      } else={

          :do { :global xparse ([[:parse ":return $line"]])
                :if ([:typeof $xparse] = "ip-prefix") do={
				    /ip firewall address-list add list="bogons" timeout=1w address=$line
				}
              } on-error={}
      }

} while=($lineEnd < ($contentLen - 1))

:log info "Clear memory"
:set content
:set contentLen
:set lineEnd
:set line
:set lastEnd
:set xparse


eh eh eh eh.... :))

add only if is valid ip-prefix like 6.6.6.0/24 or if a single IP like 6.6.6.1 or for some reason on the file are wited 6.6.6.2/32
I'm Italian, not English. Sorry for my imperfect grammar.

Who is online

Users browsing this forum: Baidu [Spider] and 30 guests