Page 1 of 1

A script to look up IP to RIR using ARIN WhoIs

Posted: Wed Oct 12, 2011 12:58 pm
by TealFrog
This is my first MikroTik script. I wrote it primarily to learn MT scripting. Originally I wrote the script to monitor log entries and then take action, i.e. block probing IP addresses. I changed the script to use /ip firewall address-list created lists instead. As designed and written the script is meant to determine the Regional Internet Registry (RIR) of IP addresses. The IP list the script uses are IP address flagged by the firewall rules as having exhibited suspect or hostile behavior. Without modifying the script's default RIR list, the script will determine if the IP address is part of ARIN. If it is another RIR owned IP address, then it will be add the IP to the output list. In my implementation the output address list is used to block access from those IP addresses. I'm posting the script here in hopes that someone else may find it or portions of the script useful. I believe the script can be modified rather easily to perform other types of whois based look ups.

The script as written uses an IP address list created by IP firewall filtering rules. The firewall rules are used to detect potential probing against the MT router. The firewall rules add the IP addresses to an address list to be further examined. The script is scheduled to run periodically, for example once every hour. When executed the script takes the "/ip firewall address list" as input ($ipInList) and outputs a filtered address list ($ipOutList) based on the five Regional Internet Registries (RIRs) by using ARIN's whois. Based on a list of specified RIRs ($strNetReg), the script outputs the IPs that match the given RIRs to the specified ($ipOutList) address list. The output list of IP addresses can then be further processed or some other action can be taken such as blocking.

Some notes regarding script usage. The default behavior of the script is to delete the input address list. Keep this in mind if the input is used for any other purposes. ARIN refers to IP addresses registered to itself as being direct registers, so the word "Direct" represents ARIN in the script. I did not write the script just for ARIN, but ARIN is used by the default settings. The script also has an array that can be used to not process (ignore) certain IP addresses (the IP addresses used by default are more as an example than anything). The script provides debugging levels using the variable $debugLevel. If $debugLevel is zero, then no additional output is provided. If set to a number from 1 to 3, then the script will provide additional output.

Regards
#
# V 1.0 - Originally written for ROS v5.7
# IMPORTANT: $progName should be set to script file name on ROS
# Script takes an /ip firewall address-list and does ARIN whois and creates
# another filtered list based on ARIN registrar old list is removed by default
#
:local progName "LookupProbes";
:local ipInList "probe_list";
:local ipOutList "banned_list";
:local outFile "whois-out";
:local arrIgnoreIps {"0.0.0.0";"127.0.0.1";"255.255.255.255"};
:local blnRemoveOldList true;
# The following can be modified if a different whois server is used
:local whoServer "whois.arin.net";
:local uriPath "/rest/ip/";
:local uriSuffix ".txt";
:local eolMark "\n";
# String used as array to determine the blocking of network allocations
# NOTE: Next line represent valid ARIN registrar, "Direct" represents ARIN allocated
#:local strNetReg "Direct,ARIN,RIPE,APNIC,LACNIC,AfriNIC";
# Example of valid registrars to match and take action
:local strNetReg "RIPE,APNIC,LACNIC,AfriNIC";
# Generally no modification beyond this point should be necessary
:local arrIdList;
:local arrIpIgnoreList {};
:local arrNetReg;
:local blnFound false;
:local curDate [/system clock get date];
:local curTime [/system clock get time];
:local content;
# Debug level zero (0) is off, 1-3 verbose levels 3=most
:local debugLevel 0;
:local ignoreIp false;      
:local ipAddr;
:local ipArrList {};
:local ipProbeAddress;
:local lastEnd 0;
:local line;
:local lineEnd 0;
# MAIN PROGRAM START
:put "$progName: Running...";
:set arrNetReg [ :toarray $strNetReg ];
:set arrIdList [ :toarray [ /ip firewall address find where list=$ipInList ] ];
:if ($debugLevel > 1) do={
    :put "Read $[ :len arrIdList ] items from firewall address list $ipInList.";
}
:if ($arrIdList = "") do={
    :put "No probes found";
} else={
# Before continuing make a master list of IP addresses to ignore
# First add previously flagged IP addresses from firewall address list
    foreach ipListId in [ /ip firewall address-list find where list=$ipOutList ] do={
        :set ipAddr [ /ip firewall address-list get $ipListId address ];
        :set arrIpIgnoreList ( $arrIpIgnoreList, [ :toip $ipAddr ] );
    }
# Add IPs from script ignore list    
    :foreach ipAddr in=$arrIgnoreIps do={
        :if ($debugLevel > 1) do={
            :put "Adding IP address $ipAddr to big list of IPs to ignore.";
        }
        :set arrIpIgnoreList ($arrIpIgnoreList, [ :toip $ipAddr ]);
    }
    :if ($debugLevel > 1) do={
        :put "Ignore list now has $[ :len arrIgnoreIpBigList ] addresses.";
    }
    :foreach id in $arrIdList do={
        :set ignoreIp false;      
        :set ipProbeAddress [ :toip [ /ip firewall address get $id address ] ];
        :foreach ipAddr in=arrIpIgnoreList do={
           :if ($ipProbeAddress = $ipAddr) do={
               :if ($debugLevel > 1) do={
                   :put "Duplicate IP Address found in ignore list.";
                   :if ($debugLevel > 2) do={
                       :put ("\$ipProbeAddress: $[ :tostr $ipProbeAddress ] equals " . \
                       " $[ :tostr $ipAddr ]");
                   }
               }
               :set ignoreIp true;
           }
        }        
        :if ($ignoreIp = false) do={
            :set ipArrList ($ipArrList, $ipProbeAddress);
            :set arrIpIgnoreList ($arrIpIgnoreList, $ipProbeAddress); 
        }
    }
    :if ($debugLevel > 1) do={
        :put "Read $[ :len $ipArrList ] IP addresses awaiting look up.";
    }
# Clean up and remove from IN list
    :if ($blnRemoveOldList = true) do={
        :foreach id in $arrIdList do={
            /ip firewall address-list remove $id;
        } 
    }
}
# Next Part of program
:if ([ :len $ipArrList ] > 0) do={
    :if ($debugLevel > 1) do={
        :put "Starting whois lookup process for $[ :len $ipArrList ] IP addresses.";
    }
    :foreach ipAddr in $ipArrList do={
        :put "$progName: Now processing whois information for IP Address: $ipAddr";
        :log info ("$progName: Now processing whois information for IP Address: $ipAddr");
        /tool fetch url="http://$whoServer$uriPath$ipAddr$uriSuffix" mode=http dst-path=("/$outFile")
        :delay 1;
        :set content [/file get [/file find name=$outFile] contents];
        :if ([ :len $content ] < 1 ) do={
            :put "$progName: Error content file is empty or no content was fetched.";
            :log info ($progName . ": Error content file is empty or no content was fetched.");
        } else={
            :set lastEnd [ :find $content "NetType:" ];
            :if ($debugLevel > 2) do={
                :put "NetType is: $lastEnd.";
            }
            :set lineEnd [ :find $content $eolMark $lastEnd ];
            :if ($debugLevel > 2) do={
                :put "Found \$eolMark.";
            }
            :set line [ :pick $content $lastEnd $lineEnd ];
            :set blnFound false;
            :foreach registrar in=$arrNetReg do={
                :if ( !$blnFound ) do={
                    :if ( [ :find $line $registrar -1 ] != nil ) do={ 
                        :set blnFound true;
                        :put ("$progName: IP Address: $ipAddr is of Net type " . \
                        "$registrar added to address list $ipOutList."); 
                        :log info ("$progName: IP Address: $ipAddr is of Net type " . \
                        "$registrar added to address list $ipOutList."); 
                        /ip firewall address-list add list=$ipOutList address=$ipAddr comment="$curDate $curTime $registrar"
                    }
                }
            }
            /file remove $outFile
        }
    }
}

Re: A script to look up IP to RIR using ARIN WhoIs

Posted: Wed Jul 10, 2019 8:39 pm
by ultimus247
I like the scripting your doing. I have a project but do not have your coding skills would you be open to some freelance work to help me out with some Mikrotik automation?