Today I had a slow day so made a script which fetches all the current days Open Threat Exchange (OTX) Indicators of Compromise (IoC) IPv4 addresses and adds them to a firewall address-list (if they don’t already exist). This way you can automatically block the naughty IPv4 addresses where malware and exfiltration has been found by others.. I’d call it a free IPS for IPv4 :
The addresses are added as AVOTX_IndicatorAddresses so use that in firewall rules.
Open my “syncOTXIndicators_v3” script in a text editor, change the “PUT_YOUR_API_KEY_HERE” to your actual API Key
Install my “syncOTXIndicators_v3” script which you edited (/import syncOTXIndicators_v3)
Set up a scheduled task to run the script once per day (at 23:55 UTC for example, IoC’s resets at 0:00 UTC) NOTE! If you run it early in day (UTC time) there may not be any, or any all day! Uncomment line 12 to set custom start date
Add two raw firewall rules to drop & log all src and dst traffic using the “AVOTX_IndicatorAddresses” address-list
.
.
# Fetch IP indicators on OTX from past day to update local blacklist address-list
#
# PUT API KEY HERE:
:local otxApiKey "X-OTX-API-KEY:PUT_YOUR_API_KEY_HERE";
#
:local today [/system clock get date];
:local montonum {"jan"="01";"feb"="02";"mar"="03";"apr"="04";"may"="05";"jun"="06";"jul"="07";"aug"="08";"sep"="09";"oct"="10";"nov"="11";"dec"="12"};
:local otxDate ([:tostr [:pick $today 7 11]]."-".(:$montonum->[:pick $today 0 3])."-".[:tostr [:pick $today 4 6]]."T00:00:00+00:00");
# Uncomment below to set custom start date
#:set otxDate "2021-05-09T00:00:00+00:00"
#
# Ask OTX for all IP indicators
:local otxUrl ("https://otx.alienvault.com/api/v1/indicators/export?types=IPv4&modified_since=".$otxDate);
:log info "AVOTX: Syncing OTX indicators from date $otxDate";
:local otxResponce [:tostr [/tool fetch url=$otxUrl http-header-field=$otxApiKey mode=https output=user as-value]];
#
# Loop for addresses in OTX responce JSON
:local start ([:find $otxResponce "indicator\": "]+13);
:local end ([:find $otxResponce "\"type"]-3);
:local caughtIp [:pick $otxResponce $start $end];
/ip firewall address-list
:while ($end >0) do={
:if ([:len [find where address=$caughtIp and list="AVOTX_IndicatorAddresses"]] = 0) do={
add address=$caughtIp list="AVOTX_IndicatorAddresses";
:log info "AVOTX: Added $caughtIp to blacklist";
} else={ :log info "AVOTX: $caughtIp already in blacklist" }
:set start ([:find $otxResponce "indicator\": " $start]+13);
:set end ([:find $otxResponce "\"type\":" ($end+3)]-3);
:set caughtIp [:pick $otxResponce $start $end];
}
:log info "AVOTX: Sync completed";
.
. WARNING
The script is CPU intensive, it will max out a single core to 100% until it finishes, this will harm performance on single core or already high load routers for a few minutes.
The OTX lists refresh once per day, so use this late at night (UTC time!!) to get the full days list.
Over time the IPs will be ceased and possibly be returned to good-guy addresses, after a while if you start getting false-positives this may be why, investigate each log!
NOTES:
If you want to run the script with a custom start date, for example, 1 month behind to catch up; uncomment line 11 with your own UTC datetime. I’d run this script once with this line saying 1 month behind, then comment it out again.. it will take a while to parse.
Hello!
I think I followed the instructions, but the script doesn’t add addresses to the Address Lists.
There is no execution in the log:
:log info “AVOTX: Added $caughtIp to blacklist”;
:log info “AVOTX: $caughtIp already in blacklist”};
You must add the ** before the ? to the path in the editor.
The script works - it’s great because it is not limited to downloading entire lists, which cannot exceed 64k!
Hi, thank you very much for you suggestion! You have a cleaner method I agree. I will update my script and upload a newer version when I get home today.
Here is an example of the JSON responce from the API. The IPs are fake examples.
if you do not already have noticed the ~ instead of = is because the accepted value on address list can be ip (1.2.3.4) or ip-prefix (1.2.3.0/24)
no other RouterOS variable type contain “ip” (except v6)
ad search only “ip” is right, alternate, but right, ~ “^(ip|ip-prefix)$” is excessive, if for some reason IPv6 go on list, the regexp for parse IPv4 block it without errors.
I do not know if the returned .json contain or not prefix like 1.2.3.0/24
but if are contained inside, the script must be careful to not write only 1.2.3.0 cutting off /24
Hmmmm I will email OTX help and ask if there is ever a 1.2.3.4/24 style address and subnet, becuase if there never is we can cut out a few lines of code! The regex should then only find 1.2.3.4 style addresses and code like this will work nicely?
Edit: They will only ever be normal single IP addresses, never a CIDR range.
The last modification made by Faux works, but the last Rextended suggestion causes a loop if the address already exists (in my excample: AVOTX: xxx.xxx.xxx.xxx alredy in blacklist).
