I have a list of roughly 3,500 IP subnets that I want to whitelist for a specific connection on the MikroTik RB5009 router.
What is the recommended approach for this?
Should all subnets be added to IP → Firewall → Address Lists and then referenced from a single firewall rule?
Or is it possible to reference a file (for example a text file containing the subnets, uploaded under Files and accessed via the terminal) directly from a firewall or NAT rule?
I’m mainly concerned about manageability and performance with such a large number of subnets and the very long Address List this would create
Any best practices or real-world experience would be appreciated.
This is a common request. You are on the right track: address lists are the appropriate feature to use. Internally these are stored in hash tables, so lookup cost becomes almost independent of the number of addresses.
Unfortunately there is no built-in way for importing address lists directly from a file, however the script for doing it is only a few lines.
My preference is for the script to obtain the list via http (preferably https) and check for a new version weekly/daily/hourly.
The only issue with large address lists is when used on devices with small flash (the 16MB ones): if these entries are static, then they get written to flash and they may consume considerable amount of storage space. Solution is to create dynamic address list entries (with timeout set), but that comes with its own side effects: entries get lost with reboot (but you can schedule the script which updates them at boot time plus a short while so that interfaces get up&running) and they get eventually removed (longest timeout is around 32w) so you have to have mechanizm to update list periodically.
I had once list over 60K records (https://public-dns.info/nameservers.txt) on 16Mb device to block requests to common known DoH servers, along with some other manually created. As @mkx wrote this large list is fetched, parsed and inserted with script as dynamic records with long expiration (32w I think) and it’s was refreshed periodically with scheduler and on startup, all worked fine, but I dropped this list with some other reduced list with only common DoH servers just to speed up filter rule processing, even it is fast it was still resulting some latency because of not so powerful CPU.
Side issue with these large lists it populating them in WinBox, it can take time, especially with such large as was mine, also WinBox interface was a bit laggy when was Address Llst tab selected when there is no other address list selected in filter with smaller amount of records.
I guess blocklist for blocking inbound traffic, it is not the same as blocking outbound, especially HTTPS which is most used where rule gets triggered on each new HTTPS connection.
